…Hi everyone. our Pipeline supe Raised the possibility of openpipe/Ayon as an evolution of our pipe at The Embassy vfx. I’ve been doing some reading and I see some potential here, and a great many hurdles. The primary one being that our current lighting and lookdev is done in Houdini and we’ll be moving to LOPs as our future workflow. Seems like we would be doing a lot of our own heavy lifting there. I’ll post a pair of follow up questions that I’d love thoughts on by long time users…
1: Our current pipeline uses 5 levels deep hierarchy for work files (well more as then we get into software specific folders). I’d like to know if that can be maintained in Ayon. Project Sequence Shot Step(anm ,lgt) and Artist.
That keeps all the mess and versions local to the artist container until they are ready to publish data up to the step level which then releases it for other users in the same shot. Can this logic be maintained? It allows multiple artists to keep their own versions and sandbox in the same step (say 2 animators, one working on the fg characters, one working on the bg ones) to maintain their own versioning. #does this concept map into Ayon?
2: we also use a concept of ‘make products’ then ‘check products’ then ‘publish to release’. Nothing enters the database as products that are expected to be used by downstream users until they have been vetted by the artist that generated them.( nearly all of which are submitted farm jobs, farm cooks the products). Then those files, already on disk are checked by the artist and then published(which creates a hard link to a folder outside of the artist folders, gives the product a new publish version folder and entry into the database). That way you aren’t moving data forward that you haven’t validated as correct (the idea being that if you haven’t seen the result, you can’t know you want to send it on). Can the work products in Ayon be written without publishing?
Assets (named Folders in Ayon) can best nested to any abitrary depth. So project, episodes, sequence, shots, subshots or whatever you come up with as a hierarchy is fine.
Step is what is Task is in OpenPype.
An artist folder for your workfiles is doable just fine I believe by including the {user} or another token in the workfile path*.
*Only potential downside is that the workfiles tool might not be able to show the workfiles of other artists than yourself since the artist folder would be dynamically based on your user or alike and would be used to format where to look for work files.
There isn’t a built-in “artist container” location that is enforced so some logic related to that might be missing. I do know some do use workfile folders per artist but I can’t comment myself on how well that works in OpenPype since I haven’t used it.
It allows multiple artists to keep their own versions and sandbox in the same step (say 2 animators, one working on the fg characters, one working on the bg ones)
Just want to note that in practice in OpenPype it makes more sense to separate this into a anim_fg and anim_bg task. This also makes it trivial to assign these steps/tasks individually in project management, etc.
Question 2
As far as I know, a lot of OpenPype isn’t designed around the way that your output is not to be published directly. So there isn’t a manual check products step.
I recall there being some production cases by OpenPype studios where they require sign off of products/publishes by a supervisor before being considered a correct publish which I suppose is exactly what you are describing too with the process of a human signing off before publish.
I don’t know however how that workflow works in practice with OpenPype. So I guess someone from @ynteam might have better knowledge on that topic.
(the idea being that if you haven’t seen the result, you can’t know you want to send it on).
I wanted to highlight that this is somewhat reduced of course by publish time validations which OpenPype does have. The idea being that most studio requirements are validated or ensured. The ‘artistic side’ of things - the purely subjective visual check - of course would be up to the user.
As @BigRoy mentioned. AYON is not designed this way at the moment, however as we’re digging deeper into USD workflow designs, it is apparent that we’ll have to introduce a concept of persistent staging (for a lack of better term right now) folder sooner rather than later.
Currently staging folder only acts as an intermediary between data being extracted from a DCC and it being published to DB. Really its just a slightly glorified cache for the time of publishing. That being said we’d like to give option to stage a publish, get most of the way there and resume/finish the publish once the data is validated/approved, whatever you might consider. This should, however, be non-versioned, until the data is actually published. We haven’t yet considered the implications of potentially “reserving” a version number for example.
There’s a lot to unpack there, but long story short. It’s not there out of the box at moment, but we’d like it to be in some capacity.
We could come around to ‘everything is published… but must be approved’ but then that kind of redefines the word publish. Publishing then is the staging logic, and human approval becomes what we now think of as publishing.
Can the loader/browsers filter only to approved versions? And can the approved versions be versioned independently? So bob’s v007 gets approved becoming ‘approved v001’ and then Jim’s v002 gets approved as the next version, becoming approved v002?
This is just trading the word approved for the word publish. At this point publish just means ‘made a thing to disk’ not ‘good for use’.
This publish/approval thing is a lot more important in lighting… where until you test the renders in your own comp we don’t want them released as you don’t quite know if say, moving that light, did what the supervisor needed it to do. (Same goes for long sims in the farm for fx)
If the default mechanism that render passes are submitted as published to the DB regardless of their output (say we forgot to include a light in the render pass) the all black frames are ‘automatically published’ as you don’t include a ‘staging area’? That seems kind of wild to me.
@MichaelBlackbourn I share the same requirements for our pipeline but I realized this would be a feature that we would be able to make use only once we migrate OpenPype to Ayon thanks to the Status being a first class citizen field of the publishes. Until then, we might come up with some other hacky workarounds on which version is the “approved” one. In my studio we haven’t yet used OP in production so I’m hoping that by the time this becomes a bigger need we will already be testing Ayon side to side.
Not sure if you have already checked the great presentation Milan did of Ayon a couple months ago https://www.youtube.com/watch?v=gacSfhnIxeU , somewhere around minute 24 he talks about statuses
We do that over here (not with AYON yet).
Everything is published, but not everything is approved. It took some time for the artists to adapt to the concept. Publishing does not mean you are done with your work, it just means you send it to the DB for someone else to look at.
The approval step can be done in two steps. The artist approves their own work, then a supervisor can approve it. It can even go through the departement lead too if desired.
Then, when loading, you load the latestApproved version or simply filter out the unapproved ones. What I like about this is when no version is approved, you just load the latest one, converting the pipeline to a push system. This is useful for small unimportant things. THen as soon as you approve something, it locks it to this version.
My two cents.
F
That would work. Most of the time our artists ‘self approve’ their work, either after a supervisor review early in the creative process, or self review later in the process when nothing major is changing and we’re just running tweaks from upstream users or fixing tech issues. But we never ‘auto publish’ as we always want artists to vet their work. It really does sound like ‘publish’ in Ayon as a concept would be better thought of as ‘registration into the DB’
New question. We have a concept of ‘build scene’ that works like this:
Set context in terminal
Build scene and set arguments for step/discipline. Which then (in the most used configuration) launches a farm job, when when picked up launches Houdini batch and loads the latest published+approved contents for that discipline, sets new render paths and launches render passes to the farm.
So the demon monster anim has been updated? Build scene flagged to dynamics and farm jobs cooking the new falling sparks and ashe sims go to the farm, including dependencies to exr render jobs after the geo completes.
Or the lookdev of a an aircraft changed that’s in 10 shots, and three of those shots also have updated cameras? Run build scene in lgt mode for every shot using those planes and with the updated cameras and stack up the farm with Houdini batch jobs that each open a new scene , loads contents including published and approved lighting and rop hda’s , sets render paths and frame ranges, and then launches all the archive and exr jobs to the farm.
I’m assuming this is all possible in the op/Ayon framework with a little scripting?
That’s correct, that’s how I think of it as well, a publish is just registering into the DB (artists shouldn’t be ashamed or worried about registering versions into the DB) and “approve” is just setting a status on a specific version. Later when using Ayon I’d like to use another status which is exclusive only on a single version of the subset/product, something like “preferred” or “starred” so that version is the best of all that should be prioritized over “approved”.
However, @flord, how are you currently “approving” versions through OP?
Yeah I would hope all of that’s posible once you adopt the full OP framework with some scripting. Re: the Houdini automatic scene creation we should be able to look into the “Workfile template” framework where you can set your own scene templates with “placeholders” that have logic on dynamically loading the subsets based on any logic (ie. latest approved). I was just doing a quick search for the template workflow in the Houdini OP integration though and I don’t see it so perhaps we would need to implement that (I’m about to start looking at better support for the CG pipeline in my studio so I might be interested in looking into that as well soon).
You touched also the concept of dependencies so we can control what things get triggered automatically. I feel like this is another one where we will need Ayon’s backend to be able to set incoming/outcoming links between products…