Hi, thanks @BigRoy for drawing my attention to this topic, it is really interesting!
The cleanest pipeline terminology and concepts that I have encountered were in the Rhythm & Hues pipeline, which was really unique. One of the original developers, Josh Tomlinson, also wrote an open-source implementation, which you can find on GitHub: GitHub - Clemson-DPA/dpa-pipe: DPA pipeline (worth checking out the DPA video and also the original R&H Siggraph video). When we wrote our own production management system (Limes), we implemented this approach and consulted with Josh. As far as I know, the idea and term of “representation” originated in the Rhythm & Hues pipeline.
Based on this experience, I am with @milan , I think Folder[type] , Task[type]' , Product[type] is a really clean approach, R&H pipeline has something very similar. You can grasp it immediately - and it is really important as some of you already discussed.
We’ve been pondering this again and again, internally and publicly and at the moment we’re leaning quite strongly towards renaming subset and family into product and product[type] with 1.0 AYON release.
We need to examine how distruptive this would be and how much it might affect the codebase and will report back here.
Just to be clear though, we’re not talking about touching pyblish families. This is about the datamodel once something is published.
Having that distinction between pyblish families and resulting product types I think could be great - way more explicit in the long run that a pyblish family isn’t necessarily the resulting producting type.
Thanks for tagging me on this. I had admittedly already read through this discussion when I started learning about OP and didn’t dare to comment because I was still very confused with the terminology, which I guess reiterates the importance of the discussion.
Just watched the video @gabor.marinov linked and I really resonated with a lot of the concepts they brought up and it also felt very familiar with the pipelines I experienced at MPC Film, MPC Advertising and The Mill.
Please watch the video if you haven’t yet and later you can continue reading my post. A few highlights that I think are relevant to this discussion and a mix of others that struck me as “I’m not sure if this exists already in OP/Ayon” and “this would be really cool to have”:
Custom definition of PTasks. This includes OP’s Tasks (they call it Step) and also Workfiles and they can be versioned at the leaf level. I love how this greatly simplifies and abstracts the pipeline to any customization you’d want, there’s no opinion on Tasks like OP has - there’s really no distinction between what a Task is versus a Folder and you can do whatever hierarchy makes more sense to your studio, at any level. What benefit do we get on separating Task and Folder?
Product “categories” (products are categorized). Another term to consider, instead of Product[type], Product[category]. I don’t have a strong opinion of category over type but I like how it fits the semantic more.
URIs syntax are VERY powerful!! I can’t stress enough how useful these are and how relevant they will be once we adopt a USD pipeline and we need to write an asset resolver. On the other hand, these are very easy to read by artists on any UI and abstracts away their constant need of using paths for every asset loading (i.e., load the latest approved imgseq product on the specific context: “testProject:prod:AB:001:light:hero:products:TigerDiffuse:imgseq:approved”, URI of a representation “testProject:prod:AB:001:light:hero:products:TigerDiffuse:imgseq:0017:exr:1920x1080”…). Do we have these in Ayon?
Product versions can’t be consumed by downstream departments until they are “published”. This allows to consume the versions of the products on other DCCs before deciding what to publish.
The term “publish” in OP is something that’s confused me a little bit and not sure if it’s making it clear to the artist what it means and maybe it should be more clear in Ayon moving forward, specially with the introduction of statuses. In my mind, every version that goes into the DB is “registered” and a “publish” happens when you explicitly choose to do so. Having said this, I could get used to using “publish” instead of the implicit “register” step of every version but then we need to be clear on the terminology of the different statuses and how they get authored for the downstream of data.
Subscriptions: a snapshot of dependency upstream versions. I think this is another key workflow that is very important when implementing a pull data pipeline (very common in USD pipelines) and you need a way to bake your versions when submitting tasks to the farm
Hierarchical config system that can override anything on any of the PTask levels. Does Ayon settings allow us to set them per Folder (i.e., sequence, shot)?
Actions command line that exposes all the registered actions. I know that a lot of these should be easily achieved with the OP/Ayon framework but from my OP experience it didn’t strike me as it’s trying to be as command line friendly for power users (i.e., can we “publish” assets through CLI? do we have good CLI for doing queries?)
** Final thoughts **
In practical terms, without trying to disturb much the current efforts… I think that the renaming of subset and family to product and product[type] is really spot on and makes a lot of sense and I’m giving a big thumbs up from my side!
I’m still just asking questions about Ayon. But much of @fabiaserra comment resonates with how our current pipeline works. We really would need a robust, user driven ‘release’ mechanism for output products that is different from just registering in the DB. With versioning based on released versions and filtering in all loaders to only view released.
Also family and subset don’t resonate with me at all and if I continue to learn more about Ayon I was bracing myself to have a sticky note mapping the names to our current name logic on the side of my monitor forever.
I’m not sure yet. I’m still wrapping my brain around how the logic of everything works here. I’ll try to do this later today though and see what I come up with.
I had the same feeling as you @MichaelBlackbourn when I started reading about OP trying to do the mapping of my brain terminology but after 4 months in OP I think have connected some of my few neurons with these terms haha
On my case it was in general lines something like:
asset type - > family asset - > subset asset version → version component → representation context → asset
steps and tasks are different concepts in Shotgrid so the mapping from Shotgrid to OpenPype makes some of these sometimes hard to follow: Help
FYI I’m as new to Shotgrid as to OpenPype so for me this distinction of Steps and Tasks is still quite confusing and I’m trying to get into good terms with it, although I can see some benefits on having two different concepts of it. So far I have come up to think of Steps as just the different disciplines required on the whole pipeline of a show (i.e., fx, comp, layout, animation…) and tasks as the very specific things artists do on each of those disciplines (i.e., water sim, background crowd…)
I admittedly have no experience whatsoever with OP and Ayon.
Maybe it makes my opinion irrelevant, maybe it brings an interesting salt to the conversation ?
I wouldn’t know, and I’m happy to let you Ayoners decide ^^
But I couldn’t miss a chance to celebrate the mention of Clemson DPA Pipeline ! (thanks for the opportunity Gàbor ^^)
It had a big influence in my career, and I’ve been using it as a base, reference, documentation in many educational sessions, talks, brainstorms, etc…
Som as you would have guessed after reading Fabià’s detailed post, I’d vote for product and product type.
Product
I had the opportunity to design a handful of variations of the Build>Edit>Export loop, trying to ingest influences from other industries’ ETL/ELT loops, etc…
From the discussions, presentations and tutoring I conducted, I’ve learned that the commun idiomatic ground for everyone (techs and talents) is:
“You create and refine products for others to consume”
This usually hits and it became my go-to introduction to kickoff most collaborations.
People understand that products are produced by someone/something, and are meant to be consumed by someone/something else
Product is a good term for the creative loop we’re fostering with our pipelines.
Product Type
I have a personal preference for category because type, to me, is either a class or a style for text ^^
But most importantly: it looks like product type fits Ayon’s terminology quite perfectly.
When things line up, there’s no point in fighting it
Hope this can help.
Feel free to discard, or even better: to argue against !
Yes, step and discipline are the same to us. We don’t often have clearly defined tasks and unless we need to draw clear lines (mostly for production to try and track progress) we assume whomever is assigned to the step will handle everything needed in that step/discipline for a shot.
I also vote for product and product Type for consistency.
As for tasks vs pipeline steps, I worked in a studio where they were using Shotgrid and they used to create multiple tasks for the same output. For example, one animator would have three tasks for the animation of a character (blocking, normal and final) even though the files on disk were all in one place. Because of that, we ignored the tasks completely in the pipeline and only relied on the pipeline step, which roughly translates to the department.
F
Exactly how we handle it. Sometimes production wants to track the smashed glass dynamics seperstely from the projectile air trails. But we don’t bother representing that in the pipeline other than two different hda’s get published+approved to the shot dyn step which then makes them available to the lgt step as inputs. But artists are free to also make smashed glass fast, and air trail zippy or whatever they want. The creation of these ‘fx elements’ is up to the artists working together and their leads. No one has to go create containers in the pipeline for them to use. But we encourage consistent naming. This is also how we handle renderpasses to comp. need a special shadow pass, just make it, name it shadowsoft or whatever and publish+approve it to send it to comp. no one had to make entries that fit this stuff. It’s between the creative people solving problems to make as much of this as needed.
You can do all that already. In AYON pipeline step| effectively translates to task type, which is a usable token in the project anatomy.
So it’s really up to you.
aah poteto potato . We considered category in various places as well actually, but for consistency, type seems better fit exactly as you say. folder.type product.type task.type link.type
It just rolls of the tongue.
