I’m getting a bit stuck with the new publisher on refactoring e.g. the Maya workflow for rendering or implementing the Substance Painter integration. The issue I have is that I seem to be dealing with the concept of an instance or export that will result in multiple subsets of which I might or might not want to show all instances in the UI.
In Maya render we always had a single renderglobalsDefault or renderingMain instance or something. Upon publishing this would split into separate instances (one per renderlayer) which is what the user would be seeing in the publishing UI which they could then turn on/off for publishing as they pleased.
With the new UI I’m not sure how to do that.
I want a single creator creating that single instance node? But still want the single creator to only when ever created result in multiple instances visually in the UI? Almost sounds like an AutoCreator creating multiple instances ONLY when a specific node exists in the scene, but then how does the user create that node from the publisher?
If I were to implement instance settings like in the screenshot above and there were multiple renderlayers then magically they wouldn’t all have their own “set” of settings so toggling an option for ONE of the renderlayers would technically be toggling it for all of the renderlayers because it’s on the same single instance node in the scene.
I have a feeling somehow that we’d actually need to start separating this explicitly into the instances per layer as publish set to, like is already somewhat done in the outliner:
And potentially thus moving the settings to those individual layers? However that logic currently maintaining those renderlayer sets isn’t too stable yet for that to be reliable enough? E.g. delete a layer, undo… and voila they are not in sync anymore:
The substance painter integration issue is slightly different because in that case I CAN have multiple top instances but each of those instances would split into a multiple texture subsets.
We can decide to either:
keep it visually like a single texture pack you’d export in publish UI (less instances user toggles, etc.)
or per texture pack show all subset instances, showing each individual UDIM texture sequence it will output (this output however is dependent on the “export preset” chosen for the instance.
In this case I think option 1) is visually the best case. However, not sure how we could then per instance, based on the export preset chosen let the user choose WHICH channels/maps of the texture set to export.
Say PBR Metallic Roughness exports pack (simplified):
Diffuse
Metallic
Roughness
Now the user might want to toggle one of those three off for the export? Maybe?
References
Related topics:
Substance Painter Integration #4283
Maya Refactor to new Publisher #4388
Let me know if anything is a bit unclear - was a bit in a rush while I typed this up.
Ad. multiple instances. I think that similar thing uses editorial creator. There is not way how to hide instance. But e.g. editorial did it a way that the UI creator actually does not create any instance but based on data from edl is triggering other creators to create instances with information they need so the instances have attribute definitions they need too.
(HiddenCreator can be used for that) src
@iLLiCiTiT mentioned that on discord - it would be great if additional documentation, links or references to that implementation could be shared!
The creation system gives multiple options.
Few fact:
Creator can create 0-n instances during create
Creator has access to any other creator or already created instance
There are different types of creator plugin Creator, AutoCreator and HiddenCreator
So single creator may create multiple subsets during single create, but all will have the same family and attribute definitions. Only way how to change family or attribute definitions is to implement helper HiddenCreator which will care about it. Hidden creators are not shown in UI but can be accessed by identifier. Possible issue would be that the instances can’t be dependent on each other or information from one instances does not match the other (must be handled by validator).
Thanks - that helps but admittedly is also a massive bulk of code so it’s a bit hard to parse what the approach would be.
The way I understand what I need is:
Create a ‘regular’ Creator which creates an instance in the scene, however allow it to only ever create one instance and do NOT return the instance nor add it to the current context so that it never visually appears in the UI. Then have a renderlayer Creator which manages the data for each individual renderlayer in the background (automatically, and not ‘created’ by the user through the UI).
Two Creators?
So, two Creators:
CreateRender: Create a node or setting in the scene to define whether renderlayers should be collected or not.
CreateRenderLayer: If the “CreateRender” node exists then this should collect the individual renderlayers (and potentially store any related data of them to the scene).
The CreateRender would only ever be visible in the Create tab and would itself never list any instances - it’s just there in the scene to notify that Renderlayers should be collected or not. The CreateRenderLayer is a (hidden?) Creator which detects whether a CreateRender instance exists - if it does, then it starts to collect for the UI the individual renderlayer instances. This Creator should NEVER explicitly need create triggered since it’s based of off the Renderlayers in the scene.
As I think HiddenCreator doesn’t automatically trigger to collect the renderlayers the next time when the CreateRender node exists I have the feeling that what I’m needing here is an AutoCreator. A hidden creator wouldn’t allow to do that? Because, the way I see it:
The HiddenCreator is just a regular Creator but not visible in the UI. So it needs to be triggered for it to create? Which I could ‘do’ on create of another visible Creator (like a main “Render Creator”), but technically if the user would manually delete all “renderlayer” instances but the parent global ‘render’ instance still exists I’d still want to collect the renderlayers which the Renderlayer Creator then wouldn’t detect. The same goes for if the user had deleted all renderlayers, and created new ones. I’d want to creator to still detect and create the relevant instances.
The AutoCreator ALWAYS gets its ‘Create’ triggered on Reset.
Whenever the user creates a new renderlayer the RenderlayerCreator should detect on collect that now it should have an extra instance to store data to or even remove an instance if a relevant renderlayer for the instance does not exist anymore.
Save on reset and scene changing on reset - that shouldn’t happen!
Also, now reading what I’m writing here I actually have a bad feeling about “AutoCreator” ‘creating’ on reset too. I think the intent would have been way better if the AutoCreator would persist to the scene only once the user performs the first save/publish - not solely on “reset”. In that way, yes - an AutoCreator does show the instance on first launch of the publisher tool but it only stores its data in the scene once the user initiates the publish or explicitly confirms to save any data.
I feel like “Reset” should never change the active scene - not even from an AutoCreator. Thoughts @iLLiCiTiT@mkolar ?
So I have played a bit with a Creator to create the Render instance which ends up just being a node in the scene to define whether a different HiddenCreator Renderlayer instance should actually collect any Renderlayers for publishing but I’m pretty sure this is not how it should work.
It’s extremely confusing to both the artist and the developer
“Render” shows up in Create. Upon create nothing happens because the other HiddenCreator of course doesn’t collect directly. This could be solved by having the CreateRender creator directly perform create of the HiddenCreator - but I just couldn’t figure out how to make that work sensibly.
there is no “Render” instance in the UI so you cannot delete it after, so it’s unclear in the UI how you can later disable Renderlayers from being collected. Before the new publisher this was less of a problem because none of the creators had that… but now suddenly only this one you can’t clean up from withingl the UI
there are RenderLayer instances in the UI per RenderLayer but you can’t really delete them from within the UI. I could make it so that delete would delete the physical RenderLayer but I am pretty sure that’s not what the user wants?
the UI allows to change variant, asset and task of these renderlayer instances but they just have nothing to do with it really?
I still have no clue what the way would be to correctly name the subset for RenderLayer instances? There’s not really any correct pipeline way to do it, yes? Like there are no subset templates for that or? I have no idea.
I need to think about this more. I think maybe it just makes more sense to have an auto creator always list all Renderlayers for rendering no matter if a certain node exists or not? But then any Renderlayers from a scene would always appear in the publish window. Maybe that’s a bit too much?
Admittedly I’m close to a point where I’d rather go back to the old publishing system. In my test runs of creating and publishing I was able to go through the old publishing workflow quicker, with clearer visual feedback on what was actually getting processed from my scene (and also what actually finished!) and if an error occurred I honestly believe the old way was much better. Yes, for newcomers it’s nice to get a thorough documentation and report about a certain validation that failed… But if you are publishing 15 models a day you really don’t care about a lengthy explanation of the normals being invalid. You care about seeing which instance, which model, select it rapidly and fixing it. And if a validation didn’t pass that I haven’t seen before then and only then I would care about getting a lengthy report on how to solve this issue. The new system feels like it might help someone who publishes a few models in a system they are new to but it’ll just be much of a slowdown to anyone who has worked with the system for years.
Just selecting the invalid nodes for five failed validations individually took me 40 seconds with the new system on four instances. With the old it took me about 6 seconds (with the potential downside that I couldn’t filter the selection per instance but in practice I didn’t really care too much. If it didn’t pass a UV validation I just quickly want to see which meshes didn’t pass - not necessarily which meshes per which instance)
I think there are features the new system brings are very welcome, like managing creators and the settings too. The still lacking stability of the UI is something still forgivable too but the UX of the actual publishing itself to me is actually a step back.
Also wanted to note that most creators can work with your selection “Use selection” and add the currently selected nodes or things into the instance but the Publisher UI being this massive I often find myself spending a lot of time moving the UI to the corner of my screen, scaling it down, checking my scene selection, scaling/moving the UI back and the clicking create. It feels like the UI is about 75% of my screen space by default, I even have the tendency to make the window even bigger to visually parse what all the data tells me on the pages because everything is super big. The old create UI was so small that it didn’t have this issue.
Gave it another go at refactoring Maya’s CreateRender to the new publisher - quite the same as before and very unhappy with the result. Here’s the draft commit in #4388 and probably best readable on Github like this.
The functionality is really odd - both in code to maintain and also in the scene. Also the subset names for the actual renderlayers are currently arbitrary and the context for the renders based on legacy io session. I’m quite stuck and literally have no idea how to continue with this.
However I felt it was best to just push the commits I’ve made to show at least what I’ve tried and felt was the completely wrong approach in the end. Would love to push/urge someone else with more knowledge of the publishing system to refactor this code to how it should be done, or at least provide a feature-complete example of it - since I’m pretty stuck with this.
Also, funnily enough this is actually a case where as a user I’d rather have the order of the instances sorted by the order the renderlayers are visible in the Render Setup layer manager than sorted by name. E.g. it’s very odd to see the renderHello above the renderMain to me personally - but maybe the old publishing did that too.
I’ve moved some of the settings for Deadline submissions to the Deadline plug-ins as can be seen from the screenshots with the Primary Pool, Secondary Pool and Publish Job State.
Would be great to review #4388 since yes, I’m stuck with this not-so-very-nice-implementation. Would love to get input on how I should do this differently, preferably simplifying the workflow.
Please help! 