I was contemplating moving my gaffer stuff into an addon since that’s the way of the future. But I started wondering if there is a recommended way (or simply, a way) to inject host names into core publishing plugins, e.g. modules/deadline/plugins/publish/submit_publish_job.py, since I use this plugin to publish the renders from gaffer and it feels redundant to duplicate it to put it inside the future gaffer addon.
In short, was wondering if there is a way to modify a plugins host list/families list from something like an addon.
As far as I know there currently isn’t really. I believe @iLLiCiT did put some thought in options for this but I don’t think there’s a ready to go answer currently.
Do note that eventually AYON Deadline will also move to its own ayon-deadline addon and like the other modules in core will get separated out of core.
I actually think it may make more sense to have some sort of farm family on instances, that when found - the relevant deadline plug-ins would know what to do. So instead of adding more families to the Deadline plug-ins it would instead be that other plug-ins can ‘subscribe’ to be relevant for farm publishing, preferably not a deadline specific family because we’d basically be getting into the same issues.
@BigRoy
Isn’t the standard process to add farm support as follows ? or Am I just confused ?
For Gaffer
implement submit_gaffer_deadline.py (in `modules/deadline/plugins/publish/)
using hosts = ["gaffer"] will mark it as a gaffer plugin even though it’s placed in deadline plugins.
And, then we can relay on families to decide if submitter should be triggered for particular instances.
Also, you’d be able to inject as much as environment variables as such job_info.EnvironmentKeyValue[key] = value
add some collector (in hosts\gaffer\plugins\publish)
That collector can include a simple toggle (bool attribute def) that add e.g. "farm" to families or what ever value used in families in the gaffer submitter.
Also, Find a way to ignore running the local render and publish.
Even if that’s the standard process - it would still mean needing to ‘tweak’ the Deadline addon (which is currently still part of Core) and thus that would basically answer @sjt’s answer as “yes, for now you need to modify deadline core code”.
The question is, can we build it so we don’t need to modify anything in the Deadline addon which will be greatly beneficial as we separate addons from each other.
I think I’ll just go ahead with moving my Gaffer stuff into an addon, ayon-gaffer and since most of the changes to the core stuff is just adding "gaffer" into host lists (well gafferNodes is there as well). It shouldn’t be something that can’t be in the core repo for the time being.
We’d just need to make sure that it doesn’t become to specific to ‘enlisting’ for Deadline-specific logic but maybe just for farm logic as described - but I do think that might be the ‘way forward’.
I believe this question was answered long time ago (that’s how the deadline addon was created!)
As in the current implementation we are required to implement the AbstractSubmitDeadline for each Host.
So, How could we refactor the deadline addon to have a general solution ?
maybe by having one class that expects some collected data per Host ?
Few notes about the topic. Ad. farm publishing “job creation” we have big holes in the logic.
First that came to my mind is that we should be able to define environment variables for jobs out of the plugins which are sending the job, especially because the environments are hardcoded and cannot be modified by addon based on it’s logic (speaking about ftrack, shotgrid etc.).
More related to this topic, From my point of view we should define a family (e.g. "deadline.publish") to be able to tell if publish job should be created and which farm plugin should be used for that (considering studio might have enabled both RR and deadline). Now the question is when and how the family is added to instance? Should deadline care about it? Or host integration? Or core?
BTW we should also separate plugins in deadline per host. They are all discovered now (maya plugin loaded in nuke etc.).