Deadline Publish job pass along additional metadata for custom addon

General
,
1

Is there any way, without changing Deadline plug-ins to pass on extra metadata to the instance or representations that the SubmitPublishJob generates and hence the publish job on the farm also reads for the instances or representations.

I want to add some extra data to the instance or representations, (e.g. an additional tag or something else) and then with a plug-in respond to that on the publish job.

Like:

  1. User allows to pick a setting in publisher UI to post-process the render (either after local render, or on the farm)
  2. Their chosen option should be passed on to the metadata json so that the publish process on the farm can use it for its processing.

Preferably there would be some custom render data that the publish job would passing along, e.g. instance.data["renderMetadata"] = {} or some other reserved key that it always passes along so that other addons can put custom data in there without having to customize ayon-deadline itself.

Reference

Any ideas?

2

Env var to sidecar json file? Those will get picked up automatically by deadline, saved in the submission parameters and you can save it in deadline’s output dir to make it ok for farm work.

Then you do a collector checking for the env var, and if it’s there then you just load json and voila!

You probably already figured this out, kinda late sorry

3

Yup - that’s doable - but I didn’t want to mess with additional files; preferably I just wanted to send some data across that would find its way into the native metadata .json so I wouldn’t need to handle the cross platform paths and everything (plus have to deal with an additional file too).

Anyway, definitely a possibility - just wished there was a simple entry point to add some custom data to the publish job on the farm without needing to change plug-ins in ayon-deadline or ayon-core.

4

Well IIRC there is an instance_transfer dict in the submit_publish_job.py plugin that does exactly that. Key is family, value is list of parameters. Everything that matches gets transferred over.
Actually instance parameters get written to json by the plugin itself.

If you can’t modify that code…

  • monkey patch in some way that variable since it’s in the class (I don’t know if it’s doable in pyblish and it’s inheritance scheme)
  • do a post deadline plugin that opens the json coming from deadline and injects that data in the instance
  • do a json with envs as I suggested before
  • override an instance var that is there. I don’t know which one you can use tho.

Maybe monkeypatching could be your best bet if the plugin is loaded before executing, you just insert into instance_transfer your family and the instance props you want?

Anyways this is all on OP, don’t know if AYON changed stuff around

5

I just checked out ayon deadline repo and it’s still there in the ProcessSubmittedJobOnFarm class.
Luckily it’s class declared so you can try monkeypatching.

You can try { “RoyFarmStuff” : [ “my”, “fantastic”, parameters”] } and update the instance_transfer dict.

Then you just insert “RoyFarmStuff” in families of your instances.

That all depends on the state of the plugin in pyblish though, I don’t know enough of it to understand if its’s accessible at any point before execution…

6

@BigRoy, how did you solve this eventually?

Hope this is somehow related to your topic:
I’m currently struggling to get my productType forwarded to the publish job. create_skeleton_instance would always write “render” or “prerender” to the metadata.json.

Having some kind of “renderMetadata”, which would override the value before the json.dump, would help.

I’m sure there should be a way, that Deadline jobs publish something else than “render” or “prerender” while using the SubmitPublishJob plugin, but I couldnt’t find a way. Using “attachTo” in the Instance seems to be built for aovs only too.

7

It’s already there. You can check the generated json file when submitting to farm.
and in the publish jop you can find it extracted for example:

2025-09-22 14:43:47:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Anatomy data for instance renderGenericMainSlapComp_Beauty: {
.
.
2025-09-22 14:43:47:  0: STDOUT:     "product": {
2025-09-22 14:43:47:  0: STDOUT:         "name": "renderGenericMainSlapComp_Beauty",
2025-09-22 14:43:47:  0: STDOUT:         "type": "render"
2025-09-22 14:43:47:  0: STDOUT:     },

it works with instances with type render and review only.

8

I’m not sure if I got your explanation @mustafa_jafar. You mean it’s there with the instance_transfer attribute of ProcessSubmittedJobOnFarm?
By patching this, like @max.pareschi suggests too, I’m able to transfer other other product types than “render” and “prerender” to the Deadline publish job/ the json file.
This just feels a bit clunky and unintuitive though, as the plugin itself does not provide any functionality to modify instance_transfer. But this is the official way to adress it?

The lines of pyblish_functions I linked, show the part, where the “instance skeleton” is built, which then is stored in the generated json file. These lines ignore which product type the Instance had before. So everything published via the un-patched ProcessSubmittedJobOnFarm plugin ends up to be product type “render” (like in the extracted example of the publish job, you shared), as far as I understand.