Nuke Publish duplicating EXR files in two separate paths

[sergio]

Hello,

I ran my first Nuke Publish successfully, but there are two separate sets of EXR files in the project folder. The first one is in:

“X:\DOT_AYON\shots\DOT_AYON_102_046_0010\render\renderCompositingMain\v001\DOT_DOT_AYON_102_046_0010_renderCompositingMain_v001.####”

and the second in:

“X:\DOT_AYON\shots\DOT_AYON_102_046_0010\work\compositing\renders\nuke\renderCompositingMain\renderCompositingMain.1001”

This is not ideal as then every render would take up twice as much disk space as required. I will attach the project anatomy as well as a screenshot of the create write render plugin. Please ask if you need any other information to help solve this issue, and thank you for any help you can give.

Temporary rendering path template: {work}/renders/nuke/{product[name]}/{product[name]}.{frame}.{ext}

project_anatomy.JSON (9.4 KB)

[mustafa_jafar]

I believe this is the question was answered in another post. let me link the answer here as well.

Publishing from Nuke to AYON

This is related to how the publishing system work in AYON’s Pipeline. where publish process exports the files to an intermediate location called Staging Directory and then move it to the publish location Publish Directory as explained here. In the project anatomy there should be settings for staging and publish where you can specify templates for each of them. It’s also worth mentioning that some integration addons don’t support staging directory templates and in this case the staging direc…

[BigRoy]

I think that this reply fails to answer that on publish the publish job can automatically clean up the staging dir for you if you really want that enabled.

I believe it’s setting:

ayon+settings://core/publish/CollectRenderedFiles/remove_files

But it could also be any of these (not sure):

ayon+settings://core/publish/CleanUp/remove_temp_renders
ayon+settings://core/publish/CleanUpFarm/enabled

We may want to clarify in the settings there what is the exact difference between each of those toggles and to which cases they apply.

[mustafa_jafar]

@BigRoy

I think that this reply fails to answer that on publish the publish job can automatically clean up the staging dir for you if you really want that enabled.

I agree however, this might goes outside of the scope of this post.

I recall testing these plugins in my Houdini: Cleanup after publishing PR. Let me quote my test run:

From digging around discord and the code I found out that there are plugins that do the cleaning up:

• ayon-core/plugins/cleanup.py which remove the staging dir if it were in temp path
• ayon_core/plugins/publish/cleanup_explicit.py which removes in path in this list context.data["cleanupFullPaths"]
• ayon_core/plugins/publish/cleanup_farm.py which I’m not sure how to use it.

---

tbh, I find this feature dangerous and requires pipeline admins to be aware of how to use it.
I think these settings are related, I don’t know if they are working with Houdini :

• ayon+settings://core/tools/publish/custom_staging_dir_profiles
• ayon+settings://core/publish/CleanUp
• ayon+settings://core/publish/CleanUpFarm

[sergio]

Thank you for your responses. I do not have a “CollectRenderedFiles” as part of my core 1.3.2 addon, and if I update to a newer version I get an incompatible error. I am trying but AYON is not cleaning up the temp files when the publish is finished, so i end up with two different sets of EXR files. I believe the clean up is not happening but I do not know what I am configuring wrong.

[sergio]

sorry for the double post but I am still having trouble with the clean up. Chatgpt suggets that the clean up is not happening as my staging directory is not a OS temp directory, can anyone confirm this? Ideally I want the staging directory to be in the project folder which is in another drive, not C:.I also noticed that when I copied the “Nuke addon configuration” I saw a “patern” misspelled which might be causing AYON to not include them? Please any help you can provide would be great.

publish-report-250904-12-56.json (451.9 KB)

[BigRoy]

Pretty sure you need the newer ayon-core release for this to work. I believe 1.3.2 lacks this particular toggle.

Hmm - no. It should be this PR which is in ayon-core since 1.1.9. Nonetheless, worth updating to latest addon releases to ensure it may not have been fixed by a bugfix in the meantime.

[sergio]

but if I put a newer version of core addon I get this warning message saying I have it is incompatible with the server, should I ignore this warning?:

[BigRoy]

@sergio

it is incompatible with the server, should I ignore this warning?:

No - you should update your AYON server.