Houdini caches missing instance "task" attribute (SG+deadline)

reecemulley · 3 October 2024 05:27

Hello!
We’ve had artist reports of farm errors on houdini .bgeo caches, with the Shotgrid addon’s Collected Instance Shotgrid Task failing.

Digging into metadata.json files, successful publishes from other apps have a "task": "Comp" (or whatever taskname) attribute on their instances. Manually adding this to the houdini job’s .json file in a text editor results in a successful publish.

Here’s the json for a random comp job

comp

{
    "comment": "publish tst",
    "deadline_publish_job_id": "66fdef2a324cb1908e04b643",
    "folderPath": "/101/sq001/sh010",
    "fps": 24.0,
    "frameEnd": 1092,
    "frameStart": 1009,
    "instances": [
        {
            "bakingNukeScripts": [
                {
                    "bakeRenderPath": "Y:/projects/ayon_dev/101/sq001/sh010/work/Comp/renders/nuke/renderCompMain/renderCompMain.baking.mov",
                    "bakeScriptPath": "Y:/projects/ayon_dev/101/sq001/sh010/work/Comp/renders/nuke/renderCompMain/renderCompMain.baking.nk",
                    "bakeWriteNodeName": "Write1"
                }
            ],
            "colorspace": "scene_linear",
            "comment": "publish tst",
            "deadline": {
                "auth": null,
                "require_authentication": false,
                "url": "http://10.0.4.5:8081",
                "verify": true
            },
            "extendFrames": null,
            "families": [
                "render",
                "review",
                "render2d"
            ],
            "folderPath": "/101/sq001/sh010",
            "fps": 24.0,
            "frameEnd": 1092,
            "frameEndHandle": 1100,
            "frameStart": 1009,
            "frameStartHandle": 1001,
            "handleEnd": 8,
            "handleStart": 8,
            "inputVersions": [],
            "jobBatchName": "",
            "multipartExr": false,
            "overrideExistingFrame": null,
            "pixelAspect": 1.0,
            "productName": "renderCompMain",
            "productType": "render",
            "representations": [ ... ],
            "resolutionHeight": 2160,
            "resolutionWidth": 3840,
            "source": "{root[work]}/AYON_dev/101/sq001/sh010/work/comp/ad_sh010_Comp_v123.nk",
            "stagingDir_persistent": true,
            "task": "Comp",
            "useSequenceForReview": false,
            "version": 123
        }
    ],

with the "task": "Comp" near the end
And here’s the first section of the metadata.json for a houdini bgeo job:

bgeo

{
    "comment": "",
    "deadline_publish_job_id": "66fe032a324cb1908e04b664",
    "folderPath": "/101/sq001/sh010",
    "fps": 24.0,
    "frameEnd": 1100,
    "frameStart": 1001,
    "instances": [
        {
            "comment": "",
            "deadline": {
                "auth": null,
                "require_authentication": false,
                "url": "http://10.0.4.5:8081",
                "verify": true
            },
            "extendFrames": null,
            "families": [
                "render",
                "pointcache"
            ],
            "family": "pointcache",
            "folderPath": "/101/sq001/sh010",
            "fps": 24.0,
            "frameEnd": 1100,
            "frameEndHandle": 1100,
            "frameStart": 1001,
            "frameStartHandle": 1001,
            "handleEnd": 0,
            "handleStart": 0,
            "inputVersions": [],
            "jobBatchName": "",
            "overrideExistingFrame": null,
            "productName": "pointcacheMain",
            "productType": "pointcache",
            "representations": [ ... ],
            "source": "{root[work]}/ayon_dev/101/sq001/sh010/work/FX/ad_sh010_FX_v007.hip",
            "stagingDir_persistent": false,
            "version": 3
        }
    ],

which has no task attribute.
How would I go about passing this attribute along to deadline?

I should note that we’re on Core 0.4.3, Deadline 0.3.0, Houdini 0.3.13 and Shotgrid 0.4.4, however this persists in other versions of all of these addons.

mustafa_jafar · 3 October 2024 07:39

Hey,
On my side ,

Render Jobs submitted from AYON Houdini includes "task" inside instance dictionary in "instances" list. Also, "AYON_TASK_NAME" in "Env" keyword inside "job".
Cache Jobs submitted from AYON Houdini only includes "AYON_TASK_NAME" in "Env" keyword inside "job".

And that env var should be propagated to the publish job.

Render Publish	Cache Publish
image.png1148×554 23.2 KB	image.png1148×554 23.2 KB

tbh, I’m not sure what has gone wrong on your side.
Could you

check the full json and confirm if "AYON_TASK_NAME" exists?
check the deadline job environment variables?
check that you have the latest deadline custom plugins?
publish the error message/deadline log of the failed job?

BigRoy · 3 October 2024 20:23

This is exactly the issue that @reecemulley describes - so that confirms this is a bug when combined with the shotgrid integration because it requires task in the instance on publishing. (Do note that technically that should also be fixed on the Shotgrid integration, because it is totally allowed to publish without a task) but in this case it’s a bug in Houdini submissions, because we did initially publish with the task set from the host integration.

So, what’s missing?

The new instance defined here should also contain task similar to is done here for render submissions.

But even better - they should both use the same ‘skeleton data’ and only deviate where needed so there’s less code duplication and we can rely on ‘more similar behaviors’.

reecemulley · 3 October 2024 23:43

hey! following up to your questions @mustafa_jafar - the AYON_TASK_NAME and associated env_var do exist, and with the latest custom plugins.
Here’s the job error-

Summary

=======================================================
Error
=======================================================
Error: Renderer returned non-zero error code, 1. Check the log for more information.
   at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type
=======================================================
RenderPluginException

=======================================================
Stack Trace
=======================================================
   at Deadline.Plugins.SandboxedPlugin.d(DeadlineMessage bgu, CancellationToken bgv)
   at Deadline.Plugins.SandboxedPlugin.RenderTask(Task task, CancellationToken cancellationToken)
   at Deadline.Slaves.SlaveRenderThread.c(TaskLogWriter ajz, CancellationToken aka)

=======================================================
Log
=======================================================
2024-10-04 09:07:27:  0: Loading Job's Plugin timeout is Disabled
2024-10-04 09:07:27:  0: SandboxedPlugin: Render Job As User disabled, running as current user 'administrator'
2024-10-04 09:07:29:  0: Executing plugin command of type 'Initialize Plugin'
2024-10-04 09:07:29:  0: INFO: Executing plugin script 'C:\ProgramData\Thinkbox\Deadline10\workers\rndr04\plugins\66ff20e5324cb1908e04b6ea\Ayon.py'
2024-10-04 09:07:29:  0: INFO: Plugin execution sandbox using Python version 3
2024-10-04 09:07:29:  0: INFO: Initializing process with AYON plugin 1.0.0
2024-10-04 09:07:29:  0: INFO: Single Frames Only: False
2024-10-04 09:07:29:  0: INFO: About: Ayon Plugin for Deadline
2024-10-04 09:07:29:  0: INFO: The job's environment will be merged with the current environment before rendering
2024-10-04 09:07:29:  0: Done executing plugin command of type 'Initialize Plugin'
2024-10-04 09:07:29:  0: Start Job timeout is disabled.
2024-10-04 09:07:29:  0: Task timeout is disabled.
2024-10-04 09:07:29:  0: Loaded job: Publish - pointcacheMain (66ff20e5324cb1908e04b6ea)
2024-10-04 09:07:29:  0: Executing plugin command of type 'Start Job'
2024-10-04 09:07:29:  0: DEBUG: S3BackedCache Client is not installed.
2024-10-04 09:07:29:  0: INFO: Executing global asset transfer preload script 'C:\ProgramData\Thinkbox\Deadline10\workers\rndr04\plugins\66ff20e5324cb1908e04b6ea\GlobalAssetTransferPreLoad.py'
2024-10-04 09:07:29:  0: INFO: Looking for legacy (pre-10.0.26) AWS Portal File Transfer...
2024-10-04 09:07:29:  0: INFO: Looking for legacy (pre-10.0.26) File Transfer controller in C:/Program Files/Thinkbox/S3BackedCache/bin/task.py...
2024-10-04 09:07:29:  0: INFO: Could not find legacy (pre-10.0.26) AWS Portal File Transfer.
2024-10-04 09:07:29:  0: INFO: Legacy (pre-10.0.26) AWS Portal File Transfer is not installed on the system.
2024-10-04 09:07:29:  0: INFO: Executing global job preload script 'C:\ProgramData\Thinkbox\Deadline10\workers\rndr04\plugins\66ff20e5324cb1908e04b6ea\GlobalJobPreLoad.py'
2024-10-04 09:07:29:  0: PYTHON: *** GlobalJobPreload 1.1.3 start ...
2024-10-04 09:07:29:  0: PYTHON: >>> Getting job ...
2024-10-04 09:07:29:  0: PYTHON: >>> Injecting render job id ...
2024-10-04 09:07:29:  0: PYTHON: >>> Dependency IDs: System.String[]
2024-10-04 09:07:29:  0: INFO: Setting Process Environment Variable RENDER_JOB_IDS to 66ff20e4324cb1908e04b6e9
2024-10-04 09:07:29:  0: PYTHON: >>> Injection end.
2024-10-04 09:07:29:  0: Done executing plugin command of type 'Start Job'
2024-10-04 09:07:29:  0: Plugin rendering frame(s): 0
2024-10-04 09:07:30:  0: Executing plugin command of type 'Render Task'
2024-10-04 09:07:30:  0: INFO: Stdout Redirection Enabled: True
2024-10-04 09:07:30:  0: INFO: Stdout Handling Enabled: True
2024-10-04 09:07:30:  0: INFO: Popup Handling Enabled: False
2024-10-04 09:07:30:  0: INFO: Using Process Tree: True
2024-10-04 09:07:30:  0: INFO: Hiding DOS Window: True
2024-10-04 09:07:30:  0: INFO: Creating New Console: False
2024-10-04 09:07:30:  0: INFO: Running as user: administrator
2024-10-04 09:07:30:  0: INFO: Executable: "Y:\Pipeline\Apps\Ynput\AYON 1.1.0\ayon.exe"
2024-10-04 09:07:30:  0: INFO: Argument: --headless publish "{root[work]}/ayon_dev/101/sq001/sh010/work/FX/pyblish/pointcacheMain_metadata.json" --targets deadline --targets farm
2024-10-04 09:07:30:  0: INFO: Full Command: "Y:\Pipeline\Apps\Ynput\AYON 1.1.0\ayon.exe" --headless publish "{root[work]}/ayon_dev/101/sq001/sh010/work/FX/pyblish/pointcacheMain_metadata.json" --targets deadline --targets farm
2024-10-04 09:07:30:  0: INFO: Startup Directory: "Y:\Pipeline\Apps\Ynput\AYON 1.1.0"
2024-10-04 09:07:30:  0: INFO: Process Priority: BelowNormal
2024-10-04 09:07:30:  0: INFO: Process Affinity: default
2024-10-04 09:07:30:  0: INFO: Process is now running
2024-10-04 09:07:33:  0: STDOUT: WARNING:pyblish.plugin:Path already registered: C:\Users\administrator\AppData\Local\Ynput\AYON\addons\applications_0.2.5\ayon_applications\plugins\publish
2024-10-04 09:07:33:  0: STDOUT: WARNING:pyblish.plugin:Path already registered: C:\Users\administrator\AppData\Local\Ynput\AYON\addons\shotgrid_0.4.5-FA-02\ayon_shotgrid\plugins\publish
2024-10-04 09:07:33:  0: STDOUT: >>> [ Setting OCIO environment to config path: Y:\pipeline\ocio\aces_1.3\config.ocio ] 
2024-10-04 09:07:33:  0: STDOUT: >>> [ Running publish ... ] 
2024-10-04 09:07:33:  0: STDOUT: ERROR:pyblish.plugin:Skipped: "extract_usd_layer_contributions" (No module named 'pxr')
2024-10-04 09:07:34:  0: STDOUT: ERROR:pyblish.plugin:Skipped: "extract_usd_layer_contributions" (No module named 'pxr')
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:*** AYON addons initialization.
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] applications (0.2.5)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] ayon_ocio (1.1.1)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] ayon_third_party (1.1.1)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] blender (0.2.3)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] deadline (0.3.1)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] equalizer (0.1.0-dev.1)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] houdini (0.3.13)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] launcher_tool (1.0.0)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] loader_tool (1.0.0)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] maya (0.2.13)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] nuke (0.2.4-dev-3)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] photoshop (0.2.2)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] python_interpreter (1.0.0)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] resolve (0.2.2)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] shotgrid (0.4.5-FA-02)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] substancepainter (0.2.1)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] traypublisher (0.2.4)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:[X] unreal (0.2.2)
2024-10-04 09:07:34:  0: STDOUT: DEBUG:AddonsManager:Has 18 enabled addons.
2024-10-04 09:07:34:  0: STDOUT: INFO:pyblish.CollectCurrentContext:Collected project context
2024-10-04 09:07:34:  0: STDOUT: Project: AYON_dev
2024-10-04 09:07:34:  0: STDOUT: Folder: /101/sq001/sh010
2024-10-04 09:07:34:  0: STDOUT: Task: FX
2024-10-04 09:07:34:  0: STDOUT: INFO:pyblish.CollectMachineName:Machine name: rndr04
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectSettings:Collecting settings for project: AYON_dev
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectAnatomyObject:Anatomy object collected for project "AYON_dev".
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectRenderedFiles:Getting root setting for project "AYON_dev"
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectRenderedFiles:Anatomy roots: {'dir': ayon_dev, 'work': Y:/projects, 'delivery_extension': dpx}
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectRenderedFiles:  - processing instance for pointcacheMain
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectRenderedFiles:Filling stagingDir with root to: Y:/projects/ayon_dev/101/sq001/sh010/work/FX/pyblish
2024-10-04 09:07:34:  0: STDOUT: DEBUG:GlobalServerAPI:Executing [GET] projects/AYON_dev
2024-10-04 09:07:34:  0: STDOUT: DEBUG:urllib3.connectionpool:http://--.-.-.-:4000 "GET /api/projects/AYON_dev HTTP/1.1" 200 11712
2024-10-04 09:07:34:  0: STDOUT: DEBUG:GlobalServerAPI:Response <RestApiResponse [200]>
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectContextEntities:Collected Project "{removed}"
2024-10-04 09:07:34:  0: STDOUT: DEBUG:urllib3.connectionpool:http://--.-.-.-:4000 "POST /graphql HTTP/1.1" 200 799
2024-10-04 09:07:34:  0: STDOUT: DEBUG:GlobalServerAPI:Response <RestApiResponse [200]>
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectContextEntities:Collected Folder "{'attrib': {'frameStart': 1009, 'description': None, 'pixelAspect': 1.0, 'fps': 24.0, 'clipOut': 1, 'shotgridId': '13240', 'shotgridType': 'Shot', 'resolutionWidth': 3840, 'endDate': None, 'resolutionHeight': 2160, 'clipIn': 1, 'startDate': None, 'handleEnd': 8, 'frameEnd': 1092, 'handleStart': 8, 'tools': []}, 'folderType': 'Shot', 'id': '2a0b068efd7511ee97110011328a44f8', 'label': None, 'status': 'Not ready', 'thumbnailId': '98b0da8481da11efbb650242ac130004', 'name': 'sh010', 'data': {'sg_ayon_id': '2a0b068efd7511ee97110011328a44f8', 'sg_ayon_sync_status': 'Synced'}, 'active': True, 'path': '/101/sq001/sh010', 'parentId': 'f1d16f9afc8b11eea5020011328a44f8', 'tags': []}"
2024-10-04 09:07:34:  0: STDOUT: DEBUG:urllib3.connectionpool:http://--.-.-.-:4000 "POST /graphql HTTP/1.1" 200 728
2024-10-04 09:07:34:  0: STDOUT: DEBUG:GlobalServerAPI:Response <RestApiResponse [200]>
2024-10-04 09:07:34:  0: STDOUT: DEBUG:pyblish.CollectContextEntities:Collected Task "{'assignees': [], 'attrib': {'frameStart': 1009, 'description': None, 'pixelAspect': 1.0, 'fps': 24.0, 'clipOut': 1, 'shotgridId': '49619', 'shotgridType': 'Task', 'resolutionWidth': 3840, 'endDate': None, 'resolutionHeight': 2160, 'clipIn': 1, 'startDate': None, 'handleEnd': 8, 'frameEnd': 1092, 'handleStart': 8, 'tools': []}, 'id': '2c278c94fd7511ee97110011328a44f8', 'taskType': 'FX', 'label': None, 'folderId': '2a0b068efd7511ee97110011328a44f8', 'status': 'Not ready', 'name': 'FX', 'data': {'sg_ayon_id': '2c278c94fd7511ee97110011328a44f8', 'sg_ayon_sync_status': 'Failed'}, 'active': True, 'tags': []}"
2024-10-04 09:07:34:  0: STDOUT: INFO:pyblish.CollectShotgridSession:User login: username
2024-10-04 09:07:34:  0: STDOUT: INFO:pyblish.CollectShotgridSession:Creating Shotgrid Session for user: username at https://f---------------.shotgunstudio.com
2024-10-04 09:07:34:  0: STDOUT: DEBUG:win32ctypes.core.cffi:Loaded cffi backend
2024-10-04 09:07:35:  0: STDOUT: INFO:pyblish.CollectShotgridSession:Succesfully logged in into the Shotgrid API.
2024-10-04 09:07:35:  0: STDOUT: INFO:pyblish.CollectShotgridSession:Shotgrid local storage enabled: False
2024-10-04 09:07:35:  0: STDOUT: DEBUG:GlobalServerAPI:Executing [GET] users/me
2024-10-04 09:07:35:  0: STDOUT: DEBUG:urllib3.connectionpool:http://--.-.-.-:4000 "GET /api/users/me HTTP/1.1" 200 1137
2024-10-04 09:07:35:  0: STDOUT: DEBUG:GlobalServerAPI:Response <RestApiResponse [200]>
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectCurrentAYONUser:Collected user "username"
2024-10-04 09:07:35:  0: STDOUT: DEBUG:GlobalServerAPI:Executing [GET] users/me
2024-10-04 09:07:35:  0: STDOUT: DEBUG:urllib3.connectionpool:http://--.-.-.-:4000 "GET /api/users/me HTTP/1.1" 200 1137
2024-10-04 09:07:35:  0: STDOUT: DEBUG:GlobalServerAPI:Response <RestApiResponse [200]>
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyContextData:Global Anatomy Context Data collected:
2024-10-04 09:07:35:  0: STDOUT: {
2024-10-04 09:07:35:  0: STDOUT:     "studio": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "Future Associate",
2024-10-04 09:07:35:  0: STDOUT:         "code": "FA"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "user": "username",
2024-10-04 09:07:35:  0: STDOUT:     "project": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "AYON_dev",
2024-10-04 09:07:35:  0: STDOUT:         "code": "ad"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "folder": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "sh010",
2024-10-04 09:07:35:  0: STDOUT:         "type": "Shot",
2024-10-04 09:07:35:  0: STDOUT:         "path": "/101/sq001/sh010"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "asset": "sh010",
2024-10-04 09:07:35:  0: STDOUT:     "hierarchy": "101/sq001",
2024-10-04 09:07:35:  0: STDOUT:     "parent": "sq001",
2024-10-04 09:07:35:  0: STDOUT:     "task": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "FX",
2024-10-04 09:07:35:  0: STDOUT:         "type": "FX",
2024-10-04 09:07:35:  0: STDOUT:         "short": "fx"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "app": "houdini",
2024-10-04 09:07:35:  0: STDOUT:     "d": "4",
2024-10-04 09:07:35:  0: STDOUT:     "dd": "04",
2024-10-04 09:07:35:  0: STDOUT:     "ddd": "Fri",
2024-10-04 09:07:35:  0: STDOUT:     "dddd": "Friday",
2024-10-04 09:07:35:  0: STDOUT:     "m": "10",
2024-10-04 09:07:35:  0: STDOUT:     "mm": "10",
2024-10-04 09:07:35:  0: STDOUT:     "mmm": "Oct",
2024-10-04 09:07:35:  0: STDOUT:     "mmmm": "October",
2024-10-04 09:07:35:  0: STDOUT:     "yy": "24",
2024-10-04 09:07:35:  0: STDOUT:     "yyyy": "2024",
2024-10-04 09:07:35:  0: STDOUT:     "H": "9",
2024-10-04 09:07:35:  0: STDOUT:     "HH": "09",
2024-10-04 09:07:35:  0: STDOUT:     "h": "9",
2024-10-04 09:07:35:  0: STDOUT:     "hh": "09",
2024-10-04 09:07:35:  0: STDOUT:     "ht": "AM",
2024-10-04 09:07:35:  0: STDOUT:     "M": "7",
2024-10-04 09:07:35:  0: STDOUT:     "MM": "07",
2024-10-04 09:07:35:  0: STDOUT:     "S": "34",
2024-10-04 09:07:35:  0: STDOUT:     "SS": "34",
2024-10-04 09:07:35:  0: STDOUT:     "username": "username"
2024-10-04 09:07:35:  0: STDOUT: }
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectContextLabel:Context label is changed to "Houdini - <shell>"
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Collecting anatomy data for all instances.
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Querying folder entities for instances.
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:All instances already had right folder entity.
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Querying task entities for instances.
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Querying task entities
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Querying latest versions for instances.
2024-10-04 09:07:35:  0: STDOUT: DEBUG:urllib3.connectionpool:http://--.-.-.-:4000 "POST /graphql HTTP/1.1" 200 420
2024-10-04 09:07:35:  0: STDOUT: DEBUG:GlobalServerAPI:Response <RestApiResponse [200]>
2024-10-04 09:07:35:  0: STDOUT: DEBUG:urllib3.connectionpool:http://--.-.-.-:4000 "POST /graphql HTTP/1.1" 200 237
2024-10-04 09:07:35:  0: STDOUT: DEBUG:GlobalServerAPI:Response <RestApiResponse [200]>
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Storing anatomy data to instance data.
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Anatomy data for instance pointcacheMain: {
2024-10-04 09:07:35:  0: STDOUT:     "studio": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "Future Associate",
2024-10-04 09:07:35:  0: STDOUT:         "code": "FA"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "user": "username",
2024-10-04 09:07:35:  0: STDOUT:     "project": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "AYON_dev",
2024-10-04 09:07:35:  0: STDOUT:         "code": "ad"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "folder": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "sh010",
2024-10-04 09:07:35:  0: STDOUT:         "type": "Shot",
2024-10-04 09:07:35:  0: STDOUT:         "path": "/101/sq001/sh010"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "asset": "sh010",
2024-10-04 09:07:35:  0: STDOUT:     "hierarchy": "101/sq001",
2024-10-04 09:07:35:  0: STDOUT:     "parent": "sq001",
2024-10-04 09:07:35:  0: STDOUT:     "task": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "FX",
2024-10-04 09:07:35:  0: STDOUT:         "type": "FX",
2024-10-04 09:07:35:  0: STDOUT:         "short": "fx"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "app": "houdini",
2024-10-04 09:07:35:  0: STDOUT:     "d": "4",
2024-10-04 09:07:35:  0: STDOUT:     "dd": "04",
2024-10-04 09:07:35:  0: STDOUT:     "ddd": "Fri",
2024-10-04 09:07:35:  0: STDOUT:     "dddd": "Friday",
2024-10-04 09:07:35:  0: STDOUT:     "m": "10",
2024-10-04 09:07:35:  0: STDOUT:     "mm": "10",
2024-10-04 09:07:35:  0: STDOUT:     "mmm": "Oct",
2024-10-04 09:07:35:  0: STDOUT:     "mmmm": "October",
2024-10-04 09:07:35:  0: STDOUT:     "yy": "24",
2024-10-04 09:07:35:  0: STDOUT:     "yyyy": "2024",
2024-10-04 09:07:35:  0: STDOUT:     "H": "9",
2024-10-04 09:07:35:  0: STDOUT:     "HH": "09",
2024-10-04 09:07:35:  0: STDOUT:     "h": "9",
2024-10-04 09:07:35:  0: STDOUT:     "hh": "09",
2024-10-04 09:07:35:  0: STDOUT:     "ht": "AM",
2024-10-04 09:07:35:  0: STDOUT:     "M": "7",
2024-10-04 09:07:35:  0: STDOUT:     "MM": "07",
2024-10-04 09:07:35:  0: STDOUT:     "S": "34",
2024-10-04 09:07:35:  0: STDOUT:     "SS": "34",
2024-10-04 09:07:35:  0: STDOUT:     "username": "username",
2024-10-04 09:07:35:  0: STDOUT:     "family": "pointcache",
2024-10-04 09:07:35:  0: STDOUT:     "subset": "pointcacheMain",
2024-10-04 09:07:35:  0: STDOUT:     "product": {
2024-10-04 09:07:35:  0: STDOUT:         "name": "pointcacheMain",
2024-10-04 09:07:35:  0: STDOUT:         "type": "pointcache"
2024-10-04 09:07:35:  0: STDOUT:     },
2024-10-04 09:07:35:  0: STDOUT:     "version": 3,
2024-10-04 09:07:35:  0: STDOUT:     "fps": 24.0
2024-10-04 09:07:35:  0: STDOUT: }
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectAnatomyInstanceData:Anatomy Data collection finished.
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectResourcesPath:publishDir: "Y:\projects\ayon_dev\101\sq001\sh010\publish\pointcache\pointcacheMain\v003"
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectResourcesPath:resourcesDir: "Y:\projects\ayon_dev\101\sq001\sh010\publish\pointcache\pointcacheMain\v003\resources"
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectSourceForSource:Parsing paths for {root[work]}/ayon_dev/101/sq001/sh010/work/FX/ad_sh010_FX_v008.hip
2024-10-04 09:07:35:  0: STDOUT: DEBUG:pyblish.CollectCustomStagingDir:Not adding custom staging dir for instance with 'pointcache'
2024-10-04 09:07:36:  0: STDOUT: INFO:pyblish.CollectShotgridEntities:Collected corresponding shotgrid project: {'type': 'Project', 'id': 2272}
2024-10-04 09:07:37:  0: STDOUT: ERROR:pyblish.plugin:Traceback (most recent call last):
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\pyblish\plugin.py", line 528, in __explicit_process
2024-10-04 09:07:37:  0: STDOUT:     runner(*args)
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\addons\shotgrid_0.4.5-FA-02\ayon_shotgrid\plugins\publish\collect_shotgrid_entities.py", line 43, in process
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\addons\shotgrid_0.4.5-FA-02\ayon_shotgrid\plugins\publish\collect_shotgrid_entities.py", line 117, in _get_sg_entities_by_id
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\addons\shotgrid_0.4.5-FA-02\ayon_shotgrid\plugins\publish\collect_shotgrid_entities.py", line 165, in _get_sg_tasks_by_id
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\shotgun_api3\shotgun.py", line 1057, in find
2024-10-04 09:07:37:  0: STDOUT:     result = self._call_rpc("read", params)
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3423, in _call_rpc
2024-10-04 09:07:37:  0: STDOUT:     self._response_errors(response)
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3740, in _response_errors
2024-10-04 09:07:37:  0: STDOUT:     raise Fault(sg_response.get("message", "Unknown Error"))
2024-10-04 09:07:37:  0: STDOUT: shotgun_api3.shotgun.Fault: API read() CRUD ERROR #10: Read failed for entity type [Task]
2024-10-04 09:07:37:  0: STDOUT: Traceback (most recent call last):
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\pyblish\plugin.py", line 528, in __explicit_process
2024-10-04 09:07:37:  0: STDOUT:     runner(*args)
2024-10-04 09:07:37:  0: STDOUT:   File "<string>", line 43, in process
2024-10-04 09:07:37:  0: STDOUT:   File "<string>", line 117, in _get_sg_entities_by_id
2024-10-04 09:07:37:  0: STDOUT:   File "<string>", line 165, in _get_sg_tasks_by_id
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\shotgun_api3\shotgun.py", line 1057, in find
2024-10-04 09:07:37:  0: STDOUT:     result = self._call_rpc("read", params)
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3423, in _call_rpc
2024-10-04 09:07:37:  0: STDOUT:     self._response_errors(response)
2024-10-04 09:07:37:  0: STDOUT:   File "C:\Users\administrator\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3740, in _response_errors
2024-10-04 09:07:37:  0: STDOUT:     raise Fault(sg_response.get("message", "Unknown Error"))
2024-10-04 09:07:37:  0: STDOUT: shotgun_api3.shotgun.Fault: API read() CRUD ERROR #10: Read failed for entity type [Task]
2024-10-04 09:07:37:  0: STDOUT: !!! ERR: 2024-10-04 09:07:36,895 >>> { CLI-publish }: [ Failed CollectShotgridEntities: API read() CRUD ERROR #10: Read failed for entity type [Task] -- ('C:\\Users\\administrator\\AppData\\Local\\Ynput\\AYON\\addons\\shotgrid_0.4.5-FA-02\\ayon_shotgrid\\plugins\\publish\\collect_shotgrid_entities.py', 3740, '_response_errors', 'raise Fault(sg_response.get("message", "Unknown Error"))') ]