Shotgrid Integration addon

I will try update my OpenPype fork to ayon-core on the following weeks so I will try create PRs for the differences that I encounter to get this to work

EDIT:
Here it is the first PR to get this to work Fix AYON shotgrid username passed to Deadline by fabiaserra · Pull Request #379 · ynput/ayon-core · GitHub

I’m back. For some reason I get a CRUD error when deadline attempts to publish anything to SG. The full error is below. I have double checked both the Ayon and SG ID’s for the task (as I had assumed that was the error) but they match on both ends. Where should I be looking to solve this. Fyi these are tasks that were created in SG then synced to Ayon if that makes a difference and I havent manually changed anything, these are just as they come.

Error
DEBUG:pyblish.CollectResourcesPath:publishDir: "Y:\TestAyon\BJ0003_Ayon_SGCreated_Test\SGC\0010\publish\render\renderLayoutMantra_ropMain_beauty\v001"
DEBUG:pyblish.CollectResourcesPath:resourcesDir: "Y:\TestAyon\BJ0003_Ayon_SGCreated_Test\SGC\0010\publish\render\renderLayoutMantra_ropMain_beauty\v001\resources"
DEBUG:pyblish.CollectSourceForSource:Parsing paths for {root[work]}/BJ0003_Ayon_SGCreated_Test/SGC/0010/work/Layout/BJ0003_0010_Layout_v003.hip
DEBUG:pyblish.CollectCustomStagingDir:Not adding custom staging dir for instance with 'render'
INFO:pyblish.CollectShotgridEntities:Collected corresponding shotgrid project: {'type': 'Project', 'id': 519}
ERROR:pyblish.plugin:Traceback (most recent call last):
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
    runner(*args)
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\addons\shotgrid_0.3.2\ayon_shotgrid\plugins\publish\collect_shotgrid_entities.py", line 43, in process
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\addons\shotgrid_0.3.2\ayon_shotgrid\plugins\publish\collect_shotgrid_entities.py", line 117, in _get_sg_entities_by_id
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\addons\shotgrid_0.3.2\ayon_shotgrid\plugins\publish\collect_shotgrid_entities.py", line 165, in _get_sg_tasks_by_id
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\shotgun_api3\shotgun.py", line 1057, in find
    result = self._call_rpc("read", params)
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3423, in _call_rpc
    self._response_errors(response)
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3740, in _response_errors
    raise Fault(sg_response.get("message", "Unknown Error"))
shotgun_api3.shotgun.Fault: API read() CRUD ERROR #10: Read failed for entity type [Task]
Traceback (most recent call last):
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
    runner(*args)
  File "<string>", line 43, in process
  File "<string>", line 117, in _get_sg_entities_by_id
  File "<string>", line 165, in _get_sg_tasks_by_id
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\shotgun_api3\shotgun.py", line 1057, in find
    result = self._call_rpc("read", params)
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3423, in _call_rpc
    self._response_errors(response)
  File "C:\Users\deadline\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3740, in _response_errors
    raise Fault(sg_response.get("message", "Unknown Error"))
shotgun_api3.shotgun.Fault: API read() CRUD ERROR #10: Read failed for entity type [Task]
!!! ERR: 2024-04-16 09:51:38,815 >>> { CLI-publish }: [ Failed CollectShotgridEntities: API read() CRUD ERROR #10: Read failed for entity type [Task] -- ('C:\\Users\\deadline\\AppData\\Local\\Ynput\\AYON\\addons\\shotgrid_0.3.2\\ayon_shotgrid\\plugins\\publish\\collect_shotgrid_entities.py', 3740, '_response_errors', 'raise Fault(sg_response.get("message", "Unknown Error"))') ]

Hi there, Also running into this same issue when Deadline Attempts to publish to SG. Likewise AYON and Shotgrids ID task do match, and we are Syncing our SG to Ayon too. Seems like the error is replicable, unsure on what could be causing it however.

Glad to see others are affected to as it means I haven’t done anything too dumb hopefully. I haven’t tried it yet but I would like to try on a task that is created in Ayon first though I was weary of how well SG would handle that. Might need to consult @fabiaserra or @Minkiu as I am not aware of best practices when it comes to this sort of stuff.

I haven’t had a chance to test the addon fully from end to end yet as we are still running OpenPype in production and I think I will hold a while until we move to AYON given is not really stable yet. However, to debug this error you guys are running into I’d probably attach a debugger or add a few prints on this function ayon-shotgrid/client/ayon_shotgrid/plugins/publish/collect_shotgrid_entities.py at 2bdc5bb3a187c8fc4ac85f91d87ee387bff26430 · ynput/ayon-shotgrid · GitHub . At the end, the error is boiling down to this call, which is using the ShotGrid API:

        sg_tasks = sg_session.find(
            "Task",
            filters=filters,
            fields=["content", "entity"],
            filter_operator="any"
        )

so as long as you find what’s the content of filters, you should be able to replicate it on a console to try figure out what’s wrong with the filters or that call, perhaps it’s a corrupted task in your SG?

2 Likes

Cheers, thanks for reaching back. Agreed, I believe that there is a filter, incorrect task or something not syncing. Will give it a couple more tests in a few weeks when im back in an office.

1 Like

Ok I finally came across this same bug… I had raised this issue already in the OpenPype AYON branch a few months ago and I created a PR but I guess it fell into the void when Ynput migrated into ayon-core. Here it is again: Add `task` to skeleton instance passed to Deadline by fabiaserra · Pull Request #457 · ynput/ayon-core · GitHub

3 Likes

Hi here, some fixes had been included in new release for user publishing Release 0.4.1 · ynput/ayon-shotgrid · GitHub

Hey, testing out 0.4.1 - I can’t get the module to initialize :frowning:
0.4.0 and lower works just fine, but no luck with 0.4.1 and the current 0.4.2-dev

The console shows whether the module is loaded or not (0.4.1, the line where ‘ShotgridAddon’ should be is blank), but where can I check for errors?

Same issue over here… Somehow the addon seems not to be loaded for the launcher and no publishes to Flow are happening. While the leecher, processor and transmitter are running fine…
Btw. could somebody explain a bit more about the client login types? How exactly are the Environment variables working? And is the client login used for uploading the Versions to SG? Is it in that case based on the Login of the machine doing the publish or on the login from the artist machine that submitted the publish?

Thank you in advance

Hey all, updated to 0.4.1 using the bootstrap setup, it was just old dependencies causing my problems.
After tweaking sg_login_dialog.py and the credentials.py to support proxies properly, Ayon can now create versions on SG (at least when publishing locally), furthest I’ve ever gotten.
However I get a big scary error when uploading review MOVs

Summary
DEBUG: {'colorspaceData': {'colorspace': 'scene_linear',
                    'config': {'path': 'C:\\Users\\username\\AppData\\Local\\Ynput\\AYON\\addons\\ayon_ocio_1.1.1\\ayon_ocio\\configs\\OpenColorIOConfigs\\aces_1.2\\config.ocio',
                               'template': '{BUILTIN_OCIO_ROOT}/aces_1.2/config.ocio'}},
 'ext': 'exr',
 'files': ['renderCompMain.1001.exr',
           'renderCompMain.1002.exr',
           ...
           'renderCompMain.1479.exr',
           'renderCompMain.1480.exr'],
 'name': 'exr',
 'published_path': 'Y:/projects/AYON_dev/101/sq001/sh010/publish/render/renderCompMain/v041/ad_sh010_renderCompMain_v041.1001.exr',
 'stagingDir': 'Y:/projects/AYON_dev/101/sq001/sh010/work/Comp/renders/nuke/renderCompMain',
 'tags': []}
DEBUG: {'ext': 'mp4',
 'ffmpeg_cmd': 'C:\\Users\\username\\AppData\\Local\\Ynput\\AYON\\addons\\ayon_third_party_1.0.0\\ayon_third_party\\downloads\\ffmpeg\\bin\\ffmpeg.exe '
               '-apply_trc gamma22 -i '
               'Y:\\projects\\AYON_dev\\101\\sq001\\sh010\\work\\Comp\\renders\\nuke\\renderCompMain\\renderCompMain.baking.mov '
               '-pix_fmt yuv420p -crf 18 -intra -y '
               'Y:\\projects\\AYON_dev\\101\\sq001\\sh010\\work\\Comp\\renders\\nuke\\renderCompMain\\renderCompMain.baking_h264.mp4',
 'files': 'renderCompMain.baking_h264burnin.mp4',
 'fps': 25.0,
 'frameEnd': 1481,
 'frameEndFtrack': 1481,
 'frameStart': 1001,
 'frameStartFtrack': 1001,
 'name': 'h264_mov',
 'outputDef': {'bg_color': [0, 0, 0, 0.0],
               'burnins': [],
               'ext': 'mp4',
               'ffmpeg_args': {'audio_filters': [],
                               'input': ['-apply_trc gamma22'],
                               'output': ['-pix_fmt yuv420p',
                                          '-crf 18',
                                          '-intra'],
                               'video_filters': []},
               'filename_suffix': 'h264',
               'filter': {'custom_tags': [],
                          'families': ['render', 'review', 'ftrack'],
                          'product_names': [],
                          'single_frame_filter': 'multi_frame'},
               'height': 0,
               'letter_box': {'enabled': False,
                              'fill_color': [0, 0, 0, 1.0],
                              'line_color': [255, 0, 0, 1.0],
                              'line_thickness': 0,
                              'ratio': 0.0},
               'name': 'h264',
               'overscan_color': [0, 0, 0, 1.0],
               'overscan_crop': '',
               'scale_pixel_aspect': True,
               'tags': ['burnin', 'shotgridreview'],
               'width': 0},
 'outputName': 'h264',
 'published_path': 'Y:\\projects\\AYON_dev\\101\\sq001\\sh010\\publish\\render\\renderCompMain\\v041\\ad_sh010_renderCompMain_v041_h264.mp4',
 'resolutionHeight': 1080,
 'resolutionWidth': 1920,
 'stagingDir': 'Y:\\projects\\AYON_dev\\101\\sq001\\sh010\\work\\Comp\\renders\\nuke\\renderCompMain',
 'tags': ['baking', 'review', 'reformatted', 'burnin', 'shotgridreview']}
DEBUG: {'ext': 'jpg',
 'files': 'renderCompMain.baking_thumb.jpg',
 'name': 'thumbnail',
 'outputName': 'baking',
 'published_path': 'Y:\\projects\\AYON_dev\\101\\sq001\\sh010\\publish\\render\\renderCompMain\\v041\\ad_sh010_renderCompMain_v041_baking.jpg',
 'stagingDir': 'C:\\Users\\REECE~1.MUL\\AppData\\Local\\Temp\\pyblish_tmp_1va3dbzv',
 'tags': ['thumbnail'],
 'thumbnail': True}
INFO: Integrating Shotgrid version with code: sh010_renderCompMain_Comp_v041
INFO: Create Shotgrid version: {'id': 92481, 'project': {'id': 2272, 'name': 'AYON_dev', 'type': 'Project'}, 'entity': {'id': 13240, 'name': 'sh010', 'type': 'Shot'}, 'code': 'sh010_renderCompMain_Comp_v041', 'sg_task': {'id': 49573, 'name': 'Comp', 'type': 'Task'}, 'type': 'Version'}
INFO: Upload review: Y:\projects\AYON_dev\101\sq001\sh010\publish\render\renderCompMain\v041\ad_sh010_renderCompMain_v041_h264.mp4 for version shotgrid 92481
Traceback (most recent call last):
  File "C:\Program Files\Nuke15.0v1\python310.zip\urllib\request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "C:\Program Files\Nuke15.0v1\python310.zip\http\client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Program Files\Nuke15.0v1\python310.zip\http\client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Program Files\Nuke15.0v1\python310.zip\http\client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Program Files\Nuke15.0v1\python310.zip\http\client.py", line 1037, in _send_output
    self.send(msg)
  File "C:\Program Files\Nuke15.0v1\python310.zip\http\client.py", line 975, in send
    self.connect()
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\shotgun_api3\shotgun.py", line 4253, in connect
    self.sock = context.wrap_socket(self.sock)
  File "C:\Program Files\Nuke15.0v1\python310.zip\ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Program Files\Nuke15.0v1\python310.zip\ssl.py", line 1071, in _create
    self.do_handshake()
  File "C:\Program Files\Nuke15.0v1\python310.zip\ssl.py", line 1342, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
    runner(*args)
  File "C:\Users\username\AppData\Local\Ynput\AYON\addons\shotgrid_0.4.2-dev.5\ayon_shotgrid\plugins\publish\integrate_shotgrid_version.py", line 116, in process
    sg_session.upload(
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\shotgun_api3\shotgun.py", line 2460, in upload
    return self._upload_to_storage(entity_type, entity_id, path, field_name, display_name,
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\shotgun_api3\shotgun.py", line 2497, in _upload_to_storage
    self._upload_file_to_storage(path, upload_info["upload_url"])
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\shotgun_api3\shotgun.py", line 4008, in _upload_file_to_storage
    self._upload_data_to_storage(fd, content_type, file_size, storage_url)
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\shotgun_api3\shotgun.py", line 4106, in _upload_data_to_storage
    result = self._make_upload_request(request, opener)
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\shotgun_api3\shotgun.py", line 3626, in _make_upload_request
    result = opener.open(request)
  File "C:\Program Files\Nuke15.0v1\python310.zip\urllib\request.py", line 519, in open
    response = self._open(req, data)
  File "C:\Program Files\Nuke15.0v1\python310.zip\urllib\request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "C:\Program Files\Nuke15.0v1\python310.zip\urllib\request.py", line 496, in _call_chain
    result = func(*args)
  File "C:\Users\username\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403071252_windows.zip\dependencies\shotgun_api3\shotgun.py", line 4272, in https_open
    return self.do_open(self.create_https_connection, req)
  File "C:\Program Files\Nuke15.0v1\python310.zip\urllib\request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:997)>

My services don’t seem to be able to start at any higher addon version than 0.3.2, no idea if that has anything to do with it

Hey @reecemulley, it does look a bit off. Have you resolved that problem? Also, there’s a newer version out, 0.4.3, which might help if the issue persists. Check it out here: . I suggest using the Via Tray App with shared API key for client login—it could be the fix you need. More info here: https://ayon.ynput.io/docs/addon_flow_admin.

Hey @jakub.jezek , unfortunately my issue still happens in 0.4.3. I’m gonna keep digging. Big fan of Via Tray App with shared API key login method though

1 Like

Occasionally, we’ve encountered issues with outdated SSL certificates when systems are not current and the root OS certificate has lapsed.

Hey again, back on this SSL issue- We’ve just updated the certificates on client + render machines, they can ping our SG URL successfully, but no success with Ayon’s SG publishing. Just a quick clarification, with your previous issues @jakub.jezek , did you ever have to update the SSL certs on a particular docker image or its host machine?

quick followup, we solved this :grinning:

1 Like

Hey @reecemulley, thank you very much for the solving post. Yeah I will add this into our backlog and we will put it into settings as an option.

UPDATE:
following is the issue No SSL validation settings switch · Issue #138 · ynput/ayon-shotgrid · GitHub