Deadline Submit Error (Broken response from DL)

Hi!

One last big hurdle before fully operational is submitting to Deadline. The publish process errors at the “Submit Nuke to Deadline” step.

image

I have followed this guide and checked the forum and the internet for similar issues, but no solution yet.

The Deadline Web service is running correctly and I can connect to it in the browser, but somehow when submitting from the publish menu, it gets a broken response [200].

Here’s the whole error (I’ve hidden the project name and the IP address).

INFO: Using U:\projectname\sequences\kbr\kbr_0010\publish\workfile\workfile2d_prep\v003\KBR_kbr_0010_workfile2d_prep_v003.nk for render/export.
DEBUG: Starting new HTTP connection (1): ip:8081
DEBUG: http://ip:8081 "POST /api/jobs HTTP/1.1" 200 355
WARNING: Broken response <Response [200]>. Try restarting the Deadline Webservice.
Traceback (most recent call last):
  File "C:\Program Files\Ynput\AYON 1.1.1\dependencies\requests\models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Program Files\Nuke15.1v3\python310.zip\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\Nuke15.1v3\python310.zip\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\Nuke15.1v3\python310.zip\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Ynput\AYON\addons\deadline_0.5.2\ayon_deadline\abstract_submit_deadline.py", line 344, in submit
    result = response.json()
  File "C:\Program Files\Ynput\AYON 1.1.1\dependencies\requests\models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Traceback (most recent call last):
  File "C:\Program Files\Ynput\AYON 1.1.1\dependencies\requests\models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Program Files\Nuke15.1v3\python310.zip\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\Nuke15.1v3\python310.zip\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\Nuke15.1v3\python310.zip\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Ynput\AYON\addons\deadline_0.5.2\ayon_deadline\abstract_submit_deadline.py", line 344, in submit
    result = response.json()
  File "C:\Program Files\Ynput\AYON 1.1.1\dependencies\requests\models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Ynput\AYON\dependency_packages\ayon_2411151105_windows.zip\dependencies\pyblish\plugin.py", line 528, in __explicit_process
    runner(*args)
  File "C:\Users\User\AppData\Local\Ynput\AYON\addons\deadline_0.5.2\ayon_deadline\plugins\publish\nuke\submit_nuke_deadline.py", line 89, in process
    job_id = self.process_submission()
  File "C:\Users\User\AppData\Local\Ynput\AYON\addons\deadline_0.5.2\ayon_deadline\abstract_submit_deadline.py", line 173, in process_submission
    return self.submit(payload, auth, verify)
  File "C:\Users\User\AppData\Local\Ynput\AYON\addons\deadline_0.5.2\ayon_deadline\abstract_submit_deadline.py", line 349, in submit
    raise KnownPublishError("Broken response from DL")
ayon_core.pipeline.publish.publish_plugins.KnownPublishError: Broken response from DL

Hopefully someone has a quick fix :slight_smile:

Thanks!

Hmm, thats a pretty lackluster error message :/.

I would try to update `C:\Users\User\AppData\Local\Ynput\AYON\addons\deadline_0.5.2\ayon_deadline\abstract_submit_deadline.py’ line 346 to:

msg = "Broken response {}. ".format(response.text)

Here’s an updated error message, only the first part changes.

INFO: Using U:\project\sequences\kbr\kbr_0750\publish\workfile\workfile2d_prep\v007\KBR_kbr_0750_workfile2d_prep_v007.nk for render/export.
DEBUG: Starting new HTTP connection (1): ip:8081
DEBUG: ip:8081 "POST /api/jobs HTTP/1.1" 200 355
WARNING: Broken response Error: An error occurred while trying to connect to the Database ip:27100. It is possible that the Mongo Database server is incorrectly configured, currently offline, blocked by a firewall, or experiencing network issues.

Full error: An exception occurred while sending a message to the server. (FranticX.Database.DatabaseConnectionException). Try restarting the Deadline Webservice.

The error message points to same Deadline issue with connecting to MongoDB. I dont know what to do about it as it is outside of AYON.

Does any direct submission to Deadline work (as in DL Monitor > Submit >)? I would assume it doesn’t.

If we submit outside of Ayon, it works normally.
Haven’t seen MongoDB before this error tho.

you might try to connect to DL webservice via normal browser or some tool like Postman.

Opening URL like http://<your_dl_webservice:8081/api/jobs should give you the list of jobs seen in Deadline monitor. If you see this error, it is most likely that the machine serving DL webservice can’t really connect to Mongo. Note that using direct submission to Deadline isn’t using webservice at all, in fact all of the Deadline native functionality isn’t using webservice.

Might be also security restrictions for mongo itself - by default it doesn’t listen to external requests and you can connect to it only from localhost.