Best way to push an initial ayon project to kitsu?

mgrosmaire · 16 July 2024 07:28

I’ve got an ayon project and I want to start using Kitsu what’s the best way of doing that.

Can I push my ayon tasks to kitsu, sync, clean up stuff and start working from there?

Or can I make a kitsu project and map my ayon tasks to kitsu tasks?

Also the docs mention a push-to-zou command but kitsu doesnt show up as a module at all in my ayon_console. Is that right?

It does also say openpype_console as well so might be outdated.

mustafa_jafar · 16 July 2024 07:35

Hello,
Currently, as far as I know, the sync works in one direction Kitsu -> AYON
You can sync/bring users, projects, tasks and etc from Kitsu to AYON but not the opposite.

The docs you mentioned is deprecated from OpenPype days.
There’s a PR for updating kitsu addon docs.

Also, for reference, here’s a the community post where kitsu sync was discussed

I think @Danell can provide further insights.

mgrosmaire · 16 July 2024 08:37

Thanks for the answer!

I thought that might be the case but was hoping I was wrong.

timsergeeff · 16 July 2024 18:39

Let me share my experiance!)

(in my case: mostly its compositing work and cgi)

When I start a project i create a project in kitsu and create one sequence with Edit task on it

after i pair kitsu project to ayon (for be its a good habbit to relaunch kitsu service because i found that sometimes it lags and not syncs properly but after relauch it works greate i think that this is problem w my architecture and how i build all my ierarhy so i believe its not a cane for everyone)

after i configure apps (because only apps not copying from production anatomy template @mustafa_jafar hope sometime pairing will copy full settings from anatomy)

after i open hiero to conform shots for comp, after conforming is done, i export working shots to exr acescg dwab and build new track from exported exrs (its nessesary because ffmpeg sometimes brake framerange and i found its better to conwert incoming plates earlier because anyway it will be conderted after comp anyway but no problems with further conforming and publishing shots)

now we have exact number of shots we need to compose so we can came back to kitsu and create needed nubder of shots (and maybe already tasks for each shot)

after i create publishible shots from exrs and then publish it

after all of it ill have wokring ierarhy of tasts and shots and all res\framerange etc data to comp\3d etc and combining it in the last steps of a proects

mgrosmaire · 17 July 2024 08:06

Yeah from now on, we will setup projects in kitsu first but I was hoping I could have some of our existing ayon projects move over to kitsu as well.

Following on from this, I’ve tried setting up a test project in kitsu, syncing to ayon, it failed so I deleted that test project and restarted my ayon server and now my server is stuck on an endless loop of restarting.

Here’s the cycle the server is going through:

start.sh: Starting setup
2024-07-17 17:59:03 INFO       Log collector initialized
2024-07-17 17:59:03 INFO       setup           Starting setup
2024-07-17 17:59:04 GOOD NEWS  setup           Setup is finished
start.sh: Starting gunicorn server
2024-07-17 17:59:04 INFO       Log collector initialized
2024-07-17 17:59:04 ERROR      server          API plug-in 'actions' has no router
2024-07-17 17:59:05 ERROR      server          API plug-in 'review' has no router
2024-07-17 17:59:06 DEBUG      server          Adding access_group editor
2024-07-17 17:59:06 DEBUG      server          Adding access_group user
[2024-07-17 17:59:06 +1000] [10] [ERROR] Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 732, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 608, in __aenter__
    await self._router.startup()
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 709, in startup
    await handler()
  File "/backend/ayon_server/api/server.py", line 403, in startup_event
    await AccessGroups.load()
  File "/backend/ayon_server/access/access_groups.py", line 35, in load
    async for row in Postgres.iterate(
  File "/backend/ayon_server/lib/postgres.py", line 155, in iterate
    statement = await connection.prepare(query)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 636, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 654, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 433, in _get_statement
    statement = await self._protocol.prepare(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 166, in prepare
asyncpg.exceptions.UndefinedTableError: relation "project_corsican_kitsu.access_groups" does not exist

[2024-07-17 17:59:06 +1000] [10] [ERROR] Application startup failed. Exiting.
[2024-07-17 17:59:07 +1000] [1] [ERROR] Worker (pid:10) exited with code 3
[2024-07-17 17:59:07 +1000] [1] [ERROR] Shutting down: Master
[2024-07-17 17:59:07 +1000] [1] [ERROR] Reason: Worker failed to boot

That corsican_kitsu here being my deleted test project.
I’m not sure if this is a kitsu issue or a deleting an ayon project issue.

mgrosmaire · 17 July 2024 13:21

Looks like this might already be fixed by this :

BigRoy · 17 July 2024 14:30

Yes. Without that fix. What’s best to clear that REDIS cache? @martin.wacker

martin.wacker · 17 July 2024 22:29

Without that fix, the easiest way is to just restart redis container (docker compose restart redis)… It will also log out users - I’m sorry about this one - we aim to release the next version today/tomorrow in the case you don’t want to use dev image.

mgrosmaire · 18 July 2024 06:25

All good, I ended up panic pulling the dev image as soon as I saw that fix and that solved my issues.

Damien · 15 September 2024 13:01

Hi @timsergeeff ,
Thanks for your sharing.
Could you please elaborate how you create shots ? I do not find a way to push “plates” into Kitsu from Hiero and automatically create shots.
For now the only way I found working is:

Create a project in Kitsu
Pair it within Ayon
publish plates from Hiero within Ayon (i do not manage to make a publish from resolve work properly yet)
use a CSV to create shots in Kitsu (because publish of plates did not create any review in Kitsu, even if there is a conform task at episode level (did not try at sequence level yet).
work within NUKE for instance and publish (therefore the kitsu addon is working to change status and upload reveiw (still need to make the review better in terms of color (ACES transform seems not OK, i need to dig into OCIO & ffmpeg parameters to avoid stretched videos especially when using anamorphic shots).
(next step) manage to publish a model from Blender - for now i do not manage to have a publish of an asset into Kitsu (probably because there is no review created for models)

timsergeeff · 15 September 2024 18:27

There are no functions like that afaik
For now im thinking about upgraiding kitsu service to create shots whitch is in ayon but not in kitsu so backwards sync but idk when ill do it

But we need to maybe change some logic of hiero plugin too because it publishes platet to SHOTS not to TASKS so there is just no context for plugin to publich plates to

For now workflow is something like that

Im doing like that:

create production in kitsu
pair it ro ayon
create sequence and task for conform
launch task in hiero (da vinci brocken for me too but i find hiero is better for that type of task)
import prestored files (usually from temp folder whitch ill delete later)
conform
transcode to exr aces cg and copy sourse clips to project folder ( i have specific hiero presset)
replase all files on timeline with exrs ( this step is nessesary because i have several sourse clips whitch fails to encode when publishing (ayon uses ffmpeg for trim) resulting final clip one frame shorter so in nuke it will fail to pyblish)
now u see all your plates you want to publish so u came back to kitsu and create as many shots as you have in timeline ( i mean main plates not cleanplates etc)
you make sure kitsu shots,ve made their way to ayon
create shots in hiero using same naming pattern so plates will fall inside appropriate “from kitsu shots” ( i just configured naming patters in setting so almost every time its the same)
publish
check plates in shots using loader just to make sure its ok
next manualy create some screenshots or steal them from brief and post it to kitsu tasks ( because anyway you need to tell your artists what to do so you will need to write something down anyway so for now lack of automatick uploading version 0 aka source plate is manageble but i agree annoying hope i or some smarter guy will hear us and work on it)
artists work on theris tasks

This is intresting because for shots created in ayon is newer work in kitsu since they lack of [kitsu_id] attribute idk how you’ve made it work will maybe try doing like that

This is easy. For nuke you need to turn on extract review intermediates and configure desired color under that settings and check scale pixel aspect in core extract review

For 3ds u need to configure oiio plugin but i think for now it breaks anamorphic metadata so after it mp4 will be distorted anyway (maybe my setup need to be redone idk i am myself still digging a lot)

(My outputs from nuke or houduini if exr aces cg, mov srgb, mp4 srgb w burnin)

Damien · 15 September 2024 19:02

Thank you for your input.
I do not create manually shots in Kitsu, but I plan to use a Python Script to make the CSV based on all main plates created at previous stage.
For now I manually created the CSV and it works like a charm.
It populates the thumbnail as soon as a compositing task has been published.

For now I do not understand how to publish plates (its review actually) onto Kitsu (where do you populate your conform task at sequence or episode level ?)

For me step 12. publishes inside Ayon but do not publish anything in Kitsu. Did you manage to upload something into Kitsu from this publish ?
14. for me this is coming with the CSV file (you can add comment for any task like compositing , rendering or FX)

For OCIO, I will dig into it. I am used to ACES within Resolve, and for now I cannot find a the way in Nuke to get the proper result (meaning correspondance between ACES and the REC709 ref, but this is more a Nuke related thing than Ayon - except if I want to switch to newer versions of OCIO and ACES than what’s baked in).

timsergeeff · 15 September 2024 19:18

No there are no fuctionality like that for now as i see because all platec publishes to SHOT not to task. But for kitsu publish task is required

Damien · 4 October 2024 15:35

Hi,
CSV is working like a charm in KITSU.
I wrote a Python script to scan the shots, and it creates the CSV that I just have to put into Kitsu (I juste need to check how do handle handles !).

Any advice for the color management ?
I’m a bit lost of the color management of the review, maybe I need to dig into Extract OIIO, for now I did not have a look at it.

Kr,

mustafa_jafar · 8 October 2024 06:59

Hey, do you you know that you can convert that to a cli action in Kitsu addon?

But, doesn’t the sync feature work on your side?

Damien · 8 October 2024 07:32

Hi
The sync feature works KITSU > AYON.
But what I wanted to do was to create all the shots at once based on the published plates. The CSV is a way to create all the shots at once in KITSU.
Then the sync does its way to AYON to create all the tasks.

My workflow is as follows:

Create Project in Kitsu
Create a conform task
Pair it into AYON
Publish plates through HIERO in AYON
use a python script to generate all metadata and tasks based on EXR plates for KITSU
import this csv into KITSU
The sync makes magic happens, creating all the tasks in AYON

I’m still stucked with representations and color reviews for now.