Testing AYON pipeline early access beta (Deprecated)

I’m not too familiar with docker on Windows and seem to be getting stuck running the ayon server correctly.

My getting started

These are roughly the steps I had taken:

git clone https://github.com/ynput/OpenPype.git
pushd OpenPype
git checkout release/next-minor
git submodule update --init --recursive
popd

git clone https://github.com/ynput/ayon-docker.git
:: manually comment out the volume line for Windows in docker-compose.yml

git clone https://github.com/ynput/ayon-addons-settings.git
:: copy addons to ayon docker addons
xcopy .\ayon-addons-settings .\ayon-docker\addons /E /H /C /I

cd ayon-docker
docker compose up -d
powershell.exe -executionpolicy bypass -file ./manage.ps1 demo

With docker compose up -d I initially got to:

 ✔ Network ayon-docker_default       Created  
 ✔ Volume "ayon-docker_db"           Created   
 ✔ Container ayon-docker-redis-1     Started   
 ✔ Container ayon-docker-postgres-1  Healthy
 ✔ Container ayon-docker-server-1    Started

Failing due to no disk space :wink:

But rather quickly it started showing them as Exited with postgres: Exited (139) and ynput:ayon-dev: Exited (3) and the demo creation also failed:

powershell.exe -executionpolicy bypass -file ./manage.ps1 demo
Error response from daemon: Container 987654321verylongid122 is not running
Error response from daemon: Container 987654321verylongid122 is not running
Error response from daemon: Container 987654321verylongid122 is not running

I found out that apparently this was due to my disk drive being full at that point in time. I cleared up some space

Clear everything, try again…

I removed the full stack, I tried both docker compose down -v and just deleting things through Docker Desktop. Restarted my machine (since Docker itself was giving some issues too.) The images/stack didn’t show anymore so I redid the docker compose up -d

Now it seemed like it started running as intended so I tried to set up the demo:

powershell.exe -executionpolicy bypass -file ./manage.ps1 demo
2023-06-13 20:47:04 INFO       Log collector initialized
Exception in thread Thread-1 (execute):
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/backend/ayon_server/entities/core/attrib.py", line 37, in execute
    loop.run_until_complete(self.load())
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/backend/ayon_server/entities/core/attrib.py", line 47, in load
    async for row in Postgres.iterate(query):
  File "/backend/ayon_server/lib/postgres.py", line 96, in iterate
    statement = await connection.prepare(query)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 565, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 583, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 397, in _get_statement
    statement = await self._protocol.prepare(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 168, in prepare
asyncpg.exceptions.UndefinedTableError: relation "public.attributes" does not exist
Error response from daemon: Container ca0801d51216be4c7de88e1fb3c1e49ae34a31fa8873a0c00085f5f0908f105b is restarting, wait until the container is running
Error response from daemon: Container ca0801d51216be4c7de88e1fb3c1e49ae34a31fa8873a0c00085f5f0908f105b is restarting, wait until the container is running

And then I also noticed the ayon-docker-server container kept restarting and noticed this in the logs:

2023-06-13 22:50:51 2023-06-13 20:50:51 INFO       Log collector initialized
2023-06-13 22:50:53 Exception in thread Thread-1 (execute):
2023-06-13 22:50:53 Traceback (most recent call last):
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
2023-06-13 22:50:53     self.run()
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/threading.py", line 975, in run
2023-06-13 22:50:53     self._target(*self._args, **self._kwargs)
2023-06-13 22:50:53   File "/backend/ayon_server/entities/core/attrib.py", line 37, in execute
2023-06-13 22:50:53     loop.run_until_complete(self.load())
2023-06-13 22:50:53   File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
2023-06-13 22:50:53   File "/backend/ayon_server/entities/core/attrib.py", line 47, in load
2023-06-13 22:50:53     async for row in Postgres.iterate(query):
2023-06-13 22:50:53   File "/backend/ayon_server/lib/postgres.py", line 96, in iterate
2023-06-13 22:50:53     statement = await connection.prepare(query)
2023-06-13 22:50:53                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 565, in prepare
2023-06-13 22:50:53     return await self._prepare(
2023-06-13 22:50:53            ^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 583, in _prepare
2023-06-13 22:50:53     stmt = await self._get_statement(
2023-06-13 22:50:53            ^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/asyncpg/connection.py", line 397, in _get_statement
2023-06-13 22:50:53     statement = await self._protocol.prepare(
2023-06-13 22:50:53                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "asyncpg/protocol/protocol.pyx", line 168, in prepare
2023-06-13 22:50:53 asyncpg.exceptions.UndefinedTableError: relation "public.attributes" does not exist
2023-06-13 22:50:53 [2023-06-13 20:50:53 +0000] [7] [ERROR] Exception in worker process
2023-06-13 22:50:53 Traceback (most recent call last):
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
2023-06-13 22:50:53     worker.init_process()
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/uvicorn/workers.py", line 66, in init_process
2023-06-13 22:50:53     super(UvicornWorker, self).init_process()
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/base.py", line 134, in init_process
2023-06-13 22:50:53     self.load_wsgi()
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
2023-06-13 22:50:53     self.wsgi = self.app.wsgi()
2023-06-13 22:50:53                 ^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/gunicorn/app/base.py", line 67, in wsgi
2023-06-13 22:50:53     self.callable = self.load()
2023-06-13 22:50:53                     ^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
2023-06-13 22:50:53     return self.load_wsgiapp()
2023-06-13 22:50:53            ^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
2023-06-13 22:50:53     return util.import_app(self.app_uri)
2023-06-13 22:50:53            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/gunicorn/util.py", line 359, in import_app
2023-06-13 22:50:53     mod = importlib.import_module(module)
2023-06-13 22:50:53           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
2023-06-13 22:50:53     return _bootstrap._gcd_import(name[level:], package, level)
2023-06-13 22:50:53            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2023-06-13 22:50:53   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2023-06-13 22:50:53   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2023-06-13 22:50:53   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2023-06-13 22:50:53   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2023-06-13 22:50:53   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2023-06-13 22:50:53   File "/backend/ayon_server/api/__init__.py", line 38, in <module>
2023-06-13 22:50:53     from ayon_server.api.server import app
2023-06-13 22:50:53   File "/backend/ayon_server/api/server.py", line 26, in <module>
2023-06-13 22:50:53     from ayon_server.graphql import router as graphql_router
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/__init__.py", line 29, in <module>
2023-06-13 22:50:53     from ayon_server.graphql.nodes.folder import folder_from_record
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/nodes/folder.py", line 9, in <module>
2023-06-13 22:50:53     from ayon_server.graphql.resolvers.products import get_products
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/resolvers/products.py", line 8, in <module>
2023-06-13 22:50:53     from ayon_server.graphql.nodes.product import ProductNode
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/nodes/product.py", line 9, in <module>
2023-06-13 22:50:53     from ayon_server.graphql.resolvers.versions import get_versions
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/resolvers/versions.py", line 7, in <module>
2023-06-13 22:50:53     from ayon_server.graphql.nodes.version import VersionNode
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/nodes/version.py", line 9, in <module>
2023-06-13 22:50:53     from ayon_server.graphql.resolvers.representations import get_representations
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/resolvers/representations.py", line 7, in <module>
2023-06-13 22:50:53     from ayon_server.graphql.nodes.representation import RepresentationNode
2023-06-13 22:50:53   File "/backend/ayon_server/graphql/nodes/representation.py", line 49, in <module>
2023-06-13 22:50:53     @RepresentationEntity.strawberry_attrib()
2023-06-13 22:50:53      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13 22:50:53   File "/usr/local/lib/python3.11/site-packages/strawberry/experimental/pydantic/object_type.py", line 168, in wrap
2023-06-13 22:50:53     raise MissingFieldsListError(cls)
2023-06-13 22:50:53 strawberry.experimental.pydantic.exceptions.MissingFieldsListError: List of fields to copy from <class 'ayon_server.graphql.nodes.representation.RepresentationAttribType'> is empty. Add fields with the `auto` type annotation

What obvious thing did I do wrong?

Solved :white_check_mark:

EDIT: @martin.wacker mentioned on discord this is a bug in pushed updates from today. A solution is to manually trigger setup to deploy the database.

I ran:

powershell.exe -executionpolicy bypass -file ./manage.ps1 setup

Which gave me:

Server container: server
2023-06-13 21:06:05 INFO       Log collector initialized
2023-06-13 21:06:05 INFO       Starting setup
2023-06-13 21:06:06 INFO       (re)creating database schema
2023-06-13 21:06:06 DEBUG      Creating password for user admin
2023-06-13 21:06:06 INFO       Saving user admin
2023-06-13 21:06:06 DEBUG      Creating password for user demo
2023-06-13 21:06:06 INFO       Saving user demo
2023-06-13 21:06:06 DEBUG      Creating api key for user service
2023-06-13 21:06:06 INFO       Saving user service
2023-06-13 21:06:06 INFO       Deploying settings
2023-06-13 21:06:06 INFO       Saving settings for ftrack 1.0.0
2023-06-13 21:06:06 GOOD NEWS  Setup is finished

After that the server worked and I could trigger the demo deployment with powershell.exe -executionpolicy bypass -file ./manage.ps1 demo.