Rez + Ayon

flord · 9 June 2023 01:48

Hi,
I’m not using OpenPype or Ayon yet but I’m browsing through the documentation in preparation for an eventual test drive. I noticed that the DCCs must be launched through the Ayon app that must be downloaded and installed locally on the desktop machines. This concept is not compatible with our current solution (Rez) and would complicate things considerably for us.

I’m starting this topic in order to start a discussion around the possibility of adding support for Rez, to talk about how we are using it and hoping others can chime in with their own experience.

Back Story

We are a university with over 250 machines running Windows (not my decision). At the beginning of the school year, the IT department ask all the teachers which software they need for their classes, they install them all on one computer, ask the teachers to test and then image that computer to all the others. Then we are stuck with that config for the rest of the year. Teachers don’t have admin rights on the machines so they cannot install new versions themselves if they wanted. They do have admin rights on their own computer only, in their office.

I didn’t like that. I like to update DCCs and plugins with point releases during the year for all the students. That’s why I installed Rez myself. I put it on the network in a place I have write access, I created a bunch of packages and I even installed Maya, Nuke, Houdini and Mari to that location on the network (installed locally on my machine, copied the folder to the network and created a rez package for them). I did the same with the Arnold plugins for Maya and Houdini and the maya-usd plugin.

The advantages of Rez

This way, I can update those applications and plugins whenever I want (which is usually within a week of a new release). Also, when you launch an application through Rez, it launches from the network but while you are working, it can copy the folder locally to the machine so the next time you (or anyone else) launch the same app, it will use the local copy.

Of course, Rez also manages the versions intelligently. When you request the latest version of Houdini and the latest version of htoa, it uses the latest two versions that are compatible together. That is the main reason for using Rez in the first place.

Rez packages can define environment variables, several different packages can add to one single environment variable (PATH, PYTHONPATH, etc.). They can define aliases to define commands to execute in the terminal (the houdini package defines the houdini alias, the hython alias, husk, mplay, etc.) so once you are inside a rez environment, you can simply type the name of the alias to launch the tool you want.

How we use it

I have created a set of Windows shortcuts in a folder accessible to all. Each shortcut contains a rez-env command and a list of packages to load. For example, the houdini shortcut has this command:

rez-env houdini-19.5 htoa-6 ocio_config_aces castors -- houdini

Specifying houdini-19.5 makes sure that this shortcut never launches Houdini 19.0 or Houdini 20.0 when it’s available. The same goes for htoa, it specifies version 6 so it can pick any version as long as it starts with 6. The ocio_config_aces packages is one I made that provides the config and sets the OCIO environment variable accordingly. And the castors package is for a project we were working on this winter. It adds a few custom HDAs to the houdini session and sets a few settings.

When a new version of Houdini or htoa becomes available, I can simply install it on the network, update the package for it and release it. The next time someone uses the shortcut, the rez-env command will resolve to the new version and launch it. I don’t even have to update the shortcut, or to tell the students about it. If I release a new version of Houdini but don’t release the version of Arnold that goes with it, the shortcut will not launch that new version. It only uses versions that are compatible together.

During the last semester, we needed to test a beta version of Arnold for Houdini. I created a new shortcut specifying the beta version and bypassing the filter (we have a rez filter that ignores beta versions by default). So when using this shortcut, students would fire up a houdini session with the beta version of Arnold and they were able to test it on any machine, and even in the render farm.

Conclusion

To me, it looks like Rez could do a lot of what that Ayon app does already, but in a more flexible way. There could be an ayon package that defines the environment variables it needs to send to the DCC. It could also define the aliases for the commands it has.

This way, studios could choose how they launch applications the way they want with Windows shortcuts, their custom rez launcher, or any other method. Or the Ayon app could use a list of package requests and the name of an alias to launch an app instead of a hard coded path, and inject its own package to the list.

For us, it greatly simplified the way we manage applications and plugins. I even showed it to the IT guys and they were excited how much it improved the system we had in place. I also simplifies the way I can release my own plugins to the students. So I think it would be a blocker for us if we can not launch applications through Rez.

Next year, I will create a shortcut for each application, for each project team. They can decide which version they want to use and if they want to use Arnold with Houdini or Maya, or RenderMan, and decide the version of the renderer they use. This way, each team will have a tailored made application for their needs and we will still be able to update them very easily.

F

flord · 9 June 2023 02:11

And to add to this story, here is how we used it at my previous job.

I worked for RodeoFX for 5 years as Pipeline Architect and then Head of Pipeline. We built a Rez launcher that ran in the terminal and was used by all artists to launch their applications.

It allowed a supervisor to “install” rez packages (or rather rez requests) to a given context on a show, for a given app. So lets say we want to use Nuke on show xyz. The supervisor could decide that Nuke should use Nuke-13.2v<latest>, neat_video-5.4.7, and twixtor-5.<latest>. They simply run the launcher with the install agrument and lists the package requests Nuke-13.2, neat_video-5.4.7, and twixtor-5 for app nuke, for show xyz. This would save the list of requests in the DB for that show. Later, when an artist launches Nuke on that show, the launcher would retrieve the requests and send it to rez to launch the application.

The beauty part is that we could add or remove requests for specific shots. If one shot in that show needs an additional package, we could add it just for that shot. And the shot would still inherit from the show so adding a new package to the show would still add it to all shots under it.

We used that feature pretty often. It happened that a sequence of a show was done using a beta versions of a DCC, while the rest of the shots used a release version. With this system, artists never need to know which version to launch, the launcher does it for them.

This launcher is still in use to this day, with Rez.

antirotor · 9 June 2023 15:51

Hey, let me point you to recently started discussion on Github about the way how Rez can be implemented:

github.com/ynput/OpenPype

Implement Rez for Ayon

opened 11:06AM - 16 Mar 23 UTC

BigRoy

epic type: feature

**Is your feature request related to a problem? Please describe.** Ayon and e…ven OpenPype had to handle dependencies, versions of packages and distributing vendored binaries. With Ayon resolving versions and its dependencies becomes more dynamic due to being able to mix and match addons which in itself can require specific versions of other addons. As a part of this some of the work for Ayon has been involving resolving these dependencies and building an environment for launching a specific version of Ayon, etc. and has with OpenPype (v3) involved managing environments for Applications and Tools when launching even specific assets in a project. This addon (package) management and dependency resolving with version ranges is exactly what [Rez](https://github.com/AcademySoftwareFoundation/rez) is built to solve and I feel like before we set of implementing complex resolving algorithms for the addons and their 'dependencies' we should maybe avoid time spent there and instead spent time on relying on Rez as much as we can. By adopting Rez we can also get other benefits like: - Setting up a central repository of package versions (e.g. ffmpeg, ocio, otio, python packages) without requiring each build of Ayon to embed all those dependencies. - It can support local caching of having the dependencies on a network share but copy certian packages (or all) locally to improve runtime performance. This would then also automatically become possible for things like "tools" where e.g. redshift could be loaded through ayon configuration and its resolving could use that Redshift locally to improve the startup running time for the renderer. - Since Rez is built to resolve dependencies like this we can use its feature set to also help debug dependency problems, etc. without again having to re-write and manage our own. - It can even help the "version resolving" we are using for OpenPype itself currently by turning OpenPype ZIP packages into rez packages, etc. (e.g. at deployment time or alike) to avoid maintaining our own version resolving logic and be able to rely on Rez's mechanisms which have a much stronger backbone in production already. **Describe the solution you'd like** Have Ayon use `rez` to resolve the environment to launch an application in would be one first step. I'd say the final goal would be to be able to run even Ayon itself through rez so from there you could e.g. pick a particular Ayon version to start with too - but that doesn't have to be a first priority I'd say. Being able to pick Ayon version through `rez` would open up e.g. Deadline resolving to a specific ayon version to also just move through `rez` to get the right package. **Describe alternatives you've considered** We've had Applications and Tools in OPv3 but with separating all addons for Ayon and them versioning independently with dependencies (requirements) to other addons will start complicating the version resolving logic that needs to be written. I'd say we are getting to a point where Rez might be best way forward into development time and what we gain from it in the end. **Additional context** ### Rez feature requests have come up before Even before Ayon having Rez support has come up as a request multiple times by multiple users on Discord: - @instinct-vfx [mentioned it recently with regards to Ayon and automatic downloads of addons](https://discord.com/channels/517362899170230292/890894521817260052/1085821403665805405) - [A Rez thread on Ynput community discord](https://discord.com/channels/517362899170230292/954098029135282307) showing interest from @davidlatwe @Tilix4 @Minkiu with some developers mentioning they had working prototypes with OpenPype like @Anti-Distinctlyminty - [Some implementation discussion here](https://discord.com/channels/517362899170230292/954098029135282307/1073159241751347201) and [here](https://discord.com/channels/517362899170230292/563751989075378201/1012723921113587732) - [There have also been requests to keep it separate with some answers by @mkolar at the time about OP philosophy](https://discord.com/channels/517362899170230292/913725331389710337) - [And last but not least some questions about Rez package versus JSON configs from back in 2020](https://discord.com/channels/517362899170230292/517382145552154634/770726352169074719) ### Rez is supported by Academy Software Foundation Rez has ASWF support and used by a variety of bigger studios in the animation industry. [cuID:[OP-5252](https://app.clickup.com/t/85zrrzwwg)]

To add to it - you can use Rez with OpenPype/AYON even now for launching your DCCs - you don’t have to configure application executables, you can use your shell scripts that will invoke Rez to do whatever magic you need to get app started. In the most cases, you can’t then use easily environments set by OpenPype/AYON to control plugins and other tools and that is one of the major caveats.

We are already implementing Rez on server side, to help us managing dependencies between addons btw.

Thanks for this nicely written summary of your Rez usage!

BigRoy · 27 June 2023 08:40

Just wanted to link this draft PR for a Rez integration for the Ayon Backend since it’s one of the steps of linking Rez and Ayon together. Whether it tries to solve what came up here I’m not sure but just referencing so others are aware some Rez effort seems to be put in.

georgeantonopoulos-b · 28 June 2023 12:09

Hi All,

This might relate to a different thread. Would it be possible to use rez commands instead of Application paths to executables?. For example a command such as “rez env nuke plugins … – nuke”
that would launch Nuke with all custom packages plus the OpenPype/Ayon plugins.

Thanks for your time!
George

BigRoy · 28 June 2023 12:55

That’s definitely the goal of this particular topic (or at least what I think @flord described) - so you’re in the right place.

A little above this was mentioned by @antirotor

To add to it - you can use Rez with OpenPype/AYON even now for launching your DCCs - you don’t have to configure application executables, you can use your shell scripts that will invoke Rez to do whatever magic you need to get app started. In the most cases, you can’t then use easily environments set by OpenPype/AYON to control plugins and other tools and that is one of the major caveats.

So likely it’s already possible (in a hacky way) by doing that, but I suspect it’s highly untested.

But there’s also been mention that it makes much more sense to instead replace the “Applications” module/addon with an application launcher module of its own that uses Rez in the backend instead. However, there’s no ready to go solution for that currently.

Definitely vote at the top of this topic if this is indeed what you’re interested in, just so there’s a visual cue of interest for this.

milan · 28 June 2023 13:17

It is actually well tested and has been in production with some clients for a long while. it’s the main reason why we have arguments option in the application settings.

You can play around with rex command and arguments in here: ayon+settings://applications:1.0.0/applications/maya/variants/0/executables/windows

What is not supported yet is replacing the tool groups (dcc plugins for example) with a granular rez setup that you could control from the UI. We are in the middle of conversations with a client that is interested in that thoug, so I think we’ll need to make it a lot more robust very soon.