Ayon/Openpype Env Vars and Tools Configuration Explained

[mustafa_jafar]

Agenda

• Intro
• Configuration
• Testing
• Troubleshooting
• References

Intro

Before using Ayon/OP

I would approach setting environment variables by

1. Editing Environment variables, e.g. on windows:
   

   

   .bat can be used to deploy to all artist machines, e.g.

   setx OCIO "\\storage\configs\ocio\aces_1.2\config.ocio;" /M
   setx MAYA_ENV_DIR "\\storage\configs\maya;" /M
   setx HOUDINI_PACKAGE_DIR "\\storage\configs\houdini;&" /M
   setx NUKE_PATH "\\storage\configs\nuke;" /M
   

   I can’t remember whether UNC paths are allowed or not,
   however I’m just using these values for demonstration.

2. by setting environment variables before launching an application, e.g.

   

   It will be available only for all DCCs sessions that will be run from this CMD session!

   
   
   
   

3. By editing DCCs env files, e.g. houdini env file

   It will be available for all houdini sessions.

   
   
   
   

4. DCCs can provide other methods, e.g. packages in houdini.

   It will be available for all houdini sessions.

   
   
   
   

After using Ayon/OP

Things become more dynamic and much easier, in general you can do all of the mentioned above from one UI, it’s two actually but it’s still the same program and deployment to all artist machines is as simple as clicking save button!

Ayon/OP Options

• Global OP Environment variables

  It will be available for all DCCs

OP	Ayon

• DCC Specific, and DCC Version Specific

  It will be available for all sessions of this specific DCC or DCC version

OP	Ayon

• A flexible configuration to be used on demand

  It can be set to a certain DCC or multiple DCCs, DCC/DCCs versions can be specified also!

OP	Ayon

> It can be set to work with certain projects 

OP	Ayon

> It can be set to work with certain assets

OP	Ayon

Configuration

The following steps are in OpenPype, but the process is almost the same in Ayon.
Refer to the screenshots above.

Let’s go through making few examples, you can follow the same steps to add your desired tools.

Step 1: define environment variables

OpenPype Tray Admin → Studio Settings → System

1. Global OP environment: general → environment
   

   

2. DCC Specific: application → houdini → environment
   DCC Version Specific: applications → houdini → variants → 19-5 → environment
   

   

3. Tools:

• Adding a generic tool for all DCCs,
  

  

• Adding a tool for maya specifically and for houdini 19.5 version specifically.
  

  

Step 2: Tell OP which tools to use with which projects

This is required with Tools only

OpenPype Tray Admin → Studio Settings → System
Go to Anatomy → Attributes → Tools

Step 3: Tell OP which tools to use with which assets

This is required with Tools only

OpenPype Tray Admin → Project Manager
Select an asset and set tools, you have to that on the asset itself not its parent folder.

Multiple selection (shift select) is allowed

Notes

1. Asset is a general term for assets and shots
2. OP doesn’t suppose or use tools by default with projects and assets
3. Projects can inherit tools settings from default project, however it happens while creation then save that as override. so, projects inherit while creation not after creation., that’s why step 2 is required
4. I believe the same happens with assets and their parent folder, that’s why step 3 is required

Testing

1. The most basic test “as a user” to check if the tool is there in DCC or not, user may look in menus or shelves.

   I’m using arbitrary examples in this guide so, there’s nothing to test as a user.

2. Checking available tools (OP way).

   from openpype.pipeline.context_tools import get_current_project_asset
   
   asset = get_current_project_asset()
   print("Current asset: {}".format(asset["name"]))
   tools = asset["data"]["tools_env"]
   print("Tools: {}".format(tools))
   

   Example output:

       Current asset: lamp
   Tools: ['example-tool/generic', 'test-tool/maya', 'test-tool/houdini']
   

   Note:

   It tells nothing about whether it’s already imported to your DCC or not.
   It only lists whatever was set for that asset in the Project manager.
   

   

3. Checking environment variables (the regular way).

   import os
   from pprint import pprint
   from openpype.pipeline.context_tools import get_current_project_asset
   
   env_var = "MAYA_OP_TOOL_ENV"
    
   asset = get_current_project_asset()
   print("Current asset: {}".format(asset["name"]))
   print("--- Contents for environment variable: {}".format(env_var))
   pprint(os.getenv(env_var , "").split(";"))
   print("--- End of contents")
   

   Example output: “from Maya”

   Current asset: lamp
   --- Contents for environment variable: MAYA_OP_TOOL_ENV
   ['This is a maya openpype tool environment']
   --- End of contents
   

   Example output: “from Houdini”

   Current asset: lamp
   --- Contents for environment variable: MAYA_OP_TOOL_ENV
   ['']
   --- End of contents
   

   This was the same asset lamp from the previous example.

These code snippets were provided by @BigRoy. Many thanks!

Troubleshooting

The only troubleshooting I can think of is to check

1. if environment variables were written correctly, try testing them in your DCC env file
2. if tools were set to a particular project, try checking OP studio project settings
3. if tools were set to a particular asset, try checking OP project manager

References

Configuring Tool Groups in OpenPype

Configuring Tool Groups in OpenPype Keeping track of software versions between projects is a job within itself, but what about different plugins and libraries? How do you factor for different environments per project, without a massive deployment headache? It’s something that OpenPype handles really nicely, but you might not know how to go about it, or what you might use it for. The use case for tool groups could be something like… Defining different render engines, and versions to be able …

System Settings | openPYPE

https://ayon.ynput.io/docs/admin_settings_system#tools

[BigRoy]

Existing assets/shots do not inherit the new default project settings for Tools

It’s good to note that Configuring tools for a Project sets the default value for new assets but does not influence any existing assets/shots whatsoever.

As such, if you want to apply the tools to an existing project with existing assets you’ll still need to mass-update all existing assets to be configured with the new tools via Configuring tools for an Asset.

It’s also covered by your note:

3. Projects can inherit tools settings from default project, however it happens while creation then save that as override. so, projects inherit while creation not after creation., that’s why step 2 is required

But I feel like it’s something that’s usually missed. I knew originally that was the case and often forgot about it when I started using OpenPype as a new user.

[mustafa_jafar]

Finally, I could add MOPs

@mustafa_jafar

The most basic test “as a user” to check if the tool is there in DCC

Tested it as user

Also, testing as a dev for debugging purposes
I think it added the literal '{HOUDINI_PATH}' because HOUDINI_PATH wasn’t defined in the first place, no worries anyway.

---

10 minutes ago it didn’t work because of a space
who would know that a space in the env var can be painful!

[BigRoy]

@mustafa_jafar

I think it added the literal '{HOUDINI_PATH}' because HOUDINI_PATH wasn’t defined in the first place, no worries anyway.

Yes - it does that, but for 99% of the cases that should just get ignored. But the environment resolving indeed doesn’t ignore an unset key to avoid other subtle bugs.

Houdini + htoa

It’s also good to note that if e.g. each of your Houdini application version environments e.g. define HOUDINI_VERSION=19.5.532 then your tools can also use that nicely, so that if you e.g. have a plug-in that’s different per houdini version you can then have it add the one dynamically, e.g.:

{
    "HOUDINI_PATH": ["{HOUDINI_PATH}", "//path/to/htoa-x.y.z-{HOUDINI_VERSION}"]
}

HTOA tool global environment:

I cut off the license server stuff

So that you only need to define a single HTOA entry for a version that applies to multiple houdini version instead of defining a htoa tool version PER htoa version and houdini version. Like so:

HTOA tool version environment

Maya + Yeti

Similarly for Maya + Yeti we do that like this:

{
    "YETI_ROOT": {
        "windows": "{CB_MAYA_SHARED}/modules/Yeti-v4.2.9_Maya{MAYA_VERSION}-windows",
        "linux": "{CB_MAYA_SHARED}/modules/Yeti-v4.2.9_Maya{MAYA_VERSION}-linux"
    }
}

Where the Maya Application version for 2024 sets Environment:

{
    "MAYA_VERSION": "2024"
}

Also, CB_MAYA_SHARED for us is globally defined in our pipeline for a user.

[Krishna_Avril]

Hey @BigRoy @mustafa_jafar, I’ve got this strange issue. I added four tools, but only the last package in alphabetical order is loading. The other three plugins are not loaded and are not showing. Any specific script issues?

[BigRoy]

I’ve had issues long time ago that looked similar, but that was due to that there was at least one tool that was overwriting HOUDINI_PATH instead of appending to it - it’s hard to tell if that’s the case because then we’d need to see how your global environment, application environments and each tool environment is configured to see if any of those are wrong.

E.g. if LABS/19-5 would be setting HOUDINI_PATH instead of appending it might be the case that it stops the value from AELIB/V1 to propagate, etc.

Not sure if that’s the issue you’re facing but it might be.

---

Note that the HOUDINI_OTLSCAN_PATH is currently set to “override” the existing value completely if another tool or environment than the one for QLib tool would set it. So you’d also need to do the ["newpathshere", "{HOUDINI_OTLSCAN_PATH}"] way of setting it to also preserve any existing values, etc.

[Krishna_Avril]

{
    "MOPS": "//avril/softwares/Houdini_plugins/ENV/houdini19.5/MOPS;&",
    "HOUDINI_PATH": [
        "{MOPS}",
        "{HOUDINI_PATH}"
    ],
    "method": "append"
}

THIS CODE FIXED THE ISSUE, added method:append

[mustafa_jafar]

I didn’t know that this env var works here, Glad it worked for you.

"method": "append"

---

I tested on my machine and found no issues with having multiple tools that append to "HOUDINI_PATH"

I really can’t tell where is the problem with your screenshot.

[Krishna_Avril]

Hey @mustafa_jafar thanks for looking into this,

It only happened after adding the 4th environment variable, which is SIDEFX LABS.

[BigRoy]

FYI, method append (that’s a Houdini package thing, not an environment variable thing) should not do anything actually so I suspect something else magically resolved your issue.

[mustafa_jafar]

Maybe the magical IT solution turn on/off solved the problem

[Krishna_Avril]

Hi, ahaha! @mustafa_jafar

Hey Guys, Can you try adding MOPS, AELIB, and “SideFX Labs” if possible? After some research, I started facing an issue when adding SideFx Labs as a Tool. I tried multiple ways with no luck. Please share the tool script if it’s working for you guys🙏

[BigRoy]

What type of issues?

I feel like the easiest way to investigate and try to replicate is if you can share your setup.
You seem to be using HOUDINI_PATH to point to .json files that themselves also change the environment (since that’s what Houdini packages do basically).

Could it be the .json packages themselves that are doing something different with your environment than you expect (like overriding instead of appending?)

Anyway, share your houdini packages (particularly the json files) and Tools configurations and it might help point out what’s up.

[mustafa_jafar]

I agree with BigRoy’s comment

regarding aelib:

{
	"AELIB":  "path to aelib folder",
	"PATH": ["{AELIB}" , "{PATH}"]
}

regarding sidefx labs:

I rely on houdini installer to add it. but anyways
you don’t have to add version or enable as OP will taker care of that
so you may add two settings h19.0 and h19.5

{
      "SIDEFXLABS": "path to sidefx labs folder", 
      "PATH": ["{SIDEFXLABS}" , "{PATH}"]
}

Notes:

I didn’t test this code, because I don’t have these packages explicitly outside houdini

Also note that in OP settings

$PATH will be resolved by Houdini unlike "{PATH}" will be resolved by OP this may lead to confusion at first especially if you are copying and pasting packages to OP

A rule of thumb always use "{ANY_VAR}" inside OP settings even if "$ANY_VAR" works, this should eliminate any confusion

for example your qlib configuration : in my opinion it’ll be better to be

{
      "QLIB" : "your path to qlib", 
      "QOTL" : "your path to qotl",
      "HOUDINI_OTLSCAN_PATH" : [ "{QOTL}/base" , "{QOTL}/future", "{QOTL}/experimental" , "{HOUDINI_OTLSCAN_PATH}"],
      "HOUDINI_PATH":  [ "{QLIB}" , "{HOUDINI_PATH}"]
}

you don’t have to worry about the trailing & and you needn’t to write "your path to qlib" inside square brackets because it’s one value.
square brackets are used when you have many values like "HOUDINI_OTLSCAN_PATH" in the above code

[Krishna_Avril]

{
    "MOPS": "//avril/softwares/Houdini_plugins/ENV/houdini19.5/MOPS",
    "HOUDINI_PATH": [
        "{HOUDINI_PATH}",
        "{MOPS}"
    ]
}

Hi, Finally, writing code in this order worked for me. It looks like the order of the package load is very important. Loading the packages first caused the menu bar problem.

Thanks for the great help @BigRoy @mustafa_jafar

[mustafa_jafar]

@Krishna_Avril

It looks like the order of the package load is very important.

tbh, I’m not sure if order matters.

but as long as it works then it’s fine,
good job!

[zafrs]

Hi @BigRoy
i was tried to create the new asset like you said, its only inherit when creation not for existing asset

@BigRoy

Existing assets/shots do not inherit the new default project settings for Tools

I already setup the default tools ( step 2 )
but when create the new asset in project manager/ftrack its still empty on tools column, it didnt use the default tools…
did i miss something or just missunderstanding? Im using Ftrack addons

thanks!

[BigRoy]

@zafrs

but when create the new asset in project manager/ftrack its still empty on tools column, it didnt use the default tools…

I might be mistaken, but I believe in AYON the new asset actually inherits it from the nearest parent, which in this capture would be asset/prop. I suspect @milan @martin.wacker know more about what the intended behavior there would be.

Also, if you’re using the Ftrack addon I believe the intent is that you create and manage the assets and shots via Ftrack and do not edit it in Ayon - again, I might be wrong and @milan will confirm or correct me.

[zafrs]

I also tested in Ftrack, but it seems there might be a bug, maybe?? The only attribute that doesn’t sync with the Ayon database is “tools_env”. I tried changing the frame range in Ftrack from 1001 to 900 on asset build, and inside Maya it changed accordingly, but not for the tools. It’s still empty.
and there is no log error

[zafrs]

Hi @BigRoy, I think I found the problem. The top parent folder, like “asset/shot,” wasn’t configured with tools when it was initially created. After I recreated the hierarchical structure and deleted all top folders, configuring them with the new settings, all assets inherited the tools as expected. I believe I didn’t configure it properly initially.
The important note I’ve realized is that the parent folder on entities needs to be configured first.

However, changing the “Info Tools” from Ftrack didn’t update the tools in the database. It seems that it needs to be done by the project manager, Ayon, specifically for the tools, but for some reason, it didn’t sync properly if from Ftrack Tab Info.

Thanks for your help!