Ayon Env Vars and Tools Configuration Explained

Agenda

  • Intro
  • Configuration
  • Testing
  • Troubleshooting
  • References

Intro

Before using Ayon

As far as I know, Traditionally, we would set environment variables by setting them permanently in the system environment variables OR set them before launch using shell scripts or using some tools like Rez.

OR by just editing the application env file.

Examples

I would approach setting environment variables by

  1. Editing Environment variables, e.g. on windows:

    .bat can be used to deploy to all artist machines, e.g.

    setx OCIO "\\storage\configs\ocio\aces_1.2\config.ocio;" /M
    setx MAYA_ENV_DIR "\\storage\configs\maya;" /M
    setx HOUDINI_PACKAGE_DIR "\\storage\configs\houdini;&" /M
    setx NUKE_PATH "\\storage\configs\nuke;" /M
    

    I can’t remember whether UNC paths are allowed or not,
    however I’m just using these values for demonstration.

  2. by setting environment variables before launching an application, e.g.

    It will be available only for all DCCs sessions that will be run from this CMD session!


  3. you can also do it via modifying the env files of your DCCs
    e.g. houdini env file

    It will be available for all houdini sessions.


  4. Some DCCs can provide other methods, e.g. packages in houdini.

    It will be available for all houdini sessions.


After using Ayon

Things become more dynamic and much easier, in general you can do all of the mentioned above from one UI, it’s two actually but it’s still the same program and deployment to all artist machines is as simple as clicking save button!

AYON Configuration

You can add environment variables in different places:

  • Global Studio Environment variables.
    It will be available in Launcher and for all DCCs.
    Setting location: Studio: ayon+settings://core/environments

  • Additional Project Environment variables.
    Setting location: Studio: ayon+settings://core/project_environments?project={project_name}

  • DCC Specific, and DCC Version Specific
    It will be available for all sessions of this specific DCC or DCC version.
    Setting location: Studio: ayon+settings://applications/applications/{app_name}

  • Tool Groups: A flexible configuration to be used on demand.
    where you can add it to a project, folder path and filter by DCC or DCC version
    Setting location: Studio: ayon+settings://applications/tool_groups

    Legacy Tools attributes
OpenPype - deprecated

OP Configuration

You can add environment variables in different places:

  • Global AYON Environment variables

    It will be available for all DCCs

  • DCC Specific, and DCC Version Specific

    It will be available for all sessions of this specific DCC or DCC version

  • A flexible configuration to be used on demand

    It can be set to a certain DCC or multiple DCCs, DCC/DCCs versions can be specified also!

    It can be set to work with certain projects

> It can be set to work with certain assets


The following steps are in OpenPype, but the process is almost the same in Ayon.
Refer to the screenshots above.

Let’s go through making few examples, you can follow the same steps to add your desired tools.

Step 1: define environment variables

OpenPype Tray Admin → Studio Settings → System

  1. Global OP environment: general → environment

  2. DCC Specific: application → houdini → environment
    DCC Version Specific: applications → houdini → variants → 19-5 → environment

  3. Tools:

Step 2: Tell OP which tools to use with which projects

This is required with Tools only

OpenPype Tray Admin → Studio Settings → System
Go to Anatomy → Attributes → Tools

Step 3: Tell OP which tools to use with which assets

This is required with Tools only

OpenPype Tray Admin → Project Manager
Select an asset and set tools, you have to that on the asset itself not its parent folder.

Multiple selection (shift select) is allowed

Notes

  1. Asset is a general term for assets and shots
  2. OP doesn’t suppose or use tools by default with projects and assets
  3. Projects can inherit tools settings from default project, however it happens while creation then save that as override. so, projects inherit while creation not after creation., that’s why step 2 is required
  4. I believe the same happens with assets and their parent folder, that’s why step 3 is required

Testing

  1. The most basic test “as a user” to check if the tool is there in DCC or not, user may look in menus or shelves.
    e.g. I was testing adding MOPs as tool in my Houdini. So, as I user I’m expecting to be able to find MOPs nodes after launching Houdini.

  2. Check the tools that were set in AYON applications settings
    It simply shows the settings assigned to your folder or task within the Project Editor. It does not provide information on whether the tool has been imported.

    Legacy Tools attributes
    from ayon_core.pipeline.context_tools import (
        get_current_folder_path,
        get_current_task_entity
    )
     
    folder_path = get_current_folder_path()
    print("Current folder path: {}".format(folder_path))
    current_task = get_current_task_entity()
    print("Current task: {}".format(current_task["name"]))
    tools = current_task["attrib"]["tools"]
    print("Tools: {}".format(tools))
    
    from ayon_applications.utils import get_tools_for_context
    
    from ayon_core.pipeline.context_tools import (
        get_current_folder_entity,
        get_current_task_entity
    )
    from ayon_core.pipeline import get_current_project_name
    
    
    project_name = get_current_project_name()
    folder_entity = get_current_folder_entity()
    task_entity = get_current_task_entity()
    
    tools = get_tools_for_context(
        project_name,
        folder_entity,
        task_entity,
    )
    
    print("Current folder path:", folder_entity["path"])
    print("Current task:", task_entity["name"])
    print("Tools:", tools)
    

    Example output:

    Current folder path: /Assets/Character/moco
    Current task: cfx
    Tools: ['houdini-mops/v1-6']
    
  3. Check if environment variables were actually set in your application.

    import os
    from pprint import pprint
    from ayon_core.pipeline.context_tools import (
        get_current_folder_path,
        get_current_task_entity
    )
    
    env_var = "MAYA_OP_TOOL_ENV"
     
    folder_path = get_current_folder_path()
    print("Current folder path: {}".format(folder_path))
    current_task = get_current_task_entity()
    print("Current task: {}".format(current_task["name"]))
    print("--- Contents for environment variable: {}".format(env_var))
    pprint(os.getenv(env_var , "").split(";"))
    print("--- End of contents")
    

    Example output: “from Maya”

    Current folder path: /Assets/Character/moco
    Current task: cfx
    --- Contents for environment variable: MAYA_OP_TOOL_ENV
    ['This is a maya ayon tool environment']
    --- End of contents
    

    Example output: “from Houdini”

    Current folder path: /Assets/Character/moco
    Current task: cfx
    --- Contents for environment variable: MAYA_OP_TOOL_ENV
    ['']
    --- End of contents
    

    This was the same asset lamp from the previous example.

These code snippets were provided by @BigRoy. Many thanks!

Troubleshooting

The only troubleshooting I can think of is to check

  1. if environment variables were written correctly, try testing them in your DCC env file
  2. if tools were set to a particular project, try checking OP studio project settings
  3. if tools were set to a particular asset, try checking OP project manager

References

3 Likes

Existing assets/shots do not inherit the new default project settings for Tools

It’s good to note that Configuring tools for a Project sets the default value for new assets but does not influence any existing assets/shots whatsoever.

As such, if you want to apply the tools to an existing project with existing assets you’ll still need to mass-update all existing assets to be configured with the new tools via Configuring tools for an Asset.

It’s also covered by your note:

  1. Projects can inherit tools settings from default project, however it happens while creation then save that as override. so, projects inherit while creation not after creation., that’s why step 2 is required

But I feel like it’s something that’s usually missed. I knew originally that was the case and often forgot about it when I started using OpenPype as a new user.

1 Like

Finally, I could add MOPs :smiley:

Tested it as user

Also, testing as a dev for debugging purposes
I think it added the literal '{HOUDINI_PATH}' because HOUDINI_PATH wasn’t defined in the first place, no worries anyway.
image


10 minutes ago it didn’t work because of a space :sweat_smile:
who would know that a space in the env var can be painful!

image

Yes - it does that, but for 99% of the cases that should just get ignored. But the environment resolving indeed doesn’t ignore an unset key to avoid other subtle bugs.

Houdini + htoa

It’s also good to note that if e.g. each of your Houdini application version environments e.g. define HOUDINI_VERSION=19.5.532 then your tools can also use that nicely, so that if you e.g. have a plug-in that’s different per houdini version you can then have it add the one dynamically, e.g.:

{
    "HOUDINI_PATH": ["{HOUDINI_PATH}", "//path/to/htoa-x.y.z-{HOUDINI_VERSION}"]
}

HTOA tool global environment:
image
I cut off the license server stuff

So that you only need to define a single HTOA entry for a version that applies to multiple houdini version instead of defining a htoa tool version PER htoa version and houdini version. Like so:

HTOA tool version environment

Maya + Yeti

Similarly for Maya + Yeti we do that like this:

{
    "YETI_ROOT": {
        "windows": "{CB_MAYA_SHARED}/modules/Yeti-v4.2.9_Maya{MAYA_VERSION}-windows",
        "linux": "{CB_MAYA_SHARED}/modules/Yeti-v4.2.9_Maya{MAYA_VERSION}-linux"
    }
}

Where the Maya Application version for 2024 sets Environment:

{
    "MAYA_VERSION": "2024"
}

Also, CB_MAYA_SHARED for us is globally defined in our pipeline for a user.

2 Likes

Hey @BigRoy @mustafa_jafar, I’ve got this strange issue. I added four tools, but only the last package in alphabetical order is loading. The other three plugins are not loaded and are not showing. Any specific script issues?

I’ve had issues long time ago that looked similar, but that was due to that there was at least one tool that was overwriting HOUDINI_PATH instead of appending to it - it’s hard to tell if that’s the case because then we’d need to see how your global environment, application environments and each tool environment is configured to see if any of those are wrong.

E.g. if LABS/19-5 would be setting HOUDINI_PATH instead of appending it might be the case that it stops the value from AELIB/V1 to propagate, etc.

Not sure if that’s the issue you’re facing but it might be.


Note that the HOUDINI_OTLSCAN_PATH is currently set to “override” the existing value completely if another tool or environment than the one for QLib tool would set it. So you’d also need to do the ["newpathshere", "{HOUDINI_OTLSCAN_PATH}"] way of setting it to also preserve any existing values, etc.

{
    "MOPS": "//avril/softwares/Houdini_plugins/ENV/houdini19.5/MOPS;&",
    "HOUDINI_PATH": [
        "{MOPS}",
        "{HOUDINI_PATH}"
    ],
    "method": "append"
}

THIS CODE FIXED THE ISSUE, added method:append

I didn’t know that this env var works here, Glad it worked for you.

"method": "append"

I tested on my machine and found no issues with having multiple tools that append to "HOUDINI_PATH"

I really can’t tell where is the problem with your screenshot.

Hey @mustafa_jafar thanks for looking into this,

It only happened after adding the 4th environment variable, which is SIDEFX LABS.

FYI, method append (that’s a Houdini package thing, not an environment variable thing) should not do anything actually so I suspect something else magically resolved your issue. :slight_smile:

Maybe the magical IT solution turn on/off solved the problem :sweat_smile:

Hi, ahaha! @mustafa_jafar

Hey Guys, Can you try adding MOPS, AELIB, and “SideFX Labs” if possible? After some research, I started facing an issue when adding SideFx Labs as a Tool. I tried multiple ways with no luck. Please share the tool script if it’s working for you guys🙏

What type of issues?

I feel like the easiest way to investigate and try to replicate is if you can share your setup.
You seem to be using HOUDINI_PATH to point to .json files that themselves also change the environment (since that’s what Houdini packages do basically).

Could it be the .json packages themselves that are doing something different with your environment than you expect (like overriding instead of appending?)

Anyway, share your houdini packages (particularly the json files) and Tools configurations and it might help point out what’s up.

I agree with BigRoy’s comment

regarding aelib:

{
	"AELIB":  "path to aelib folder",
	"PATH": ["{AELIB}" , "{PATH}"]
}

regarding sidefx labs:

I rely on houdini installer to add it. but anyways
you don’t have to add version or enable as OP will taker care of that
so you may add two settings h19.0 and h19.5

{
      "SIDEFXLABS": "path to sidefx labs folder", 
      "PATH": ["{SIDEFXLABS}" , "{PATH}"]
}

Notes:

I didn’t test this code, because I don’t have these packages explicitly outside houdini

Also note that in OP settings

$PATH will be resolved by Houdini unlike "{PATH}" will be resolved by OP this may lead to confusion at first especially if you are copying and pasting packages to OP

A rule of thumb always use "{ANY_VAR}" inside OP settings even if "$ANY_VAR" works, this should eliminate any confusion

for example your qlib configuration : in my opinion it’ll be better to be

{
      "QLIB" : "your path to qlib", 
      "QOTL" : "your path to qotl",
      "HOUDINI_OTLSCAN_PATH" : [ "{QOTL}/base" , "{QOTL}/future", "{QOTL}/experimental" , "{HOUDINI_OTLSCAN_PATH}"],
      "HOUDINI_PATH":  [ "{QLIB}" , "{HOUDINI_PATH}"]
}

you don’t have to worry about the trailing & and you needn’t to write "your path to qlib" inside square brackets because it’s one value.
square brackets are used when you have many values like "HOUDINI_OTLSCAN_PATH" in the above code

{
    "MOPS": "//avril/softwares/Houdini_plugins/ENV/houdini19.5/MOPS",
    "HOUDINI_PATH": [
        "{HOUDINI_PATH}",
        "{MOPS}"
    ]
}

Hi, Finally, writing code in this order worked for me. It looks like the order of the package load is very important. Loading the packages first caused the menu bar problem.

Thanks for the great help @BigRoy @mustafa_jafar

1 Like

tbh, I’m not sure if order matters.

but as long as it works then it’s fine,
good job! :clap:

Hi @BigRoy
i was tried to create the new asset like you said, its only inherit when creation not for existing asset

image
I already setup the default tools ( step 2 )
but when create the new asset in project manager/ftrack its still empty on tools column, it didnt use the default tools…
did i miss something or just missunderstanding? Im using Ftrack addons
creating

thanks!

I might be mistaken, but I believe in AYON the new asset actually inherits it from the nearest parent, which in this capture would be asset/prop. I suspect @milan @martin.wacker know more about what the intended behavior there would be.

Also, if you’re using the Ftrack addon I believe the intent is that you create and manage the assets and shots via Ftrack and do not edit it in Ayon - again, I might be wrong and @milan will confirm or correct me. :slight_smile:

I also tested in Ftrack, but it seems there might be a bug, maybe?? The only attribute that doesn’t sync with the Ayon database is “tools_env”. I tried changing the frame range in Ftrack from 1001 to 900 on asset build, and inside Maya it changed accordingly, but not for the tools. It’s still empty.
and there is no log error
creating2asset

1 Like

Hi @BigRoy, I think I found the problem. The top parent folder, like “asset/shot,” wasn’t configured with tools when it was initially created. After I recreated the hierarchical structure and deleted all top folders, configuring them with the new settings, all assets inherited the tools as expected. I believe I didn’t configure it properly initially.
The important note I’ve realized is that the parent folder on entities needs to be configured first.

However, changing the “Info Tools” from Ftrack didn’t update the tools in the database. It seems that it needs to be done by the project manager, Ayon, specifically for the tools, but for some reason, it didn’t sync properly if from Ftrack Tab Info.

Thanks for your help!