Configuring Tool Groups in OpenPype

Configuring Tool Groups in OpenPype

Keeping track of software versions between projects is a job within itself, but what about different plugins and libraries? How do you factor for different environments per project, without a massive deployment headache?

It’s something that OpenPype handles really nicely, but you might not know how to go about it, or what you might use it for.

The use case for tool groups could be something like…

  • Defining different render engines, and versions to be able to tryout new features.
  • Allowing specific versions of plugins for DCC hosts.
  • Sharing python libraries across multiple hosts.

Host Environment Setup

You might be familiar with setting up a host to allow custom tools to propagate. A common scenario is you might want to have a set of studio scripts load automatically via your PYTHONPATH for every project. You’ll find these top level settings in System > Applications > Hostname.

In Maya, you can do something like the following -

This simply appends the path into the PYTHONPATH and the MAYA_SCRIPT_PATH environment variables.

A more granular approach

But what if you’d like to be able to configure things to have a little more control of what plugins and specifically what versions you are running? This is where tool groups come in.

With tool groups, you can target a specific tool to run within a project. It’s easy to upgrade and not destroy any previous functionality, as you are not overwriting anything. It’s allowing you to choose the environment that a particular host is running on startup.

Setting up a Tool Group for a simple plugin

To begin, add a new key to the System > Tools section. The label is the “nice name” used in the dialog, and the key should be something without spaces or odd characters. Don’t worry about the versions in this part, this will come later.

For demonstration purposes, I’ll run though the steps to configure the legendary Studio Library plugin for Maya.

Add the source folder to network

slib

It needs to be somewhere visible to all machines. We have an environment variable to set the core pipeline tools path in General > Environment. This means if we switch this out to another location in the future, we’d have to change one variable across the whole pipeline. But you can set this part up however you like.

Copy the Studio Library source folder. I use a tools > mayaPlugins > studiolibrary folder. To keep with the conventions of how the script is packaged, the version number is a suffix of the folder name.

Add a key for studiolibrary with a label set to Studio Library

Setup the version you are using.

We’ll map the version string to an environment variable called STUDIO_LIBRARY_VERSION. Add a key within the studio library tool and flag it for the version you are using. We have 2.9.6.b3

In the environments area, add the JSON value for the variable STUDIO_LIBRARY_VERSION

{
    "STUDIO_LIBRARY_VERSION": "2.9.6.b3"
}

This will allow us to add it to the main environment when a host is launched.

Setup the Tool Environment

Different tools will have specific requirements, some might have python scripts, and some might need modules to be added. It’s a good practice to define a variable with the entire root path of the plugin. This way, you can re-use it for different directories within that if needed.

We add a variable named STUDIO_LIBRARY_ROOTand within this we can use other variables to format the correct path. If we add STUDIO_LIBRARY_VERSION into this core environment, OpenPype will inject the correct version string into the file path and load the correct version of the plugin.

You can add a particular host, or even a version of the host if needed too. I don’t generally do this though.

We can then add this to whatever path we like. In the case of Maya, we’ll add to PYTHONPATH and MAYA_SCRIPT_PATH, in case there’s any sneaky mel that needs executing.

{
    "STUDIO_LIBRARY_ROOT": "{NETWORK_ROOT_CONFIG}/tools/studiolibrary/studiolibrary-{STUDIO_LIBRARY_VERSION}",
    "PYTHONPATH": [
        "{STUDIO_LIBRARY_ROOT}/src",
        "{PYTHONPATH}"
    ],
    "MAYA_SCRIPT_PATH": [
        "{STUDIO_LIBRARY_ROOT}/src",
        "{MAYA_SCRIPT_PATH}"
    ]
}

One thing to watch…when appending to an Environment variable that has multiple entries, always make sure to add the existing variable after the new entry. otherwise you’ll overwrite the list with the single, new variable.

Once done, save the settings and it should be configured. You might want to exit the settings and reload them again for the tool to show up.

Adding the Tool to a project

With the settings re-opened, switch to the project you want to use. Wait 45 mins (sorry guys, couldn’t resist that joke :stuck_out_tongue_winking_eye: )

add_studio_lib_tool

In the project anatomy > attributes section, you’ll see a section marked Tools. This will have a dropdown list where you can choose the tool you just added. That should be it!

Upgrading the Tool

To add a newer version of the tool, we repeat the steps from before, but since we have the core environment set, all we need to do is add a new source folder to the network.

sl_upgrade

Then we add an additional key for the new version folder and define the same variable, STUDIO_LIBRARY_VERSION, but set it to the latest build number.

{
    "STUDIO_LIBRARY_VERSION": "2.9.11"
}

studio_Lib_version

Add or swap the version in the project attributes, and the next time you launch, you’ll get the upgraded version.

add_studio_lib_tool_upgrade

On the maya side of things, I use this snippet to provide a studio library database for each project, this grabs the OP environment and launches Studio Library bound to a project path. This code is called by the custom scripts menu integration.

import os   

def show(): 
    """
    This file is to be placed in your global scripts/ maya env directory so that it can be called from the internal pype script menu definition.

    The script will make the following directory in the project if it doesn't already exist.
        - {OPENPYPE_PROJECT_ROOT_WORK}\{AVALON_PROJECT}\tools\studiolibrary

    Would be nice to use Pathlib but this works in python 2. 

    """   
    open_pype_server_root = os.getenv("OPENPYPE_PROJECT_ROOT_WORK") 
    open_pype_project = os.getenv("AVALON_PROJECT")

    if open_pype_server_root and open_pype_project:    
        base_path = os.path.join (open_pype_server_root , open_pype_project , "tools" , "studiolibrary")

        if not os.path.exists(base_path):
            os.makedirs(base_path)

        if os.path.exists(base_path):
            template_path = base_path
        else:
            template_path = None

    if template_path:
        import studiolibrary
        studiolibrary.main(name="Project Library - {}".format(open_pype_project), path=str(template_path))

if __name__ == "__main__":
    show()

In Summary

There’s more complex setups with some tools (for example you might need to specify module paths as well as python paths, but the principles are the same). We use this to setup Arnold, mGear, and individual tools we want to augment to the pipeline.

I hope this guide helps you to set up your own custom tools within your OpenPype deployment.

1 Like

Well deserved, but have you tried it in AYON? Smooooth as butter.

1 Like

Great write-up!

It’s good to note that the tools you setup for a project in its anatomy settings I believe are the default tools for a new asset. Existing assets in a project will not automatically get those tools assigned.

This might differ if using a project manager like Ftrack which might push attributes in a different way but the active tools are stored in the database per asset if I’m correct. So that you can even differ the version of a tool per asset or shot. The Project Manager tool in the Launcher for example does not automatically propagate the tools from the project to the assets.

So where available Applications are defined per project, the available Tools are defined per asset/shot. (Even though on a project level it does allow to set which tools are allowed to be set in the project)


By the way, you should update your Yeti label, Pergrine LabsPeregrine Labs

Yeah I’m mean and I know it :smile:. Ayon looks :fire:

Hey Roy, thanks for your reply. Yeah there is definitely some difference in terms of how these run via project manager or ftrack. (we don’t use project manager)

So much so, we specify the applications and the tools globally in the ftrack project (or in OP after project creation) and this gets picked up in the host. perhaps there’s a better way to do this?

I have a feeling the settings synchronise via the ftrack event server (or they might need a push via the sync to avalon in our admin actions in ftrack)

And you’ve basically looked into my laundry basket - As we don’t use Yeti and I’ve never removed the defaults that were populated for us way before I joined :smiley: shame on you Pete!

Can we elaborate on this for Houdini, please? I understand tools can help me install specific packages in Houdini.

I tried to paste the package JSON script to OP system settings → Tools → New tool

I added the tool to my project, but it was not loaded when I opened the application with OP.

Kindly help me if the script is wrong and rewriting the script in a proper way.

I think you should write them as if you were writing them in houdini.env because houdini packages might not use the exact env names.

I will give it a quick try as soon as I can reach my PC.

Yes! I’ve tried that too. I tried the following way:

{
    "HOUDINI_PATH": "//avril/softwares/Houdini_plugins/ENV/houdini19.5/axis"
}

Yeah Please let me know when you get a chance to check

1 Like

So this is definitely much closer to what it should be, yes.

However, note that the environment set for each application or tool sets the value - it does not append. It’s the same behavior for Application Environment Setings So if you want to add it to the existing paths of your application and merge with other tools you will have to do e.g. this:

{
    "HOUDINI_PATH": ["{HOUDINI_PATH}", "//avril/softwares/Houdini_plugins/ENV/houdini19.5/axis"]
}

This will first insert the existing HOUDINI_PATH value and then add your extra line.

Then after that it’s good to note that you’ll still need to set them on the projects (as defaults) + set the tools for the existing assets/shots. A tool doesn’t ALWAYS get added to all projects, but it set per asset per project - as described here.

Here’s an example of how different tools assigned to different assets look in the project manager (see far right) in OpenPype:

1 Like

That’s what I was missing!

I’ve tried it, and also I tried following.
{
“PATH”: [“{PATH}”, “//avril/softwares/Houdini_plugins/ENV/houdini19.5/axis”]
}

Still, the package was not loaded in Houdini. Did you test tools in Houdini? Did it work for you?

It will work, yes. Since it behaves exactly the same as Application environments which worked for you before.

Let’s debug what your issue is. You can run this in Python in the DCC you launched to see what the value is of the particular env variable:

import os

variable = "HOUDINI_PATH"
print("===")
print("--- Contents for environment variable: {}".format(variable))
for path in os.environ.get(variable, "").split(os.pathsep):
    print(path)
print("--- End of contents")

If your path is NOT in that list then likely the tool is either misconfigured or not set to load for the asset task you launched into. Let’s debug the latter. Let’s see what tools are assigned for you current asset:

from openpype.pipeline.context_tools import get_current_project_asset

asset = get_current_project_asset()
print("Current asset: {}".format(asset["name"]))
tools = asset["data"]["tools_env"]
print("Tools: {}".format(tools))

Example output:

Current asset: asset1
Tools: ['mtoa/5-3-2', 'redshift/3-5-16']

This should print the tools assigned to the current asset/shot. If it’s not listed here then the tool is not yet assigned to the current asset and you’ll need to do that first before it even does anything. :slight_smile:

If it is listed in the asset’s tools correctly but not in the environment variable that sounds like the tool itself is misconfigured. (Or that potentially another tool is misconfigured and instead of appending is overwriting the env var of this tool - this can only happen if there’s more than one tool)

It is listed in both, then likely you are using the wrong path or environment variable for what you are trying to achieve in the first place.

Does that help?

note: I typed this including code on my phone so could contain typos should be fixed now

Many thanks guys, I made a community guide with your help.

It’s working! Thanks a lot @BigRoy @mustafa_jafar

1 Like