Make addons default to picking up last settings

Hey guys,

Is there any way we could make it so every time I update the version of an addon by default it copies the settings from the prior version?

Thank you!

1 Like

100%, Yes please! I always want it to transfer my settings where it can.

I believe not doing this by default was an intentional design choice by the Ynput team so that in “Copy settings” one could at least identify what’s being transferred and what settings may not come along, but I’d like to argue that the copy settings UI is not designed to solve that either. It doesn’t show me at all which settings it failed to transfer. In 99.99% of my own use cases I always end up copying settings for all produciton variants, bundles and for all addons from my previous settings.

So yes, :100: I’d love this!

1 Like

I fully agree.
The current behaviour implies lots of manual tweaks and checks, which is cumbersome.

1 Like

Here is what Milan answered, on Discord last december :

A new version of each addon, could have different setting potentially. Hence automatic settings migration, while technically not difficult at all, can introduce totally unexpected behaviour… (this was sometimes happening in OP, where it worked automatically).

We’ll simplify the steps you need to take to bare minimum. but probably won’t be able to get rid of good old settings migration.

I like the idea… and with the current mirgration options it should work in the most cases - it is actually much simpler to do the migration server-side rather than using the editor.

But we need to make sure all conflict are properly logged and admins are aware that some settings weren’t propagated in the case of incompatibility.

Settings could be auto-migrated when the server restarts after the addon installation (addons need to be active in order to properly access their settings model).

The question is: In this scenario, should the older versions be propagated “as-is” in terms of settings variants? e.g. production to production, staging to staging, and the same for all project overrides?

That makes sense. It should probably be optional.

I’d say so, yes. Also for dev bundles.

Especially on the first server restart after addon upload that shouldn’t break anything yet, because before that point those new addons wouldn’t exist yet anyway and hence aren’t used in bundles yet. So it would only ever influence “future” bundles that still need to be created - even if it were copied for Production, etc.

However - when re-uploading an addon that existed already it should do nothing. (Which may happen in some cases.)

Also, what should happen if one were to upload an older addon version for whatever reason?

Also, is it just me, or is it super-weird that when e.g. updating addons on a dev bundle that you cannot copy settings from the older version of the addon in that dev bundle?

Note that this screenshot also suffers from the issues mentioned here.

Usually for my dev bundle I just want to use the new addon versions using the bundles’ previous setting overrides. :confused: But I can’t pick the other version for that bundle and copy settings for it?

I see dropdowns/comboboxes for the addons but they seem locked?

@martin.wacker any idea?

Yeah this is one of the areas where I spend most time with AYON, copying settings over and over, so any improvement will be very welcome from my side! As it’s already being discussed here Copy Settings: Show the actual differences being copied. · Issue #495 · ynput/ayon-frontend · GitHub even the reasoning behind doing it this way because “doing it automatically can introduce unexpected behaviour…” I have to admit that the manual way has introduced as much unexpected behaviour for me. Maybe it’s been due to schemas changing from addon to addon (and thus the copying of settings not being able to) but because there’s no clear sign anywhere of some things not being copied I have gotten already two or three times surprised of why some settings I had set a while back where no longer set… which is a pretty big worry if this happens in production as a lot of the studio “sauce” is happening in the settings.

@martin.wacker do you have any “for dummies” examples on how to copy settings via the API?

Basically what I’d like to do is automate:

  • For a variant + addon copy the settings from another variant + addon version
  • I want to do this for both studio settings and project settings

Something along the form of:

def copy_settings(source_variant: str, 
                  source_addon: str, 
                  destination_variant: str, 
                  destination_addon: str):
    settings: dict = get_settings(source_variant, source_addon)
    apply_settings(destination_variant, destination_addon, settings)

    for project in get_projects():
        project_settings: dict = get_project_settings(project, source_variant, source_addon)
        apply_project_settings(project, destination_variant, destination_addon, project_settings)

I have so many studio setting overrides per bundle and project specific overrides that it starts becoming very tedious - and worse, very dangerous! - to copy these manually one by one per bundle, per project, etc.

It would alleviate the issue here a bit by being able to do this by running a script after uploading new addons myself.

If the setting schema remains the same, you can just copy “rawOverrides” - that means data stored in the database. there is a GET/PUT endpoint pair to do that (it is what “Low-level editor” uses.

Keep in mind you shouldn’t copy studio overrides to project and vice versa as some of the keys may be scoped to a certain level - it wouldn’t hurt, but you wouldn’t be able to get rid of them using the UI for example.

This is a minimal example for that:

import logging
import os

from typing import Any

import dotenv
import requests

dotenv.load_dotenv()

headers = {
    "x-api-key": os.getenv("AYON_API_KEY"),
}

base_url = os.getenv("AYON_SERVER_URL")


def compare_schemas(
    source_schema: dict[str, Any],
    target_schema: dict[str, Any],
) -> None:
    """
    Deep compare two schemas and raise an exception if they are not equal.
    """

    # Check if the keys are the same
    assert source_schema.keys() == target_schema.keys()

    for key in source_schema.keys():
        source_value = source_schema[key]
        target_value = target_schema[key]

        # Check if the types are the same
        assert type(source_value) == type(target_value)

        # Check if the values are the same
        if isinstance(source_value, dict):
            compare_schemas(source_value, target_value)
        else:
            assert source_value == target_value, f"{source_value} != {target_value}"




def copy_project_settings(
    addon_name: str,
    source_version: str,
    target_version: str,
    source_project_name: str | None = None,
    target_project_name: str | None = None,
    source_variant: str = "production",
    target_variant: str = "production",
) -> None:
    # assert you either provide source_project_name and target_project_name or none of them

    if source_version != target_version:
        # if versions are different, compare schemas
        response = requests.get(
            f"{base_url}/api/addons/{addon_name}/{source_version}/schema",
            headers=headers,
        )
        source_schema = response.json()

        response = requests.get(
            f"{base_url}/api/addons/{addon_name}/{target_version}/schema",
            headers=headers,
        )
        target_schema = response.json()

        try:
            compare_schemas(source_schema, target_schema)
        except AssertionError as e:
            logging.error(f"Unable to migrate {addon_name} {source_version} to {target_version}. Schema mismatch: {e}")
            return


    assert (source_project_name is None and target_project_name is None) or (
        source_project_name is not None and target_project_name is not None
    )

    source_project_suffix = f"/{source_project_name}" if source_project_name else ""
    target_project_suffix = f"/{target_project_name}" if target_project_name else ""

    # Get settings from source project

    response = requests.get(
        f"{base_url}/api/addons/{addon_name}/{source_version}/rawOverrides{source_project_suffix}",
        params={"variant": source_variant},
        headers=headers,
    )

    settings_overrides = response.json()

    # Save settings to target project

    response = requests.put(
        f"{base_url}/api/addons/{addon_name}/{target_version}/rawOverrides{target_project_suffix}",
        params={"variant": target_variant},
        json=settings_overrides,
        headers=headers,
    )


if __name__ == "__main__":
    copy_project_settings(
        addon_name="blender",
        source_version="0.1.8",
        target_version="0.1.8",
        source_variant="staging",
        target_variant="production",
        source_project_name="demo_Commercial",
        target_project_name="demo_Big_Episodic",
    )