Publishing one-frame sequences

General
,
1

Hello dear Ayoneers!

When working with shots, sometimes we need to publish just one frame, instead of a frame range, but Ayon treats differently these two types of publishes which ends up in a non unified pipeline between both: different plugins will produce different results for each publish type, but it shouldn’t really matter if I publish one frame or more, right?

Single frames are treated as single images, the files key from the representation becomes a string instead of a list, which makes sense in some cases, but not all.
I tried to publish a representation where the files key is a list with just one frame and I found several issues:

  • ExtractReview plugin wouldn’t work with a single frame sequences because clique.assembly is called several times with the default minumum_items=2, thus leaving out single frame sequences.
  • IntegrateAsset plugin won’t allow the integration for similar reasons.

I can see an easy way out, which is already there: if files is a list, let’s treat it as a sequence, regardless of the number of frames, else it is not a sequence, so a different pipeline is needed.
I made this proof of concept and I managed to successfully publish a single frame sequence, but I would like to ask you guys what do you think, as I can imagine this topic must have come up before, and may have other higher implications in the Ayon framework.

Thank you!

2

The tricky thing with clique.assemble and minimum items of 1 is that it suddenly becomes very loose on what is a sequence (and quite slow too)

import clique

print(clique.assemble(["hello01.1001.exr"], minimum_items=1))

Returns:

([<Collection "hello%02d.1001.exr [1]">, <Collection "hello01.%d.exr [1001]">], [])

Essentially it’s unable to detect which of the two is the frame number.
So you can make the pattern more strict:

import clique

print(clique.assemble(["hello01.1001.exr"], minimum_items=1, patterns=[clique.PATTERNS["frames"]]))

Returns:

([<Collection "hello01.%d.exr [1001]">], [])

But then it fails to allow hello_1001.exr or hello1001.exr or other variants like that.
So then you start putting in your own pattern, like:

    pattern = "[_.](?P<index>(?P<padding>0*)\\d+)\\.\\D+\\d?$"
    patterns = [pattern]
    collections, _remainder = clique.assemble(
        files,
        patterns=patterns,
        minimum_items=1)

Which allows at least the . and _ separator, but it’s still limiting in certain cases where someone may be throwing sequences like hello1001.exr at the pipeline, for whatever (bad) reason. :slight_smile:

So you make it looser:

files = ["hello1a.exr"]
pattern = "(?P<index>(?P<padding>0*)\\d+)\\.\\D+\\d?$"
patterns = [pattern]
collections, _remainder = clique.assemble(
    files,
    patterns=patterns,
    minimum_items=1)
print(collections)

With the only rule being that the frame pattern MUST live before a dot with some alphabetical characters somewhere behind it at the end of the string.

E.g.

files = ["foo1001.a.exr", "bar.1001.exr", "nugget_01.exr", "1001.exr", "9", "hello1001"]
pattern = "(?P<index>(?P<padding>0*)\\d+)\\.\\D+\\d?$"
patterns = [pattern]
collections, remainder = clique.assemble(
    files,
    patterns=patterns,
    minimum_items=1)
print(collections)
print("Remainder", remainder)
# [<Collection "foo%d.a.exr [1001]">, <Collection "bar.%d.exr [1001]">, <Collection "nugget_%02d.exr [1]">, <Collection "%d.exr [1001]">]
# Remainder ['9', 'hello1001']

The question mostly becomes, what do you consider to still be a valid detectable sequence.
It turns ambiguous rather quickly.

3

Hello Roy,

Thank you for your very detailed answer.
Maybe it should be the users who decide that? Ayon has many useful settings that makes it very configurable, there is already a version padding setting, and there are also fields that expect regular expressions, so I think that a sensible industry standard default, that can be overridden in the settings, would be the best way out of the ambiguity you mention.

What we have now is too conservative in my opinion, and it creates this fork on how Ayon treats publishes depending on how many frames are there. If I am publishing a shot I would expect the same pipeline regardless whether it contains one or more frames, right?

4

In different places in the pipeline, you can see something like

Can we still say: “it’s the same pipeline for both”?
Idk as I’m not the one who is checking the standards and naming things.
But, I think this can be fixed during frame collection, and make any later plugins expect the files to be a list.

5

Hello @mustafa_jafar,

Thank you for your answer. The problem here is that some core plugins like IntegrateAsset and ExtractReview will not treat as a sequence a one item list, even if you collect a single frame as a list of one element; to make it work we would need to update core modules like integrate.py or the many clique.assembly calls aforementioned, which will entail changing Ayon’s current definition of what a sequence is (currently the same definition as the one given by clique.assembly with the default arguments).
My suggestion would be to make this definition less conservative and put a re pattern that is an accepted industry standard, @BigRoy mentioned a few patterns in his answer. This could even be a settings in the core addon.