Publishing annotations

The OpenRV PR started an implementation for providing comments and annotations on published content using an OpenPype integrated host (which is OpenRV itself). This brings up the question, say we’d generate annotations/comments on top of footage, a sequence, or whatever. How would we want those to be published?

The PR prototype

The current draft in the PR seems to:

Create an annotation instance per frame annotation that was drawn and publish it to the asset from which the source footage was loaded. So say you loaded char_heroreviewMain, then loading that and creating an annotation would result in a publish going to char_heroannotateChar_Hero_1001.

However, this means that you’d get MANY publish instances and with that MANY publish subsets if you create say 50 drawovers over some footage. Looking at what subsets have been up to now that seems very granular and also those individual annotation subsets have no link with the ‘reviewMain’ instance whatsoever.

So how would we want this to work?

Say I’d load four video files:

  • shots / sh010 / compRenderMain
  • shots / sh010 / reviewMain
  • shots / sh020 / compRenderMain
  • shots / sh030 / compRenderMain

In my OpenRV session I would then start adding comments and adding drawover annotations. What would I like the publisher to do? Should those turn into published subsets? Or should it do something completely different, etc.

Would we expect annotations or comments like this to even be a published subset/annotation in the OpenPype database?

TL;DR How do we want to publish drawn annotations over published loaded containers?

Would love to know your thoughts!

Where the published content goes depends on the which context OpenRV is in, similar to other hosts.

If I launched shots/sh020/animation, then loaded in shots/sh010/reviewMain, shots/sh020/reviewMain and shots/sh030/reviewMain to review in context, then any annotations would be relevant to sh020/animation.
If I launched shots/editorial and loaded in the same subsets as above to review all (3) shots, then any annotations would be relevant to shots/editorial.

Although OpenRV probably could be used as a simple editorial host, I would not aim the initial integration for that, meaning we should not aim to be publishing to any assets outside of the context.

Thanks @tokestuartjepsen.

Say you’d draw 15 annotations and leave 20 comments. How do you envision having access to those notes when published in e.g. editorial/annotationDaily. What should happen?

Should it publish a single subset that is the full review session in its entirety? Should it publish a subset per comment?

Say you’d want to find all comments for a particular subset or a particular subset version? Or all notes for a particular asset or task? How would we store it so it’s manageable and we can also query the data in a meaningful way. Those are all things to consider. But for now I’d say what is the minimum viable product that would actually make a review like this worthwhile?

If anyone dares to make some psuedocode or schematic overview that would be perfect!

I think we should publish it as a subset, link it together, but not per frame. Per your example - annotations and comments for sh010/compRenderMain/v001 should go to sh010/annotationsCompRenderMain/v001. But publishing it per frame seems to me like overkill, maybe something like json file describing time relations for annotations and comment (or even OTIO file perhaps? - I am not sure about specific support there, but I know that there is some geometry (box) support and maybe the annotations can be rendered as overlays…). With links we can track the relation between subsets and it will allow us to handle multiple versions of annotation per one version of subset for example.

Shame this never got any traction for this exact scenario;

Yeah, single subset that stores everything from an OpenRV session.

Think @antirotor suggestion of OTIO might a good idea since we can lean on an official framework. Basically treating it as an editorial output.

For reference, Here’s the new PR for OpenRV.

This has been closed. I guess, it waits to be ported to ayon-core.

Also, For reference… Here’s our WIP OpenRV Repo.

1 Like

I remember really solid annotation workflow which had been done via OpenReviewIO. Here are some references:

  1. Test data for .orio in Stax python/tests/data/OpenReviewIO · master · superprod / Stax Suite / stax · GitLab

  2. OpenReviewIO adaptation in Stax python/stax/utils/ · master · superprod / Stax Suite / stax · GitLab

@Tilix4 has done this couple years ago.

1 Like