Thanks for this proposal @BigRoy ! I can definitely see this being useful for us down the line and I’m currently also investigating on the space of “manual” headless publishing toolsets for running transcode and extract review plugins on-demand.
The way I have seen what you propose implemented in the past isn’t through “workfiles” but through the subset/product (I will call it “products” from now on following the great news of Renaming Subset to Product) dependency links and the data contained within. So for example, on an “animated cache” product, you hold the (input) dependencies of other products, such as a “rig” and “anim curves” products. With that information, you could set up a framework that once the (input) dependencies get a new version published (or any event like a specific status is set), it automatically autogenerates the products that depend on it. On an ideal pipeline you don’t even need a specific workfile to run the publish from as the “input dependencies” products should describe all the dependencies required to re-create that product, so you should be able to load them all on the compatible host and run the code that exports/publishes a new version of the product (or potentially using templated workfiles that you set as extra input processes as @tokestuartjepsen suggested). However, for some products there might be other input dependencies that aren’t tracked on the AMS and only exist on the workfile that was used to create them… on which case, we should be able to grab the “workfile” dependency and use that instead like you are proposing.
The idea behind this is that you can define processes (we called them “autogens”) between asset dependencies and have more granular control over what gets automatically generated (although as I said you can make it so it’s a push mechanism instead and set more complex flow filters) and not just “workfiles” as those can be quite generic and could be trickier to know all the products defined inside and which workfile to use (with “products” you are confident that’s your end goal). On the other hand, the asset dependencies would auto propagate through your whole dependency graph automatically… so if the “anim cache” product is another input dependency of another product (i.e., “render”) you could set it up so once it got automatically updated, it triggers the next autogeneration and so on (i.e., an update of your animation curves could end up automatically giving you a comped image with the animated cache).
All of these processes would be triggered by a dedicate machine that basically just tracks all these dependency updates and delegates the jobs to the farm (or runs them locally).
Does OpenPype (or perhaps only supported in Ayon) have the notion of input vs output link dependencies on the products?