I’m not (yet) a user of Ayon, but we (media university) are evaluating options for the VFX courses / projects.
Students will be working on projects on premise and from home. So we will be using the Site Sync functionality. If I understood correctly, Site Sync either uses SFTP or Google Drive (or any other cloud service you configure manually), right?
It’s a bit difficult to get permission to open more ports in the firewall and we for sure won’t get the budget to get Terabytes of space on a Google Drive.
We’re currently and very successfully using Syncthing (https://syncthing.net/) to sync the project data of different teams of students between the main university server and the student / teacher computers.
It has the major advantage that it doesn’t need any special ports opened or forwarded, it’s FOSS, the transfers are encrypted, compressed and it’s completely independent from a man-in-the-middle cloud service. Also it uses differential synchronization, where only blocks that aren’t already in sync get uploaded / downloaded which saves lots of time especially with large scene files, where only a fraction gets updated every time a version of the file gets updated.
All in all it’s fast, simple, secure and the source is freely available.
So my idea / question: Wouldn’t it be a worthwhile feature to have some sort of Syncthing-style “Provider” added in Site Sync?
Admittedly - you’d get little benefit of this I think since site sync really only syncs the published files - and thus they would usually always be “new” versions that need syncing, and not updates to existing files.
Note that non-published workfiles (that is files in the user’s work area) are not synced with Site Sync - which is where usually a lot of overwriting happens (saving their workfile with CTRL+S) happens a lot.
Nonetheless, if Syncthing does provide you the transfer interface you’d prefer - then I’d say you can definitely do so.
I also may want to emphasize that technically if your full project folder is synced, and on different locations just happens to be mapped to the same location (e.g. same drive letter) and Syncthing may just be quick enough to sync things for you without needing to ‘request’ the file (e.g. it’s not necessarily lazy) you may just get by without the Site Sync addon altogether. You’d basically be sharing a mapped “drive” at that point - it just happens to be kept in sync with Syncthing instead of being an actual network share like a studio environment might have. Whether that exactly suits your needs is up to you, but it may avoid the need to deal with Site Sync through AYON nor would you need a custom provider to be created for Syncthing too.
Thanks a lot for that answer. The main feature was the firewall “penetration” without any setup, compared to SFTP / Site Sync.
But indeed using Syncthing like it works right now to completely bypass Site Sync sounds very plausible.
How do you deal with the risk that someone at home mistakenly deletes some files, then SyncThing automatically deletes the same files at all other places.
And as the deletion can stay unnoticed for a while, your backups may be worthless.
Syncthing offers versioned backups of deleted / changed files. A makeshift version control so to say.
Even if someone deletes files, they can be restored to their prior state. Not as comfortable as with a proper version control like Git or SVN but better than losing data.
Despite having hundreds of students over the years, dozens of projects we never had trouble with losing data.