Flamenco - Config - Cloud Storage

A studio may use something like Google Drive, Dropbox, Nextcloud etc to host their entire set of projects, assets and output. It may also include other business or project materials and makes remote-collaboration easier.

Flamenco assumes local, fast storage for Flamenco Jobs and Workers where files are updated quickly as a project.

Flamenco itself does not support cloud storage natively and most do not behave like block storage and are object-based files that take seconds to check and transfer/sync.

The following diagram shows NextCloud Server and Clients to sync between cloud storage and shared storage used by Flamenco.

Using OpenVPN and Cloud Storage client you can create a workaround to allow users to submit work and assets via Cloud Storage but Flamenco Workers use a local shared storage (over OpenVPN connection).

It is up to you if the Workers on the local network to the shared storage also uses Cloud Storage clients. One of the Workers, or another computer, can run the cloud storage client but create its mirrored folders on the NFS mount.

The success of this is very much dependent upon network stability and speed but you know your Workers will find the same project files once the Cloud Storage client syncing NFS is up to date and sync’d.

NextCloud

You can pay for hosted or go the self-host route with your own NextCloud Server https://nextcloud.com/install/.

Co-Located Self-Hosted NextCloud

If you host your own NextCloud Server then it can be co-located and use the shared-storage as its own storage. This would be fast and eliminate the extra sync.

Platform Paths

Using Flamenco two-way variables you can use Cloud Storage client local (mirrored) path in Blender projects and have Flamenco path-swap.

This will largely depend on how you setup your cloud storage client but the general idea is to have all workstations write out to the cloud storage mirrored folders which will be sync’d to the cloud storage.

The NFS cloud client will then sync all changes found in cloud storage server. Depending on the amount of changes this could take some time before Flamenco Jobs can be launched. (There is no synchronisation between local shared storage (ie: NFS) being up to date and Jobs being scheduled.)