Datastore

If enabled (via global.datastore.enabled), in the Project Settings a configuration panel “Cloud storage & remote filesystems” appears. This allows users to mount remote filesystems into the particular project. This supports SSHFS, AWS S3 and Google Cloud Storage.

Under the hood, “Datastore” is a sidecar for the project, which mounts these filesystems according to their configuration in /data/[name] (where name is the name of the datastore). This mountpoint is propagated to the project container from the host. If the file ~/data is not taken, the project will automatically create a symlink to that global directory. Therefore, collaborators of the project can use and see this filesystem, but they do not know the secret, don’t see the raw configuration files, and also cannot interact with the actual process doing the FUSE mount. The “secret” is hidden in the user interface, it’s not sent to the web client.

• The “read-only” mode enabled the ro mount option for the FUSE mount.

• To make the filesystem perform well, it does a bit of caching, but only with a small timeout. This means if you give it a few seconds to read/write sync, it’s possible to do a bit of collaboration via the same mounted filesystem. It’s not really recommended, but possible. Also note that there is filesystem level polling of discovered directories in CoCalc’s projects, which means that remote changes to these files will eventually show up as well and update in an opened editor. Those projects are also cached on CoCalc’s side.

• Requests to support other remote filesystems are welcome, and if there is a robust tool and a way to easily configure them, we certainly consider adding it.

• Pro-tip: if a project is set to “Always Running”, you can use the SSHFS configuration in combination with the SSH Gateway to mount a directory from another project. This is a bit of a hack, but it works.