Software Environment

It is possible to customize the software environment of a project. There are various ways to do this, which are described below.

Within a Project

Users are able to install their own software packages in their projects. A project is essentially a full Linux “user” environment, without elevated privileges. This means all the usual ways to install software as a user are available, e.g. for Python:

pip install --user --upgrade [mypackage]

for R Software:

install.packages("[mypackage]", lib="~/R")

or for GNU Autotools based packages:

./configure --prefix=$HOME/.local
make
make install

Read more:

Global Software

See Projects Software about how to get read/write access to the global /ext mountpoint. This is quite powerful, because it allows you to install software packages globally, which are then available to all projects.

Note

Useful detail: if a file /ext/.bashrc exists, it is sourced by all projects via their local ~/.bashrc file. This means it is possible to extend the path, configure aliases, etc. right there. If some users want to opt out for a project, they just have to comment or delete this from the bottom of their local ~/.bashrc file.

Custom Jupyter Kernels

It is possible to globally deploy customized Jupyter Kernels. Each sub-directory of /ext/jupyter/kernels/ could hold your own kernels, where that /ext mountpoint is where the globally shared read-only filesystem is mounted in all projects (see Projects Software).

This works, because by default $JUPYTER_PATH is configured and points to that jupyter directory. Globally installed kernels with the same directory name can be overwritten, because that path takes precedence – e.g. python3.

To check if a kernel is available:

  1. Open a terminal in a project and run jupyter kernelspec list.

  2. Try to start it via jupyter console --kernel=[kernelname].

Note

For a Python kernel, we suggest to add these parameters to the argv array in the kernel.json file:

  • "--HistoryManager.enabled=False": there is no need to record the history in a local database. In particular, if you’re on an NFS file-system, the underlying Sqlite database could cause problems in the form of “database is locked” errors, preventing the kernel from starting.

  • "--matplotlib=inline": to automatically load matplotlib

Ref.:

Project Image

The entire project image can be built from a Dockerfile by you. This is the most flexible way to customize the software environment. The main requirement is that it starts as a user user with the UID 2001. Look into the ./project directory for more details. Once your built project image is on your own registry, configure your CoCalc deployment to make it available to your users.