.. index:: Software Environment
.. _ops-software-env:

Software Environment
=====================

It is possible to customize the software environment of a project.
There are various ways to do this, which are described below.

Within a Project
-----------------

Users are able to install their own software packages in their projects.
A project is essentially a full Linux "user" environment, without elevated privileges.
This means all the usual ways to install software as a user are available, e.g. for :term:`Python`::

    pip install --user --upgrade [mypackage]

for :term:`R Software <R>`::

    install.packages("[mypackage]", lib="~/R")

or for :term:`GNU Autotools` based packages::

    ./configure --prefix=$HOME/.local
    make
    make install

Read more:

* CoCalc Doc: `Installing Python Packages <https://doc.cocalc.com/howto/install-python-lib.html>`_

.. _global-software:

Global Software
----------------

See :ref:`projects-software` about how to get read/write access to the global ``/ext`` mountpoint.
This is quite powerful, because it allows you to install software packages globally, which are then available to all projects.

.. note::

  Useful detail: if a file ``/ext/.bashrc`` exists, it is sourced by
  all projects via their local ``~/.bashrc`` file. This means it is
  possible to extend the path, configure aliases, etc. right there.
  If some users want to opt out for a project, they just have to
  comment or delete this from the bottom of their local ``~/.bashrc``
  file.

.. _custom-jupyter-kernels:

Custom Jupyter Kernels
-------------------------

It is possible to globally deploy customized Jupyter Kernels.
Each sub-directory of ``/ext/jupyter/kernels/`` could hold your own kernels,
where that ``/ext`` mountpoint is where the globally shared read-only
filesystem is mounted in all projects (see :ref:`projects-software`).

This works, because by default ``$JUPYTER_PATH`` is configured and points to that jupyter directory.
Globally installed kernels with the same directory name can be overwritten,
because that path takes precedence – e.g. ``python3``.

To check if a kernel is available:

#. Open a terminal in a project and run ``jupyter kernelspec list``.
#. Try to start it via ``jupyter console --kernel=[kernelname]``.

.. note::

    For a Python kernel, we suggest to add these parameters to the ``argv`` array in the ``kernel.json`` file:

    * ``"--HistoryManager.enabled=False"``: there is no need to record the history in a local database.
      In particular, if you're on an :term:`NFS` file-system, the underlying Sqlite database could cause
      problems in the form of "database is locked" errors, preventing the kernel from starting.
    * ``"--matplotlib=inline"``: to automatically load matplotlib


Ref.:

* `JUPYTER_PATH environment variable <https://docs.jupyter.org/en/latest/use/jupyter-directories.html#envvar-JUPYTER_PATH>`_
* CoCalc's documentation about `custom jupyter kernels <https://doc.cocalc.com/howto/custom-jupyter-kernel.html>`_


.. _project-image:

Project Image
---------------

The entire project image can be built from a Dockerfile by you.
This is the most flexible way to customize the software environment.
The main requirement is that it starts as a user `user` with the :term:`UID` ``2001``.
Look into the ``./project`` directory for more details.
Once your built project image is on your own registry,
:ref:`configure your CoCalc deployment <software-env>` to make it available to your users.