.. index:: Node setup
.. _project-node-setup:

Node Setup
================

First off, you can run all services across all your nodes.
For that base case, just add a few nodes to the cluster and do not enable the :ref:`prepull` service!

The basic configuration for each node:

- The **CPU architecture** must be ``x86/64`` or (``amd64``)
- The **timezone must be UTC** – i.e. that universal time zone, which has no local timezone attached – same for the :ref:`database <database-timezone>`.
- A **larger fast disk for project nodes**, due to the 10+ GB docker image for projects.
- :ref:`scaling` gives you additional context. In particular, the number of project nodes is what has to scale up – and/or their size.

Service/Project partitioning
-----------------------------------

In paractice, it makes sense to partition into two groups:
a rather fixed number of nodes for services (:ref:`hubs`, etc.) and a possibly adjustable number of nodes for :ref:`projects`.
Those nodes for project become additional taints,
which prevent services (and other deployments from other namespaces in your cluster) to run there.
The motivation behind that is to avoid interferance between projects, with their unpredictable load and resource consumption,
to interfere with other deployments.

The following is a *minimal* configuration for node pools to get started.
For production, it depends on your overall usage: for service nodes, a size of 3 and 4 CPU cores can't hurt.
Regarding projects, this entirely depends on how many are running and their individual configuration.

+-------------+-------+--------------------------------------------------------------+
| Node pool   | Size  | Configuration (each node)                                    |
+=============+=======+==============================================================+
| service     | 2     | 2 CPU cores (x86/64), 16 GB RAM, 50 GB disk                  |
+-------------+-------+--------------------------------------------------------------+
| projects    | 1     | 4 CPU cores (x86/64), 32 GB RAM, 100 GB disk (fast)          |
+-------------+-------+--------------------------------------------------------------+

If your cluster is small and :term:`Kubernetes` services also run on serivce nodes, add at least a 3rd service node or make them larger.

Node labels
------------------------------

To make the partitioning known, you have to add labels and optionally taints.
They must be set with the configuration of the node pool or for each node that's being added separately.


+-------------+-------------------------------------+--------------------------------+
| Node pool   | Label key                           | Value                          |
+=============+=====================================+================================+
| service     | ``cocalc-role``                     | ``services``                   |
+-------------+-------------------------------------+--------------------------------+
| projects    | ``cocalc-role``                     | ``projects``                   |
+-------------+-------------------------------------+--------------------------------+


Prepull service
----------------------------

If you intend to use the :ref:`prepull` service, you have to add initial taints to the project nodes (when they are created) and also enable the Prepull service.

.. warning::

  Do **not** set any taints, if you're not enabling the prepull service.

If you go ahead with the default names, configure the initial taints of the project pools nodes like this:

+-------------+--------------------------+----------------+----------------------+
| Node pool   | Taint key                | Value          | Effect               |
+=============+==========================+================+======================+
| projects    | ``cocalc-projects-init`` | ``false``      | ``NoExecute``        |
+-------------+--------------------------+----------------+----------------------+
|             | ``cocalc-projects``      | ``init``       | ``NoSchedule``       |
+-------------+--------------------------+----------------+----------------------+


Then, **enable Prepull** in your :ref:`my-values.yaml` in ``manage.prepull.enabled: true`` (see :ref:`config-project-pods`).

It only modifies existing taints, if the :ref:`prepull` service is activated – this indicates the step in the lifecycle it is in.