Node Setup

First off, you can run all services across all your nodes. For that base case, just add a few nodes to the cluster and do not enable the Prepull service!

The basic configuration for each node:

• The CPU architecture must be x86/64 or (amd64)

• The timezone must be UTC – i.e. that universal time zone, which has no local timezone attached – same for the database.

• A larger fast disk for project nodes, due to the 10+ GB docker image for projects.

• Scaling gives you additional context. In particular, the number of project nodes is what has to scale up – and/or their size.

Service/Project partitioning

In paractice, it makes sense to partition into two groups: a rather fixed number of nodes for services (Hubs, etc.) and a possibly adjustable number of nodes for Projects. Those nodes for project become additional taints, which prevent services (and other deployments from other namespaces in your cluster) to run there. The motivation behind that is to avoid interferance between projects, with their unpredictable load and resource consumption, to interfere with other deployments.

The following is a minimal configuration for node pools to get started. For production, it depends on your overall usage: for service nodes, a size of 3 and 4 CPU cores can’t hurt. Regarding projects, this entirely depends on how many are running and their individual configuration.

Node pool	Size	Configuration (each node)
service	2	2 CPU cores (x86/64), 16 GB RAM, 50 GB disk
projects	1	4 CPU cores (x86/64), 32 GB RAM, 100 GB disk (fast)

If your cluster is small and Kubernetes services also run on serivce nodes, add at least a 3rd service node or make them larger.

Node labels

To make the partitioning known, you have to add labels and optionally taints. They must be set with the configuration of the node pool or for each node that’s being added separately.

Node pool	Label key	Value
service	cocalc-role	services
projects	cocalc-role	projects

Prepull service

If you intend to use the Prepull service, you have to add initial taints to the project nodes (when they are created) and also enable the Prepull service.

Warning

Do not set any taints, if you’re not enabling the prepull service.

If you go ahead with the default names, configure the initial taints of the project pools nodes like this:

Node pool	Taint key	Value	Effect
projects	cocalc-projects-init	false	NoExecute
	cocalc-projects	init	NoSchedule

Then, enable Prepull in your my-values.yaml in manage.prepull.enabled: true (see Project Pods).

It only modifies existing taints, if the Prepull service is activated – this indicates the step in the lifecycle it is in.