Node Setup

First off, you can run all services across all your nodes.

However, in paractice, it makes sense to partition into two groups: some nodes for services, and all other nodes for Projects.

This is a recommended set of node pools, the VM configuration and their size, to get started.

Node pool

Size

Configuration (each node)

service

2

2 CPU cores (x86/64), 16 GB RAM, 50 GB disk

projects

1

4 CPU cores (x86/64), 32 GB RAM, 100 GB disk (fast)

  • The CPU architecture must be x86/64 or (amd64)

  • The timezone must be UTC – i.e. that universal time zone, which has no local timezone attached – same for the database.

  • A larger fast disk for project nodes, due to the 10+ GB docker image for projects.

  • Scaling gives you additional context. In particular, the number of project nodes is what has to scale up – and/or their size.

  • If your cluster is small and Kubernetes services also run on serivce nodes, add at least a 3rd node or make them larger.

If you partition your nodes, use the labels mentioned below.

In addition, in order to make use of the Prepull service, you also need taints on the project pool. Do not set any taints, if you’re not enabling the prepull service.

If you go ahead with the default names, configure the pools like this:

  • “service” pool:

    • set the Kubernetes label to cocalc-role=services (that’s key=value)

  • “project” pool:

    • set the Kubernetes label to cocalc-role=projects (that’s key=value)

    • and if (and only if!) you intend to use the Prepull service, set the initial Kubernetes taints (this is key=value effect) to:

      • cocalc-projects-init=falseNoExecute

      • cocalc-projects=initNoSchedule

If you intend to use the prepull service, you also have to enable it in your my-values.yaml in manage.prepull.enabled: true (see Project Pods).