k0s in Docker

A disposable multi-node Kubernetes cluster that runs as plain Docker containers on one machine: one k0s controller, three k0s workers, Calico as the CNI, and Traefik as the ingress controller, all declared in two files and brought up with one command. Meant for local testing of Helm charts, Ingress resources, and k0s's own bootstrap extensions, not for anything production-like.

Based on the multi-node docker-compose example in the k0s documentation, reduced to a single controller and adjusted where the reference did not work as-is (see Deviations).

Requirements

• Linux with Docker Engine and Docker Compose v2.30 or newer (the compose file uses post_start hooks). Tested with Docker Engine 29 and Compose 5.
• The containers run privileged and mount /dev/kmsg; port 80 on the host must be free.
• make, curl, and kubectl on the host. sudo only for the optional /etc/hosts entry.

Quick start

make cluster                      # bootstrap, wait until usable, write kubeconfig.yaml
make webapp                       # deploy the nginx sample behind an Ingress
echo "127.0.0.1 webapp.k0scluster.local" | sudo tee -a /etc/hosts

Open http://webapp.k0scluster.local/ in a browser, or without the hosts entry:

curl -H "Host: webapp.k0scluster.local" http://127.0.0.1/

Use the cluster from the host with export KUBECONFIG=./kubeconfig.yaml. Tear everything down with make decluster.

What is in the box

File	Purpose
docker-compose.yaml	The five containers: k0s-controller and k0s-worker1..3. Workers share one YAML anchor.
k0s.yaml	k0s cluster config: kine (SQLite) datastore, Calico via k0s's built-in integration, Traefik as a Helm extension installed at bootstrap.
manifests/webapp.yaml	Sample nginx Deployment, Service, and Ingress for webapp.k0scluster.local.
Makefile	The lifecycle targets below. make help lists them.
kubeconfig.yaml	Generated by make cluster, gitignored. Admin credentials for the running cluster.

The controller is control plane only (no --enable-worker), so kubectl get nodes lists the three workers. The worker join token is minted by a post_start hook on the controller and handed to the workers through a tmpfs volume, so there is no manual token step.

Make targets

Target	What it does
make cluster	docker compose up -d, wait for 3 Ready workers and running Traefik pods (fails after ~3 minutes otherwise), write kubeconfig.yaml, print status. Safe to re-run.
make decluster	docker compose down -v --remove-orphans and delete kubeconfig.yaml. Wipes all cluster state.
make reconfig	Apply k0s.yaml changes by recreating only the controller. Workers and cluster state are kept.
make webapp	Apply manifests/webapp.yaml. Not part of make cluster.
make kubeconfig	Rewrite kubeconfig.yaml pointed at https://127.0.0.1:6443.
make status	Nodes and all pods.

Pause and resume without losing anything: docker compose stop / docker compose start.

How traffic gets in

browser or curl
  -> host port 80
  -> k0s-worker1 port 30080          (Docker port mapping)
  -> Traefik Service, NodePort 30080 (kube-proxy, forwards to whichever worker's Traefik pod)
  -> Ingress rule for the Host header
  -> your Service -> your pod

Host port 80 is published on k0s-worker1, not on the controller: only workers run kube-proxy and Calico, so only a worker can serve a NodePort. It does not matter which worker the Traefik pod that handles the request lives on.

Traefik's Service is forced to type: NodePort (service.spec.type in the Helm values). The chart's LoadBalancer default never becomes ready without a load-balancer provider, which blocks helm --wait and leaves the release in pending-install.

Everything is plain HTTP on purpose. Traefik's HTTPS entrypoint is not published and there is no certificate management; if you need TLS, add cert-manager or bring your own certificates as a TLSStore, both of which are standard Traefik setups.

Changing things

Helm extension values (k0s.yaml, spec.extensions.helm): edit, then make reconfig. A plain docker compose up -d does nothing here, because k0s.yaml is bind-mounted into the container and Compose does not track a config file's content; recreating the container is what makes k0s read the new file. If the chart fails to render the new values, k0s uninstalls the existing release and keeps retrying, so the extension is down until the values are fixed. Confirm the outcome with the Helm release status (deployed is what you want):

docker exec k0s-controller k0s kubectl get secrets -n traefik -l owner=helm -o custom-columns='NAME:.metadata.name,STATUS:.metadata.labels.status'
docker logs k0s-controller 2>&1 | grep chart-traefik | tail -3

Cluster-level settings (spec.network.provider, spec.api, spec.storage): these bake into the datastore and certificates at first bootstrap and cannot change on a running cluster. make decluster then make cluster.

Worker containers (docker-compose.yaml): editing a worker service and running docker compose up -d recreates that container with an empty /var/lib/k0s (workers use anonymous volumes so the shared anchor does not make them share state). The worker rejoins under the same name; its pods show Unknown briefly until the rejoined kubelet picks them up again.

Kubeconfig: valid for a year and signed by the CA in the controller's volume, so it survives stops, restarts and make reconfig. Only make decluster invalidates it, and make cluster writes a new one.

Deviations from the upstream example

• privileged: true instead of cap_add + seccomp:unconfined. With capabilities alone, kubelet's ContainerManager failed at startup because /proc/sys stayed read-only inside the container.
• One controller without --enable-worker, and no k0s-lb Traefik container in front of the control plane. The load balancer only earns its place with several controllers; here the API port is published straight from the controller.
• Calico comes from k0s's built-in network.provider: calico rather than the tigera-operator Helm chart. The chart's eBPF bootstrap needs /sys mounted with shared propagation in every node container, which leaks kubelet mounts onto the host; the built-in integration needs nothing from the host.

Non-goals

TLS, multiple controllers, persistent workloads, anything resembling production hardening. It is a test bench that is cheap to destroy and rebuild.