Bootstrap a k3s cluster

From one to three Ubuntu machines to a working Kubernetes: k3s pinned to a version, kubectl and helm on your laptop, the bundled Traefik ingress, cert-manager with a ClusterIssuer, and a hello-world behind HTTPS to prove the whole chain.

k3s is Kubernetes in one binary: the API server, the scheduler, the kubelet, a datastore and an ingress controller, installed by one script in under a minute. This page takes one Ubuntu machine to a cluster you drive from your laptop, with certificates issued automatically, and proves it with a hello-world behind HTTPS at .

Before you start

On each node, as root:

$swapoff -a
$sed -i '/ swap / s/^/#/' /etc/fstab
$timedatectl set-ntp true

$ufw allow 22/tcp
$ufw allow 80/tcp
$ufw allow 443/tcp
$ufw allow 6443/tcp
$ufw allow from 10.42.0.0/16 to any
$ufw allow from 10.43.0.0/16 to any

Install the first server

$curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION= sh -s - server \
$  --tls-san  --write-kubeconfig-mode 644

If the node stays NotReady

Give it a minute: the node is Ready once flannel and CoreDNS run. If it stays NotReady, journalctl -u k3s --no-pager | tail -50 usually names the cause: swap still on, a firewall dropping 8472/udp, or a leftover Docker or containerd install on the machine. Ubuntu cloud images with apparmor are fine; Raspberry Pi images need cgroups enabled in cmdline.txt, see the k3s docs.

Kubeconfig, kubectl and helm on the laptop

$mkdir -p ~/.kube
$ssh root@ cat /etc/rancher/k3s/k3s.yaml | sed "s/127.0.0.1//" > ~/.kube/config
$chmod 600 ~/.kube/config

Sensitive command — runs a remote script. Review before running.

$curl -LO "https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
$sudo install -m 755 kubectl /usr/local/bin/kubectl
$curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Check

$kubectl get nodes --no-headers | awk '{print $2}' | sort -u

Expected output

Ready

If kubectl says x509: certificate is valid for …

The API certificate does not include : the --tls-san flag was missing or given another address. Re-run the install command on the first node with the right --tls-san; the script is idempotent and only rewrites the unit and the certificate.

Ingress: Traefik and ServiceLB

$kubectl -n kube-system get svc traefik

Check

$kubectl -n kube-system rollout status deploy/traefik

Expected output

deployment "traefik" successfully rolled out

cert-manager and a ClusterIssuer

$helm repo add jetstack https://charts.jetstack.io --force-update
$helm upgrade --install cert-manager jetstack/cert-manager \
$  --namespace cert-manager --create-namespace --set crds.enabled=true --wait

Check

$kubectl -n cert-manager rollout status deploy/cert-manager-webhook

Expected output

deployment "cert-manager-webhook" successfully rolled out

clusterissuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: 
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: traefik

$kubectl apply -f clusterissuer.yaml

Storage: the local-path class

$kubectl get storageclass

Hello world behind HTTPS

hello.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
spec:
  replicas: 1
  selector:
    matchLabels: { app: hello }
  template:
    metadata:
      labels: { app: hello }
    spec:
      containers:
        - name: whoami
          image: traefik/whoami:v1.10
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: hello
spec:
  selector: { app: hello }
  ports:
    - port: 80
      targetPort: 80

hello-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  ingressClassName: traefik
  tls:
    - hosts: [""]
      secretName: hello-tls
  rules:
    - host: 
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: hello
                port: { number: 80 }

$kubectl apply -f hello.yaml -f hello-ingress.yaml
$kubectl get certificate hello-tls -w

Check

$kubectl wait --for=condition=Ready certificate/hello-tls --timeout=120s

Expected output

certificate.cert-manager.io/hello-tls condition met

Check

$curl -sI https:// | head -1

Expected output

HTTP/2 200

If the certificate stays not Ready

In order of likelihood:

does not resolve to a node yet, or resolves to a private address Let's Encrypt cannot reach. dig +short the name from outside.
Port 80 is closed somewhere (ufw, the provider's firewall): HTTP-01 needs it, even though users only use 443.
Rate limit hit after too many attempts: kubectl describe order says so. Use the staging endpoint until the setup is right.
The Challenge is pending with a wrong status code 404: Traefik is not serving the temporary Ingress, check kubectl -n kube-system logs deploy/traefik.

Upgrade, uninstall

Upgrading k3s is re-running the install script with a newer version; uninstalling is one script per node.

Done

One node, driven from your laptop, an ingress on every node, certificates that issue and renew themselves, and a StorageClass for the database. The hello-world can go:

$kubectl delete -f hello-ingress.yaml -f hello.yaml

The next page deploys indicat on this cluster: namespace, secrets, PostgreSQL, the application with its Ingress at , and an autoscaler.

Bootstrap a k3s cluster

Before you start

Install the first server

Kubeconfig, kubectl and helm on the laptop

Ingress: Traefik and ServiceLB

cert-manager and a ClusterIssuer

Storage: the local-path class

Hello world behind HTTPS

Upgrade, uninstall

Done

Did everything work?