10.52 - k3s Specifics

Why k3s exists: a Kubernetes distribution for edge/IoT/low-memory/single-box use. It is a REAL Kubernetes cluster (passes conformance), packaged lean.

What k3s bundles (the “batteries-included” list)

Component	What it does	Week 1 notes
containerd	container runtime (no Docker)	crictl is the runtime CLI if you ever need it
Flannel	CNI, pod overlay network (VXLAN)	pod IPs 10.42.x.x
CoreDNS	cluster DNS	Service names resolve via it
Traefik	default Ingress controller	DISABLED in this setup (Caddy owns 80/443); re-enable recipe in 10.41
servicelb	built-in LoadBalancer provider	DISABLED here; NodePort + port-forward cover week 1
local-path-provisioner	StorageClass local-path	default PVC storage, hostPath-backed
metrics-server	resource metrics for kubectl top + HPA	k3s ships it; give it ~60s to report

Also: embedded SQLite (single server) or etcd (HA mode) for cluster state; kube-proxy in iptables mode.

Server vs agent (the vocabulary)

• server node = control plane + runs workloads by default (like kubeadm control-plane with taint removed). For learning, ONE server node is a complete cluster.
• agent node = worker only (kubelet + kube-proxy + Flannel), no control plane. You add these when you want a real multi-node topology:

# on the server: note the token
sudo cat /var/lib/rancher/k3s/server/node-token
# on the agent (different machine):
curl -sfL https://get.k3s.io | K3S_URL=https://<server-ip>:6443 \
  K3S_TOKEN=<token> sh -
# then back on the server:
kubectl get nodes

Not needed this week; k3d (Option B in 10.11) simulates multi-node without extra machines.

Kubeconfig

• Server writes /etc/rancher/k3s/k3s.yaml (root-owned).
• The install adds --write-kubeconfig-mode 644 so you can read/copy it without sudo (done in 10.11).
• Context name: default. kubectl config current-context should say default and the cluster IP should be 127.0.0.1:6443 or your host IP.
• Server certs are valid for the loopback IP + service IPs; if you copy the kubeconfig to another machine, replace server: https://127.0.0.1:6443 with the host’s LAN IP and re-download or add a user entry. (Only matters if you admin the cluster from elsewhere.)

Useful k3s flags (install-time, via services.k3s.extraFlags)

Flag	Effect
--write-kubeconfig-mode 644	kubeconfig readable by your user
--disable traefik	drop default ingress controller
--disable servicelb	drop built-in LoadBalancer
--disable metrics-server	only if you hate metrics
--disable local-storage	drop local-path provisioner
--node-ip 0.0.0.0 / --bind-address	bind control plane to all interfaces
--secrets-encryption	encrypt Secrets at rest (etcd mode)
--disable coredns	if you bring your own DNS (don’t)

Managing the cluster as a service

systemctl status k3s
sudo journalctl -u k3s -f
sudo systemctl restart k3s              # full control plane restart
# per-component logs live under /var/log/containers (kubelet-managed)

K3s also ships a k3s CLI that bundles kubectl:

sudo k3s kubectl get nodes              # works even if your user kubeconfig is broken
sudo k3s crictl ps                      # runtime-level container list

Upgrades

# NixOS: bump the k3s package in configuration.nix and rebuild
# (or: sudo k3s kubectl apply -f <k3s upgrade manifest> on non-NixOS)
sudo nixos-rebuild switch --flake ~/nix#<host>
sudo systemctl restart k3s
kubectl get nodes

k3s upgrades keep cluster state (SQLite/etcd + manifests are on disk).

Backups (cluster state)

# SQLite mode (default single server) - back up the db directory:
sudo systemctl stop k3s
sudo cp -a /var/lib/rancher/k3s/server/db/ ~/k3s-db-backup/
sudo systemctl start k3s

(k3s etcd-snapshot only applies in HA/etcd mode.) The truly important backup is your manifests in ~/k8s/manifests/: since you apply everything declaratively, that folder is the real source of truth. Back it up with the rest of your dotfiles, and you can rebuild the whole lab from a fresh k3s install in minutes. That is the GitOps-ish habit this week builds toward.

Differences that bite people coming from vanilla k8s

1. Ports: k3s API is 6443 (same), but the kubelet listens on 10250 and the cluster uses 10.42.0.0/16 (pods) and 10.43.0.0/16 (services). Flannel in VXLAN mode.
2. kubectl get nodes shows no kubeadm-style control-plane taints; the server runs user workloads by default.
3. No kube-system/etcd pod in SQLite mode; cluster state is a file.
4. servicelb (klipper) is k3s’s answer to MetalLB; it assigns NodePort-like ports and can speak BGP in fancier setups.
5. Some things you’d install manually on vanilla k8s (ingress, metrics, storage) come preinstalled. The week-1 setup disables two of them; that is a learning choice, not a k3s requirement.

When you’d NOT use k3s

• Need a multi-tenant managed control plane: use a cloud (EKS/GKE/AKS).
• Need advanced CNI (Cilium eBPF), strict NetworkPolicies: k3s can be configured, but vanilla kubeadm/Cilium is more standard.
• HA control plane with production-grade etcd ops: k3s supports it (k3s server –cluster-init), but at that point managed k8s is usually cheaper to operate.

For a learning box and most homelab workloads, k3s is the right tool.