مناف حوراني

القائمة

كلّ الأدلّة

أدلّة التشغيل

A Kubernetes cluster on bare-metal VMs, with kubeadm

From a fresh Ubuntu VM to a working single control-plane cluster: static networking, containerd, the kernel modules nobody mentions until it fails, kubeadm init, Flannel, Helm and the dashboard.

kubeadmcontainerdFlannelHelmUbuntu10 خطوة · 45–90 min

هذه الأدلّة محفوظة بالإنجليزية. فالأوامر وملفّات الإعداد واحدة في كلّ اللغات، وتوزيعها على ثلاث نسخ لا يضيف إلّا احتمال تخلّف إحداها عن التحديث.

العناوين في هذا الدليل عناوين بديلة ضمن النطاق 10.0.0.0/24 والنطاق المستخدم هو example.dev. وما عدا ذلك منقول حرفياً عن الملاحظات الأصلية.

  1. 01

    Pin the node's address

    A control plane that changes IP is a control plane you rebuild. Give every node a static address before anything else — kubeadm bakes the advertise address into the cluster's certificates.

    sudo nano /etc/netplan/50-cloud-init.yaml
    bash
    network:
  version: 2
  ethernets:
    enp0s3:
      dhcp4: no
      addresses: [10.0.0.79/24]
      routes:
        - to: default
          via: 10.0.0.1
      nameservers:
        addresses: [1.1.1.1, 8.8.8.8]
    bash
    sudo netplan apply
  2. 02

    Disable swap, permanently

    The kubelet refuses to start with swap enabled. swapoff -a handles the running system; the fstab edit is what stops it coming back on the next reboot.

    bash
    sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

sudo apt update && sudo apt upgrade
  3. 03

    Install containerd via the Docker repository

    Remove anything the distro shipped first — mixed container runtimes are a long afternoon.

    bash
    for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do
  sudo apt-get remove $pkg
done
    Add Docker's official GPG key and repository
    bash
    sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] \
  https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" \
  | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
  4. 04

    Set SystemdCgroup — the one that bites

    containerd's default config uses the cgroupfs driver while the kubelet uses systemd. They disagree silently: the cluster comes up, then pods restart under memory pressure for reasons that make no sense. Generate the default config and flip this one value.

    bash
    containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
sudo nano /etc/containerd/config.toml
    Find SystemdCgroup and set it to true
    toml
    SystemdCgroup = true
    bash
    sudo systemctl restart containerd
sudo systemctl enable containerd
sudo systemctl status containerd
  5. 05

    Load the kernel modules and sysctls the CNI needs

    Without br_netfilter and IP forwarding, pod-to-pod traffic across nodes simply does not route — and nothing tells you why. Both are declared as files so they survive a reboot.

    bash
    sudo tee /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

sudo tee /etc/sysctl.d/kubernetes.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system
  6. 06

    Install kubelet, kubeadm and kubectl

    bash
    sudo apt-get install -y apt-transport-https ca-certificates curl gpg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key \
  | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /' \
  | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl enable --now kubelet
  7. 07

    Initialise the control plane

    The pod CIDR has to match what the CNI expects — 10.244.0.0/16 is Flannel's default, and changing one without the other produces a cluster where nothing ever becomes Ready.

    bash
    sudo kubeadm init \
  --apiserver-advertise-address=10.0.0.55 \
  --pod-network-cidr=10.244.0.0/16 \
  --kubernetes-version=v1.33.3
    Take ownership of the kubeconfig
    bash
    mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
  8. 08

    Install the CNI, then Helm

    Nodes stay NotReady until a network plugin is installed. That is expected, not a fault.

    bash
    kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
  9. 09

    Reach the cluster from your workstation

    Run kubectl from the desktop, not the server. Copy admin.conf across and install the client binary.

    Windows
    bash
    curl.exe -LO "https://dl.k8s.io/release/v1.33.0/bin/windows/amd64/kubectl.exe"
curl.exe -LO "https://dl.k8s.io/v1.33.0/bin/windows/amd64/kubectl.exe.sha256"
kubectl version --client
  10. 10

    Expose the dashboard and mint a token

    The dashboard ships as a ClusterIP service, which is unreachable from outside the cluster. Switching it to NodePort is the quickest way in on a lab cluster.

    bash
    kubectl -n kubernetes-dashboard edit svc kubernetes-dashboard-kong-proxy
# change  type: ClusterIP  ->  type: NodePort

kubectl -n kubernetes-dashboard get svc kubernetes-dashboard-kong-proxy
# then browse to https://<node-ip>:<nodeport>
    admin-user.yaml
    yaml
    apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: admin-user
    namespace: kubernetes-dashboard
    bash
    kubectl apply -f admin-user.yaml
kubectl -n kubernetes-dashboard create token admin-user