k8s the hard way

Deploying Kubernetes on Bare Metal the Hard Way

November 27, 2024

kubernetes security devops containers

Here’s a comprehensive checklist for setting up a Kubernetes Master Node. Follow this step-by-step process to ensure a smooth setup:


Preliminary Setup

  1. System Requirements:

    • OS: Ubuntu 20.04+, CentOS 7+, Debian 10+.
    • CPU: Minimum 2 cores.
    • RAM: At least 2 GB (4 GB recommended).
    • Disk: Minimum 10 GB free space.
    • Network: Stable connectivity with a static or reserved IP address.
  2. Disable Swap:

    sudo swapoff -a
    sudo sed -i '/ swap / s/^/#/' /etc/fstab
    
  3. Install Required Packages:

    • Update system:
      sudo apt update && sudo apt upgrade -y
      
    • Install dependencies:
      sudo apt install -y apt-transport-https ca-certificates curl software-properties-common
      
  4. Configure Kernel Modules:

    sudo modprobe overlay
    sudo modprobe br_netfilter
    
    • Persist the modules:
      echo -e "overlay\nbr_netfilter" | sudo tee /etc/modules-load.d/k8s.conf
      
  5. Set Kernel Parameters for Kubernetes: Add the following to /etc/sysctl.d/k8s.conf:

    net.bridge.bridge-nf-call-iptables = 1
    net.bridge.bridge-nf-call-ip6tables = 1
    net.ipv4.ip_forward = 1
    

    Apply the changes:

    sudo sysctl --system
    

Install Docker and Container Runtime

  1. Install Docker (Optional if using containerd):

    • Add Docker’s official GPG key:
      curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
      
    • Add the Docker repository:
      sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
      
    • Install Docker:
      sudo apt install -y docker-ce docker-ce-cli containerd.io
      
  2. Install containerd (Preferred for Kubernetes):

    • Install:
      sudo apt install -y containerd
      
    • Configure containerd:
      sudo mkdir -p /etc/containerd
      containerd config default | sudo tee /etc/containerd/config.toml
      
    • Enable SystemdCgroup in /etc/containerd/config.toml:
      SystemdCgroup = true
      
    • Restart containerd:
      sudo systemctl restart containerd
      

Install Kubernetes Components

  1. Add Kubernetes Repository:

    curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
    sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
    
  2. Install kubeadm, kubelet, and kubectl:

    sudo apt install -y kubelet kubeadm kubectl
    sudo apt-mark hold kubelet kubeadm kubectl
    
  3. Check Installed Versions:

    kubeadm version
    kubectl version --client
    kubelet --version
    

Initialize the Kubernetes Cluster

  1. Initialize Master Node:

    kubeadm init --apiserver-advertise-address=<STATIC_IP> --pod-network-cidr=10.244.0.0/16
    

    Replace <STATIC_IP> with the master node’s IP.

  2. Set Up kubeconfig for kubectl:

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
  3. Verify Node Status:

    kubectl get nodes
    

Install Pod Network

  1. Deploy a Network Add-on (Choose One):

    • Flannel:
      kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
      
    • Calico:
      kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
      
  2. Verify Pod Network Setup:

    kubectl get pods --all-namespaces
    

Post-Installation Setup

  1. Ensure Node Is Ready: Confirm the master node is in the Ready state:

    kubectl get nodes
    
  2. Enable Scheduling on Master Node (Optional for Single Node Cluster):

    kubectl taint nodes --all node-role.kubernetes.io/master-
    
  3. Backup Cluster Configurations: Save the kubeadm init join command for adding worker nodes later.

  4. Check API Server Logs:

    journalctl -u kubelet -f
    

Summary Checklist

Step Command/Description Status
Disable Swap swapoff -a
Install Docker/Containerd Install and configure container runtime
Load Kernel Modules Configure overlay, br_netfilter, etc.
Install Kubernetes Components kubeadm, kubelet, kubectl
Initialize Master Node kubeadm init
Configure kubectl Copy and set permissions for kubeconfig
Install Pod Network Apply Flannel or Calico
Verify Cluster kubectl get nodes and kubectl get pods --all-namespaces

Following this checklist will ensure your Kubernetes master node is correctly set up and operational. Let me know if you encounter any issues!