Container Orchestration Primer

Why This Matters

You have learned to build and run containers. You can spin up an nginx container, a database, a Python app -- all on a single machine. Things are going well.

Then the business grows. Your application now needs to handle ten times the traffic. You need to run 20 copies of your web server across 5 machines. When one copy crashes at 3 AM, something needs to restart it automatically. When you push a new version, you want to update containers one at a time with zero downtime. The containers need to find each other across machines. Health checks need to run. Logs need to be centralized. Secrets need to be distributed securely.

Doing all of this manually is a full-time job. Container orchestration automates it. An orchestrator takes your desired state ("I want 5 copies of my web server, always running, behind a load balancer") and continuously works to make reality match that desired state.

This chapter introduces orchestration concepts and gives you hands-on experience with k3s, a lightweight Kubernetes distribution that you can run on a single machine to learn the fundamentals.

Try This Right Now

Install k3s (a lightweight Kubernetes distribution) and run your first pod:

# Install k3s (takes about 30 seconds)
$ curl -sfL https://get.k3s.io | sh -

# Wait for it to be ready
$ sudo k3s kubectl get nodes
NAME       STATUS   ROLES                  AGE   VERSION
myhost     Ready    control-plane,master   30s   v1.29.2+k3s1

# Run your first pod
$ sudo k3s kubectl run my-nginx --image=nginx --port=80

# Check it is running
$ sudo k3s kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
my-nginx   1/1     Running   0          10s

You just deployed a container to a Kubernetes cluster. The cluster is managing it -- if the container crashes, Kubernetes will restart it automatically.

Why Orchestration?

Running a single container on a single machine is straightforward. Running containers at scale requires solving many problems simultaneously:

Without Orchestration:              With Orchestration:

"Container crashed at 3 AM"         Automatic restart
"How do I scale to 10 copies?"      kubectl scale --replicas=10
"Zero-downtime deployment?"         Rolling update, built-in
"Which host has capacity?"          Automatic scheduling
"How do containers find each       Service discovery + DNS
 other?"
"One host died, 5 containers       Auto-rescheduled to
 went down"                         healthy hosts
"Distribute secrets securely"       Secret management
"Health checking"                   Liveness + readiness probes

The Core Problems Orchestration Solves

Kubernetes Concepts

Kubernetes (often shortened to "k8s") is the industry standard for container orchestration. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation (CNCF).

The Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Control Plane                             │
│  ┌────────────┐ ┌────────────┐ ┌──────────────────────┐    │
│  │ API Server │ │ Scheduler  │ │ Controller Manager   │    │
│  │ (kube-     │ │ (decides   │ │ (ensures desired     │    │
│  │ apiserver) │ │ where pods │ │  state matches       │    │
│  │            │ │ run)       │ │  actual state)       │    │
│  └─────┬──────┘ └────────────┘ └──────────────────────┘    │
│        │                                                    │
│  ┌─────┴──────┐                                            │
│  │   etcd     │  (distributed key-value store)             │
│  │ (cluster   │  (stores all cluster state)                │
│  │  state)    │                                            │
│  └────────────┘                                            │
├─────────────────────────────────────────────────────────────┤
│                    Worker Nodes                              │
│  ┌──────────────────────┐  ┌──────────────────────┐        │
│  │      Node 1          │  │      Node 2          │        │
│  │ ┌──────┐ ┌──────┐   │  │ ┌──────┐ ┌──────┐   │        │
│  │ │Pod A │ │Pod B │   │  │ │Pod C │ │Pod D │   │        │
│  │ └──────┘ └──────┘   │  │ └──────┘ └──────┘   │        │
│  │                      │  │                      │        │
│  │ kubelet  kube-proxy  │  │ kubelet  kube-proxy  │        │
│  │ (agent)  (networking)│  │ (agent)  (networking)│        │
│  └──────────────────────┘  └──────────────────────┘        │
└─────────────────────────────────────────────────────────────┘

Key Kubernetes Objects

Let us walk through the fundamental building blocks.

Pods

A pod is the smallest deployable unit in Kubernetes. It is one or more containers that share networking and storage. In most cases, a pod runs a single container.

# pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-web
  labels:
    app: web
spec:
  containers:
    - name: nginx
      image: nginx:1.25
      ports:
        - containerPort: 80

Why pods and not just containers? Because sometimes closely related containers need to share resources. A web server and its log shipper, for example, can run in the same pod, sharing the filesystem and network namespace (just like Podman pods from Chapter 64).

Deployments

A Deployment manages a set of identical pods. You tell it how many replicas you want, and it ensures that many are always running. It also handles rolling updates.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: nginx
          image: nginx:1.25
          ports:
            - containerPort: 80
          resources:
            requests:
              memory: "64Mi"
              cpu: "100m"
            limits:
              memory: "128Mi"
              cpu: "250m"

This says: "I always want 3 copies of this nginx pod running. Each one gets at least 64MB RAM and 0.1 CPU, and must not exceed 128MB RAM and 0.25 CPU."

Services

A Service provides a stable network endpoint to access a set of pods. Pods are ephemeral -- they can be created, destroyed, and rescheduled anywhere. A Service gives you a stable IP address and DNS name that routes to whatever pods are currently healthy.

# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP

                   ┌─────────────────┐
                   │   web-service   │
                   │  10.43.0.100:80 │
                   │  (stable IP)    │
                   └────────┬────────┘
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
         ┌────────┐   ┌────────┐   ┌────────┐
         │ Pod 1  │   │ Pod 2  │   │ Pod 3  │
         │ nginx  │   │ nginx  │   │ nginx  │
         └────────┘   └────────┘   └────────┘

Service types:

• ClusterIP (default) -- accessible only within the cluster
• NodePort -- exposes on each node's IP at a static port (30000-32767)
• LoadBalancer -- provisions an external load balancer (in cloud environments)

Namespaces

Kubernetes namespaces provide logical separation within a cluster (not to be confused with Linux kernel namespaces, though they serve a similar conceptual purpose -- grouping and isolation).

# Default namespaces
$ kubectl get namespaces
NAME              STATUS   AGE
default           Active   1d
kube-system       Active   1d
kube-public       Active   1d
kube-node-lease   Active   1d

You can create namespaces for different teams, environments, or applications:

$ kubectl create namespace staging
$ kubectl create namespace production

Think About It: The word "namespace" appears in two different contexts in this book. Linux kernel namespaces (Chapter 62) isolate processes at the OS level. Kubernetes namespaces organize resources within a cluster. They share the concept of logical separation but operate at completely different layers.

Hands-On: k3s Kubernetes

k3s is a lightweight, production-ready Kubernetes distribution. It is perfect for learning, edge computing, IoT, and small-scale production use. It packages the entire Kubernetes control plane into a single ~70MB binary.

Installing k3s

# Install k3s
$ curl -sfL https://get.k3s.io | sh -

# Verify installation
$ sudo k3s kubectl get nodes

To avoid typing sudo k3s kubectl every time:

# Set up kubectl alias and kubeconfig
$ mkdir -p ~/.kube
$ sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
$ sudo chown $USER:$USER ~/.kube/config

# Now you can use kubectl directly
$ kubectl get nodes

Distro Note: k3s works on all major Linux distributions. On RHEL/CentOS, you may need to disable firewalld or open the required ports (6443 for API server, 8472 for flannel VXLAN).

Deploying Your First Application

Step 1: Create a Deployment

$ kubectl create deployment hello-web --image=nginx:1.25 --replicas=3
deployment.apps/hello-web created

Step 2: Watch the pods come up

$ kubectl get pods -w
NAME                         READY   STATUS              RESTARTS   AGE
hello-web-6b7b4f5c9d-abc12  0/1     ContainerCreating   0          2s
hello-web-6b7b4f5c9d-def34  0/1     ContainerCreating   0          2s
hello-web-6b7b4f5c9d-ghi56  0/1     ContainerCreating   0          2s
hello-web-6b7b4f5c9d-abc12  1/1     Running             0          5s
hello-web-6b7b4f5c9d-def34  1/1     Running             0          6s
hello-web-6b7b4f5c9d-ghi56  1/1     Running             0          7s

Press Ctrl+C to stop watching.

Step 3: Expose it as a Service

$ kubectl expose deployment hello-web --type=NodePort --port=80
service/hello-web exposed

$ kubectl get service hello-web
NAME        TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
hello-web   NodePort   10.43.0.200   <none>        80:31234/TCP   5s

The NodePort is 31234 (yours will differ). Access the web server:

$ curl http://localhost:31234
<!DOCTYPE html>
<html>
<head><title>Welcome to nginx!</title></head>
...

Step 4: Test self-healing

# Delete a pod (simulate a crash)
$ kubectl delete pod hello-web-6b7b4f5c9d-abc12
pod "hello-web-6b7b4f5c9d-abc12" deleted

# Watch Kubernetes immediately create a replacement
$ kubectl get pods
NAME                         READY   STATUS    RESTARTS   AGE
hello-web-6b7b4f5c9d-def34  1/1     Running   0          5m
hello-web-6b7b4f5c9d-ghi56  1/1     Running   0          5m
hello-web-6b7b4f5c9d-xyz99  1/1     Running   0          3s    ← new pod!

Kubernetes detected that the desired state (3 replicas) did not match the actual state (2 running), and it automatically created a new pod.

Step 5: Scale up and down

# Scale to 5 replicas
$ kubectl scale deployment hello-web --replicas=5
deployment.apps/hello-web scaled

$ kubectl get pods
NAME                         READY   STATUS    RESTARTS   AGE
hello-web-6b7b4f5c9d-def34  1/1     Running   0          10m
hello-web-6b7b4f5c9d-ghi56  1/1     Running   0          10m
hello-web-6b7b4f5c9d-xyz99  1/1     Running   0          5m
hello-web-6b7b4f5c9d-aaa11  1/1     Running   0          5s
hello-web-6b7b4f5c9d-bbb22  1/1     Running   0          5s

# Scale down to 2
$ kubectl scale deployment hello-web --replicas=2

$ kubectl get pods
NAME                         READY   STATUS    RESTARTS   AGE
hello-web-6b7b4f5c9d-def34  1/1     Running   0          11m
hello-web-6b7b4f5c9d-ghi56  1/1     Running   0          11m

Step 6: Rolling update

# Update to a new image version
$ kubectl set image deployment/hello-web nginx=nginx:1.26

# Watch the rollout
$ kubectl rollout status deployment/hello-web
Waiting for deployment "hello-web" rollout to finish: 1 out of 2 new replicas updated...
Waiting for deployment "hello-web" rollout to finish: 1 out of 2 new replicas updated...
deployment "hello-web" successfully rolled out

# Verify the new version
$ kubectl describe deployment hello-web | grep Image
    Image: nginx:1.26

# If something goes wrong, roll back
$ kubectl rollout undo deployment/hello-web
deployment.apps/hello-web rolled back

Deploying from YAML Files

While imperative commands are great for learning, production Kubernetes is managed declaratively with YAML files.

Create app.yaml:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: web
          image: nginx:1.25
          ports:
            - containerPort: 80
          resources:
            requests:
              memory: "64Mi"
              cpu: "50m"
            limits:
              memory: "128Mi"
              cpu: "200m"
          livenessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 3
            periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 80
  type: NodePort

Apply it:

# Create/update resources from the YAML file
$ kubectl apply -f app.yaml
deployment.apps/my-app created
service/my-app-service created

# View all resources
$ kubectl get all

# Delete resources defined in the file
$ kubectl delete -f app.yaml

Essential kubectl Commands

# Cluster info
$ kubectl cluster-info
$ kubectl get nodes
$ kubectl get namespaces

# Viewing resources
$ kubectl get pods                    # List pods
$ kubectl get pods -o wide            # List with more details
$ kubectl get deployments             # List deployments
$ kubectl get services                # List services
$ kubectl get all                     # List everything

# Inspecting resources
$ kubectl describe pod <pod-name>     # Detailed pod info
$ kubectl describe deployment <name>  # Detailed deployment info
$ kubectl logs <pod-name>             # View pod logs
$ kubectl logs -f <pod-name>          # Follow pod logs

# Interacting
$ kubectl exec -it <pod-name> -- bash # Shell into a pod
$ kubectl port-forward <pod> 8080:80  # Forward local port to pod

# Debugging
$ kubectl get events                  # Cluster events
$ kubectl top pods                    # Resource usage
$ kubectl top nodes                   # Node resource usage

Docker Swarm: A Brief Overview

Docker Swarm is Docker's built-in orchestration tool. It is simpler than Kubernetes but less feature-rich. It is worth knowing about because it is easy to set up and may be sufficient for smaller deployments.

# Initialize a swarm
$ docker swarm init

# Deploy a service with 3 replicas
$ docker service create --name web --replicas 3 -p 8080:80 nginx

# List services
$ docker service ls

# View service details
$ docker service ps web

# Scale the service
$ docker service scale web=5

# Update the image (rolling update)
$ docker service update --image nginx:1.26 web

# Remove the service
$ docker service rm web

# Leave the swarm
$ docker swarm leave --force

Kubernetes vs Docker Swarm

For most new projects, Kubernetes (or a lightweight variant like k3s) is the better investment. Docker Swarm is being maintained but is not seeing significant new development.

When You Need Orchestration (and When You Do Not)

This is a critical decision. Orchestration adds complexity, and complexity has costs.

You Probably Need Orchestration When:

• You run more than a handful of containers across multiple hosts
• You need zero-downtime deployments
• You need automatic failover when containers or hosts die
• Your application has multiple interconnected services (microservices)
• You need automatic scaling based on load
• You need to manage secrets and configuration centrally
• Multiple teams deploy to the same infrastructure

You Probably Do NOT Need Orchestration When:

• You run 1-5 containers on a single server
• Docker Compose handles your needs
• Your application is a monolith (one container)
• You do not need high availability
• You are a small team with simple infrastructure
• systemd + Podman (or Docker restart policies) is enough

Single server, 1-5 containers:
  → Docker Compose or Podman with systemd

Single server, 5-20 containers:
  → Docker Compose or Podman pods
  → Consider k3s if you want self-healing

Multiple servers, production:
  → Kubernetes (k3s, k0s, or managed K8s)

Large scale, multiple teams:
  → Full Kubernetes with proper infrastructure

Think About It: One of the most common mistakes in modern infrastructure is adopting Kubernetes before you need it. A single server running Docker Compose with proper backups and monitoring can handle a surprising amount of traffic. Start simple, add complexity only when the problems you face genuinely require it. The best orchestrator is the one that solves your actual problems without creating new ones.

Debug This

You deployed an application to Kubernetes, but the pods keep restarting:

$ kubectl get pods
NAME                    READY   STATUS             RESTARTS      AGE
my-app-abc12-def34      0/1     CrashLoopBackOff   5 (30s ago)   3m

Diagnosis:

# Check the pod logs
$ kubectl logs my-app-abc12-def34
Error: Cannot connect to database at db:5432

# Check events
$ kubectl describe pod my-app-abc12-def34
...
Events:
  Type     Reason     Message
  ----     ------     -------
  Normal   Pulled     Container image pulled
  Normal   Created    Container created
  Normal   Started    Container started
  Warning  BackOff    Back-off restarting failed container

The problem: The application expects a database service at db:5432, but there is no database pod or service named db in the cluster.

Fix: Either deploy the database first, or fix the application's database connection string:

# Deploy a database
$ kubectl create deployment db --image=postgres:16
$ kubectl expose deployment db --port=5432

# Or create the full stack with a YAML file that includes both

The CrashLoopBackOff status means Kubernetes is doing exactly what it should: restarting the failed container, but with increasing delays (back-off) because it keeps failing. Once the dependency is available, the next restart will succeed.

Cleaning Up

When you are done experimenting:

# Delete all resources you created
$ kubectl delete deployment hello-web my-app
$ kubectl delete service hello-web my-app-service

# To completely uninstall k3s
$ /usr/local/bin/k3s-uninstall.sh

Safety Warning: The k3s uninstall script removes all cluster data, including any persistent volumes. Make sure you do not need any data before running it.

What Just Happened?

┌─────────────────────────────────────────────────────────────┐
│                    CHAPTER RECAP                             │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Container orchestration automates: scaling, self-healing,  │
│  service discovery, rolling updates, and scheduling.       │
│                                                             │
│  Kubernetes is the industry standard orchestrator.          │
│                                                             │
│  Key Kubernetes objects:                                    │
│    Pod         → one or more containers                    │
│    Deployment  → manages replicas of pods                  │
│    Service     → stable network endpoint for pods          │
│    Namespace   → logical grouping of resources             │
│                                                             │
│  k3s is lightweight Kubernetes in a single binary.          │
│  kubectl is the CLI for interacting with the cluster.      │
│                                                             │
│  Desired state: you declare what you want.                 │
│  Kubernetes continuously reconciles actual → desired.      │
│                                                             │
│  Docker Swarm is simpler but less capable.                 │
│                                                             │
│  Not every project needs orchestration. Start with         │
│  Compose/systemd; adopt K8s when complexity demands it.    │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Try This

1. Deploy and scale: Install k3s, deploy an nginx application with 2 replicas, and expose it as a NodePort service. Verify you can access it. Scale to 5 replicas and confirm all pods are running.

2. Self-healing test: With 3 replicas running, delete one pod. Time how long it takes Kubernetes to create a replacement. Delete two pods simultaneously and observe the recovery.

3. Rolling update: Deploy nginx:1.25, then update to nginx:1.26 using kubectl set image. Watch the rollout with kubectl rollout status. Then roll back with kubectl rollout undo and verify the original version is restored.

4. YAML deployment: Write a complete YAML file that defines a Deployment (3 replicas) and a Service (NodePort). Include resource limits and health checks. Apply it with kubectl apply -f and verify everything works.

5. Debugging practice: Deliberately deploy an image that does not exist (e.g., nginx:nonexistent). Use kubectl describe pod, kubectl get events, and kubectl logs to diagnose why the pod is stuck in ImagePullBackOff.

6. Bonus Challenge: Deploy a two-tier application: a web frontend and a backend API as separate Deployments. Create a Service for the backend so the frontend can reach it by name. This demonstrates service discovery in Kubernetes.