Skip to content

cluster.md

Latest commit

 

History

History
267 lines (180 loc) · 7.62 KB

cluster.md

File metadata and controls

267 lines (180 loc) · 7.62 KB
Raw

Container cluster

Kubernetes is an open-source orchestration platform for automating deployment, scaling, and management of containerized workloads across clusters of machines. With container, you can run local Kubernetes clusters for development and testing.

Why Kubernetes on container

Running Kubernetes locally with container gives you a streamlined workflow to build and test workloads before deploying to production:

  • Fast iteration. Create and destroy clusters in seconds. Test your deployments locally before pushing to a remote cluster.
  • Multiple cluster versions. Run different Kubernetes versions side-by-side without conflicts. Test compatibility across versions with ease.
  • Load your own images. Images built with container build can be loaded directly into your cluster. Build once, run anywhere — local Kubernetes or production.
  • Lightweight VMs. Clusters run as lightweight VMs on your Mac.
  • No special setup. Uses standard Kubernetes tooling (kubectl, kubeconfig) — the same CLI and configuration files you use with production clusters.

Quickstart

# Create a cluster
container cluster create

# Verify the cluster is running
container cluster list

# Interact with the cluster using kubectl
kubectl cluster-info
kubectl get nodes

# Clean up when done
container cluster delete

The cluster is automatically added to your ~/.kube/config, so standard Kubernetes tools just work.

Working with clusters

Create and list clusters

Create a cluster with container cluster create. By default, it creates a cluster named cluster-dev:

container cluster create

You can create multiple named clusters:

container cluster create --name staging
container cluster create --name testing

List all clusters and their status:

container cluster list

Cluster lifecycle

Create a cluster, use it with kubectl, and delete it when finished:

# Create a cluster
container cluster create --name my-cluster

# Use the cluster with kubectl
kubectl --context my-cluster get pods

# Delete the cluster when finished
container cluster delete --name my-cluster

Customize resources

Allocate CPU and memory based on your needs:

# Create a cluster with 4 CPUs and 8GB memory
container cluster create --name high-resource --cpus 4 --memory 8g

By default, clusters use 1/4 of your host's CPUs (minimum 2) and 1/4 of your host's memory (minimum 2GB).

Multi-node clusters

By default, container cluster create creates a single node that acts as both control plane and worker. Use --workers to add dedicated worker nodes instead:

# Create a cluster with a control plane and 3 worker nodes
container cluster create --name my-cluster --workers 3

When --workers is greater than 0, the control-plane node is tainted so pods only schedule onto worker nodes, matching typical multi-node cluster behavior. Worker nodes share the same --node-image, --cpus, and --memory settings as the control plane, and are named <cluster-name>-worker-<n>.

Access clusters with kubectl

Once a cluster is created, kubectl works normally:

# Use your created cluster
kubectl --context cluster-dev get pods
kubectl --context cluster-dev describe node

# Switch between clusters
kubectl config use-context staging

Load container images into your cluster

Images built with container build can be loaded directly into your Kubernetes cluster, so you can test them without pushing to a registry.

Build and load

Build an image and load it into your cluster:

# Build a local image
container build -t my-app:latest .

# Load the image into every node of the cluster
container cluster load-image my-app:latest

# Load the image into a single node instead
container cluster load-image --node cluster-dev-worker-1 my-app:latest

# Load the image into specific nodes only
container cluster load-image --node cluster-dev-worker-1 --node cluster-dev-worker-2 my-app:latest

The image is placed in the k8s.io namespace, making it available for pod scheduling:

apiVersion: v1
kind: Pod
metadata:
  name: test-app
spec:
  containers:
  - name: app
    image: my-app:latest
    imagePullPolicy: Never
  restartPolicy: Never

Load into named clusters

If you have multiple clusters, specify which one to load into:

container cluster load-image --name staging my-app:latest
container cluster load-image --name testing my-app:latest

Multi-architecture images

When loading multi-architecture images, specify the architecture if needed:

# Load the amd64 variant of a multi-arch image
container cluster load-image --platform linux/amd64 my-app:latest

Kubernetes node image

By default, clusters use kindest/node:v1.35.5, a Kubernetes-in-Docker image optimized for local development. You can use a different node image when creating a cluster:

container cluster create --node-image docker.io/kindest/node:v1.34.11

Custom CNI

By default, clusters install the bundled kindnet CNI for pod networking. Use --cni to apply a different CNI manifest instead:

container cluster create --name my-cluster --cni ./my-cni.yaml

The manifest must be a plain Kubernetes YAML file (the same shape kubectl apply -f expects), not a Helm chart.

Example: Cilium

Cilium is distributed as a Helm chart, so render a plain manifest from it first:

helm repo add cilium https://helm.cilium.io/
helm repo update
helm template cilium cilium/cilium --version 1.20.1 --namespace kube-system --set ipam.mode=kubernetes > cilium.yaml

--set ipam.mode=kubernetes avoids a CIDR conflict: the chart's default (cluster-pool, 10.0.0.0/8) overlaps kubeadm's pod subnet and service CIDR on these clusters.

Then create the cluster with that manifest:

container cluster create --name cilium-demo --cni ./cilium.yaml

Verify Cilium came up:

kubectl --context cilium-demo get pods -n kube-system
kubectl --context cilium-demo get nodes -o wide

Cluster cleanup

Remove a cluster and its data:

container cluster delete --name my-cluster

To have a cluster automatically remove itself when stopped, create it with --rm:

container cluster create --name temp-cluster --rm

Recovering a stopped cluster

If a cluster's control plane node stops (for example, after container stop), delete and re-create the cluster:

container cluster delete --name my-cluster
container cluster create --name my-cluster

Common workflows

Test a deployment locally before production

# Create a test cluster
container cluster create --name test

# Build your image locally
container build -t my-service:v1.0 .

# Load it into the test cluster
container cluster load-image --name test my-service:v1.0

# Deploy to the test cluster
kubectl --context test apply -f deployment.yaml

# Verify everything works
kubectl --context test logs deployment/my-service

# Clean up when done
container cluster delete --name test

One cluster per feature branch

# Create isolated clusters for concurrent development
container cluster create --name feature-auth
container cluster create --name feature-payments

# Work on each feature in isolation, test against its own cluster
container cluster load-image --name feature-auth my-service:feature-auth
container cluster load-image --name feature-payments my-service:feature-payments

See also