Bootstrap Kubernetes clusters with Cluster API, Talos and Proxmox.
This repo sets up a k3s management cluster, installs Cluster API and performs bootstrapping of Talos workload clusters on top of Proxmox infrastructure.
The workload cluster is bootstrapped by Flux using the k8s-infrastructure repository.
- k8s-bootstrap: Kubeadm based installation.
- k8s-bootstrap-talos: Talos based installation.
Install the k3s management cluster:
sudo bash scripts/cluster.shCredentials for the management cluster will be available at the /etc/rancher/k3s/k3s.yaml file.
To install Cluster API and bootstrap a workload cluster, set the GITHUB_TOKEN environment variable and run:
sudo \
GITHUB_TOKEN="$GITHUB_TOKEN" \
bash scripts/bootstrap.shTo configure access to the workload cluster, set the WORKLOAD_CLUSTER environment variable and run:
WORKLOAD_CLUSTER="$WORKLOAD_CLUSTER" \
bash scripts/workload-cluster-credentials.shCredentials will be available in the kubeconfig and talosconfig files in the current directory. Set the following aliases to temporarily use the workload cluster:
alias k="kubectl --kubeconfig=kubeconfig"
alias t="talosctl --talosconfig=talosconfig"To deploy the infrastructure components in the workload cluster, set the GITHUB_TOKEN environment variable and run following script to bootstrap the k8s-infrastructure repository using flux:
sudo \
GITHUB_TOKEN="$GITHUB_TOKEN" \
bash scripts/workload-cluster-bootstrap.shTo upgrade the management cluster, set the K3S_VERSION environment variable to the desired version and run:
sudo \
K3S_VERSION="v1.33.1+k3s1" \
bash scripts/cluster.shTo take an on-demand backup of the management cluster and push it to object storage, run:
sudo \
S3_ACCESS_KEY="<access-key>" \
S3_SECRET_KEY="<secret-key>" \
bash scripts/backup.shTo restore a backup in a existing cluster, select the backup to restore with SNAPSHOT_NAME and and run:
sudo \
S3_ACCESS_KEY="<access-key>" \
S3_SECRET_KEY="<secret-key>" \
SNAPSHOT_NAME="on-demand-management-1749986019" \
bash scripts/restore.shTo uninstall the k3s management cluster, run:
k3s-uninstall.shProxmoxMachineTemplate and TalosConfigTemplate resources are entirely immutable. Any change to their spec (disk size, RAM, CPU, strategic patches, etc.) will be rejected by the webhook:
ProxmoxMachineTemplate.infrastructure.cluster.x-k8s.io "compute-large" is invalid: spec: Forbidden: ProxmoxMachineTemplate is immutable
To update a template, use one of these approaches:
A) New name (safer, zero-downtime rollout):
- Create a new template (e.g.
compute-large-v2) with the updated spec - Update the
infrastructureRef.name(forProxmoxMachineTemplate) and/orbootstrap.configRef.name(forTalosConfigTemplate) in the affectedMachineDeploymentresources - CAPI will rollout new machines using the new template and delete the old ones
- Delete the old template after the rollout completes
B) Same name (simpler, brief reconciliation gap):
- Delete the old template from Git
- Flux deletes it from the cluster
- Add the new template with the same name
- Flux creates it — no
MachineDeploymentchanges needed
Approach B works because MachineDeployment references templates by name, not UID. The reconciliation gap is brief (seconds to minutes) and the MachineDeployment will pick up the new template automatically.
When any Proxmox VM (control-plane or worker) is recreated by hand (e.g. restored from a Proxmox backup) it is no longer in CAPI state, so its owning controller starts provisioning a replacement. Import the restored VM by repointing the existing ProxmoxMachine at it (rename the VM to match the object + set spec.virtualMachineID) so CAPI tracks it instead of the replacement.
Full procedure: docs/runbook-adopt-proxmox-vm.md.