Skip to content

Migration guide

This guide covers the backup → sanitize → restore path for moving Rancher management to a new cluster while keeping selected downstream clusters.

Overview

Step Tool / action
1 Full backup on source Rancher (rancher-backup operator)
2 rancher-polymorph inspect --tree — review inventory
3 rancher-polymorph sanitize — Bosch: --keep-rke1-only; single cluster: --keep-cluster <id>
4 rancher-polymorph restore run on a clean target cluster
5 Install Rancher Helm on target (cert-manager already present)
6 reconnect-rke1-agents.sh on each RKE1 downstream cluster

RKE1 provisioned clusters

Rancher-provisioned RKE1 clusters cannot be cleanly “detached” like imported clusters. The supported migration patterns are:

  1. Dual Rancher (backup/restore) — sanitize backup to keep RKE1 cluster definitions (--keep-rke1-only for all RKE1 clusters, or --keep-cluster for one); restore on new mgmt; point agents at new URL.
  2. Per-cluster import — export projects/RBAC, detach/cleanup, import on target (separate runbook).

rancher-polymorph focuses on pattern 1.

Note

After restore, downstream RKE1 nodes still run Kubernetes — you must update CATTLE_SERVER / re-run registration so agents connect to the new Rancher URL.

Bosch workflow (--keep-rke1-only)

Typical Bosch cutover:

  1. Full backup on source management Rancher.
  2. Sanitize with --keep-rke1-only (all RKE1 custom clusters stay; RKE2/imported + local go).
  3. Restore on a clean target (RKE2 + cert-manager + rancher-backup only — never a cluster that already ran Rancher).
  4. Install Rancher on target using Bosch's certificate / ingress process (not the lab default ingress.tls.source=rancher).
  5. Update server-url to the new hostname.
  6. Reconnect downstream agents using Bosch's CA checksum (see below).

For append restores (2nd+ cluster from another origin onto an already-live target), sanitize with --for-append-restore --for-restore so globals are stripped; apply a new Restore CR per cluster batch, then reconnect agents.

rancher-polymorph handles steps 2–4. Steps 4–6 are customer-owned; scripts only assist step 6.

Certificates (Bosch-owned)

Bosch installs Rancher with their own CA / ingress / trust store. Do not assume:

  • curl …/v3/settings/cacerts | sha256sum is correct for CATTLE_CA_CHECKSUM
  • the tls-rancher secret hash matches what agents validate
  • the checksum from the source Rancher still applies on the target

After Bosch finishes cert + Helm on the target, derive CATTLE_CA_CHECKSUM from their runbook (or from the agent log line Configured cacerts checksum (…) on first connect). Pass it as the 4th argument to reconnect-rke1-agents.sh.

If Rancher is fronted by a public / corporate CA that agents already trust, Bosch may clear CATTLE_CA_CHECKSUM ("") per Rancher cert update docs.

scripts/rancher-ca-checksum.sh is a lab helper only — not a substitute for the Bosch cert procedure.

What can go wrong (checklist)

Phase Symptom Likely cause
Sanitize RKE2 clusters still in tarball Forgot --keep-rke1-only; used broken CLI default keep_cluster: c-xxxxx (fixed in recent rancher-polymorph)
Sanitize Ghost cluster namespaces after restore (c-wl8nj, etc.) Deleted cluster still in backup paths — add --remove-cluster <id> or run inspect for orphans
Sanitize All RKE1 removed Same c-xxxxx config bug with --keep-rke1-only
Target prep Restore stuck / namespaced resource errors Target not empty — leftover cattle-* namespaces, ext.cattle.io APIService, finalizers from a prior Rancher install
Target prep Restore never completes Rancher or partial Rancher already on target; backup file not on operator PVC path
Restore server-url still shows source hostname Expected until patched post-Helm; breaks UI links and agent registration until updated
Certs cattle-cluster-agent CrashLoopBackOff, checksum mismatch CATTLE_CA_CHECKSUM from old server or wrong hash method; restored cacerts ≠ live serving cert after Helm
Certs Agents trust wrong CA Bosch ingress cert rotated but cacerts setting / cluster caCert not updated together
Reconnect Unavailable, tunnel log 401 failed authentication Agent still uses source registration token — patch cattle-credentials-* with a new token from target (ClusterRegistrationToken); reconnect-rke1-agents.sh 5th arg or KUBECONFIG+CLUSTER_ID
Reconnect Cluster Connected but Ready false Only subset of nodes reconnected; stale Ready condition; see node registration
Reconnect Script fails: connection refused 127.0.0.1:6443 SSH target is a worker without API — use any control-plane node that runs kube-apiserver
Reconnect Some nodes still call source Rancher Standalone docker run rancher-agent registration containers (RKE join) — not fixed by reconnect-rke1-agents.sh; need new node registration token from target
Reconnect Hairpin / timeout to new URL Nodes in same VPC as Rancher — need /etc/hosts + hostAliases mapping new hostname → private IP (script does this)
DNS Agents resolve public IP but can't reach it Same-VPC hairpin; or downstream CoreDNS still maps old Rancher hostname (patch CoreDNS hosts on downstream if used)
Ops Terraform / SSH to wrong IP Ephemeral public IPs after recreate — refresh terraform output / terraform refresh

Minimum validation before cutover sign-off

# After sanitize — only RKE1 cluster CRs remain
./bin/rancher-polymorph inspect -i backups/sanitized-rke1-only.tar.gz

# After restore — Completed, no Rancher pre-installed
kubectl get restore rancher-restore -n cattle-resources-system

# After Bosch Helm + cert — server-url matches new hostname
kubectl get settings.management.cattle.io server-url -o yaml

# After reconnect — no checksum errors in agent logs
kubectl logs -n cattle-system -l app=cattle-cluster-agent --tail=20   # on a downstream CP node

# Rancher API — clusters Active, nodes visible
# UI or: curl …/v3/clusters

If backups live in S3:

rancher-polymorph s3 pull migrations/source-full.tar.gz -o ./backups/in.tar.gz
# Bosch: keep all RKE1 clusters
rancher-polymorph sanitize -i ./backups/in.tar.gz -o ./backups/out.tar.gz --keep-rke1-only
rancher-polymorph s3 push ./backups/out.tar.gz migrations/sanitized.tar.gz

Or use the TUI: Full migration → Download from S3.

Post-restore checklist

  • [ ] Restore CR status Ready
  • [ ] cert-manager installed on target local cluster
  • [ ] Rancher Helm installed (matching your target version policy)
  • [ ] DNS / ingress points to new Rancher URL
  • [ ] Downstream agents reconnected
  • [ ] Fleet / GitRepo repos reconciled

Further reading