nxtgauge-gitops/ops/k3s

k3s Registry Node Configuration

This repo uses registry.nxtgauge.com for backend images.

Why

Image pulls happen on k3s nodes via containerd, not inside cluster DNS context. Using *.svc.cluster.local for image pulls can fail with DNS lookup errors from node runtime.

Required node config

Each node must have /etc/rancher/k3s/registries.yaml configured with auth for the registry.

Template file:

• ops/k3s/registries.yaml

Recommended node DNS/hosts override (prevents Cloudflare/proxy path)

Even if registry.nxtgauge.com is set to "DNS only" in Cloudflare, k3s nodes can still end up resolving to public/IPv6 records depending on upstream DNS/caches.

For reliable large image pulls/pushes (avoids 413 Payload Too Large from proxies), point nodes directly at the in-cluster ingress VIP:

• Traefik VIPs: 10.0.0.2, 10.0.0.3, 10.0.0.5
• Recommended: pick one stable VIP (example 10.0.0.2) and map registry.nxtgauge.com to it on every node.

Apply to all nodes

1. Export required env vars:

export K3S_NODES="node1 node2 node3"
export REGISTRY_USERNAME="<registry-user>"
export REGISTRY_PASSWORD="<registry-pass>"
export REGISTRY_VIP_IP="10.0.0.2"   # optional but recommended

2. Apply config and restart k3s on each node:

./ops/k3s/apply-registries.sh

Manual steps (if needed)

On each node:

1. Copy registries.yaml to /etc/rancher/k3s/registries.yaml
2. Restart runtime:

sudo systemctl restart k3s
# or for agents
sudo systemctl restart k3s-agent

3. Verify pod pulls:

kubectl -n nxtgauge get pods
kubectl -n nxtgauge describe pod <failing-pod>

Notes

• Ensure DNS for registry.nxtgauge.com resolves from every k3s node.
• If DNS is not available, use a stable node-reachable IP and update:
  • backend GitOps manifests
  • backend Woodpecker registry push target
  • ops/k3s/registries.yaml