55 lines
1.3 KiB
Markdown
55 lines
1.3 KiB
Markdown
# k3s Local Registry Node Configuration
|
|
|
|
This repo now uses `registry.nxtgauge.internal:5000` for backend images.
|
|
|
|
## Why
|
|
Image pulls happen on k3s nodes via containerd, not inside cluster DNS context.
|
|
Using `*.svc.cluster.local` for image pulls can fail with DNS lookup errors from node runtime.
|
|
|
|
## Required node config
|
|
Each node must have `/etc/rancher/k3s/registries.yaml` configured to trust and use the registry.
|
|
|
|
Template file:
|
|
- `ops/k3s/registries.yaml`
|
|
|
|
## Apply to all nodes
|
|
|
|
1. Export required env vars:
|
|
|
|
```bash
|
|
export K3S_NODES="node1 node2 node3"
|
|
export REGISTRY_USERNAME="<registry-user>"
|
|
export REGISTRY_PASSWORD="<registry-pass>"
|
|
```
|
|
|
|
2. Apply config and restart k3s on each node:
|
|
|
|
```bash
|
|
./ops/k3s/apply-registries.sh
|
|
```
|
|
|
|
## Manual steps (if needed)
|
|
On each node:
|
|
|
|
1. Copy `registries.yaml` to `/etc/rancher/k3s/registries.yaml`
|
|
2. Restart runtime:
|
|
|
|
```bash
|
|
sudo systemctl restart k3s
|
|
# or for agents
|
|
sudo systemctl restart k3s-agent
|
|
```
|
|
|
|
3. Verify pod pulls:
|
|
|
|
```bash
|
|
kubectl -n nxtgauge get pods
|
|
kubectl -n nxtgauge describe pod <failing-pod>
|
|
```
|
|
|
|
## Notes
|
|
- Ensure DNS for `registry.nxtgauge.internal` resolves from every k3s node.
|
|
- If DNS is not available, use a stable node-reachable IP:port and update:
|
|
- backend GitOps manifests
|
|
- backend Woodpecker registry push target
|
|
- `ops/k3s/registries.yaml`
|