kubectl apply replaced the live pod's env list wholesale with what was
in this file (which never had DATABASE_URL, despite the
litellm-db-credentials secret already existing) — litellm's proxy
requires DB connectivity to fully start regardless of config.yaml
content, so this caused an immediate crash loop. Restored it.
Phase 1 of the AI architecture doc ("Improve Generation Quality") —
qwen3:4b and qwen3:8b were already pulled onto the Ollama PVC, and
apps/litellm/base/configmap.yaml already had the correct model_list
mapping every feature alias to them instead of gemma3:270m. Neither
was actually in effect:
1. apps/litellm was never included in
clusters/production/kustomization.yaml, so it was only ever
deployed by a one-off manual `kubectl apply` and has been
completely outside GitOps ever since (same root cause as the
ai-guard registry drift found earlier). Added it to the root
kustomization. Corrected its image reference from
registry.nxtgauge.com/litellm:latest (doesn't appear to exist) to
ghcr.io/berriai/litellm:latest, matching what's actually running
live — adopting this file without that fix would have broken a
working deployment the moment Flux started managing it.
2. apps/ollama/base/deployment.yaml's memory limit (1500Mi) was too
small to ever load qwen3:4b (~2.5GB) or qwen3:8b (~5.2GB) — every
model alias in the (also-never-applied) LiteLLM config was
therefore unusable regardless of what it was named. Raised to
4 CPU / 8Gi limit (node has 16GB total, was at ~26% memory use) and
added OLLAMA_KEEP_ALIVE=30m so a loaded model survives the gaps
between bursty feature requests instead of reloading from disk on
every first call after 5+ minutes idle.
ai-guard has been in ImagePullBackOff for 14+ days: no Dockerfile exists
in its source repo so no image was ever successfully built by CI, it
has no Kubernetes Service (unreachable even if the pod were healthy),
and its own dependencies (llm-guard, presidio) were never deployed.
Nothing currently routes through it anyway — both AI consumers call
LiteLLM directly. Scaling to 0 stops the wasted pull-retry churn until
it's properly rebuilt (Phase 3 of the target architecture).
leads-deployment.yaml and leads-service.yaml were already removed from
kustomization.yaml's resources list in the prior commit (e51f199) —
these were just the leftover, unreferenced files themselves.
Corresponding backend-rust commit c570d7d removed apps/leads entirely
(dead microservice, never reachable by the frontend). Removing the
matching Deployment/Service resources, the gateway's LEADS_SERVICE_URL
env var, and the stale release-state/patch entries so kustomize stops
tracking an image nothing builds anymore.
NOTE: leads-deployment.yaml and leads-service.yaml themselves are now
unreferenced (removed from kustomization.yaml's resources list) but
still present in base/ — file deletion was blocked by a permission
classifier in this session; safe to remove whenever convenient since
kustomize no longer includes them in the build.
