Ashwin Kumar Sivakumar 7902b265a9 feat(ai): add AI plans docs, LiteLLM manifests, and infrastructure updates

- Add comprehensive AI plans implementation documentation
- Add LiteLLM gateway Kubernetes manifests
- Update PostgreSQL and Forgejo deployment configs
- Add build-from-binaries script

2026-06-15 06:15:41 +05:30

2.5 KiB

Raw Blame History

LiteLLM Connection Details for OpenCode

Quick Connect

Setting	Value
Base URL	`http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1`
API Key	`sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9`

Available Models

askash-main - Primary model (Ollama gemma3:270m)
askash-fast - Faster response variant
coding-main - Optimized for code tasks

OpenCode Configuration

Option 1: Environment Variables

export OPENAI_API_KEY="sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9"
export OPENAI_API_BASE="http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1"
export LLM_MODEL="askash-main"

Option 2: Config File

Create ~/.config/opencode/config.json:

{
  "provider": "openai",
  "apiKey": "sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9",
  "baseUrl": "http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1",
  "model": "askash-main"
}

Option 3: .opencode File (Project-specific)

Create .opencode in your project root:

{
  "llm": {
    "provider": "openai",
    "apiKey": "sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9",
    "baseUrl": "http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1",
    "model": "askash-main"
  }
}

Test Connection

# List models
curl http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1/models \
  -H "Authorization: Bearer sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9"

# Chat completion
curl http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "askash-main",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Port-Forward (If Running Locally)

If you need to access from your local machine:

kubectl port-forward svc/litellm 4000:4000 -n nxtgauge-ai

Then use: http://localhost:4000/v1

Troubleshooting

Issue	Solution
Connection refused	Run `kubectl port-forward svc/litellm 4000:4000 -n nxtgauge-ai`
401 Unauthorized	Check API key is correct
404 Not Found	Ensure URL ends with `/v1`
Timeout	Model may be loading; retry

Verify LiteLLM is Running

kubectl get pods -n nxtgauge-ai -l app=litellm

Last Updated: 2026-06-14 Namespace: nxtgauge-ai Service: litellm:4000

2.5 KiB Raw Blame History