b4f371d415
- Add .sops.yaml (age) and encrypt every plaintext Secret manifest in apps/ - Commit the Flux GitRepository/Kustomization (previously only applied manually) with a decryption stanza referencing the sops-age key (created out-of-band, not committed) - Rotate JWT_SECRET (previously exposed in plaintext) and set a real AI_SERVICE_KEY (was an empty placeholder), shared between nxtgauge-backend-rust and nxtgauge-ai-assistant - Wire JWT_SECRET into the ai-assistant deployment (was missing entirely, causing every authenticated request to fail) - Redact the leaked LiteLLM production master key from README/OPENCODE_CONNECT docs; move the litellm prod overlay off a plaintext secretGenerator onto an encrypted patch
100 lines
2.5 KiB
Markdown
100 lines
2.5 KiB
Markdown
# LiteLLM Connection Details for OpenCode
|
|
|
|
## Quick Connect
|
|
|
|
| Setting | Value |
|
|
|---------|-------|
|
|
| **Base URL** | `http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1` |
|
|
| **API Key** | `<REDACTED - see nxtgauge-litellm-secrets Secret, key LITELLM_MASTER_KEY>` |
|
|
|
|
## Available Models
|
|
|
|
- `askash-main` - Primary model (Ollama gemma3:270m)
|
|
- `askash-fast` - Faster response variant
|
|
- `coding-main` - Optimized for code tasks
|
|
|
|
## OpenCode Configuration
|
|
|
|
### Option 1: Environment Variables
|
|
|
|
```bash
|
|
export OPENAI_API_KEY="<REDACTED - see nxtgauge-litellm-secrets Secret, key LITELLM_MASTER_KEY>"
|
|
export OPENAI_API_BASE="http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1"
|
|
export LLM_MODEL="askash-main"
|
|
```
|
|
|
|
### Option 2: Config File
|
|
|
|
Create `~/.config/opencode/config.json`:
|
|
|
|
```json
|
|
{
|
|
"provider": "openai",
|
|
"apiKey": "<REDACTED - see nxtgauge-litellm-secrets Secret, key LITELLM_MASTER_KEY>",
|
|
"baseUrl": "http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1",
|
|
"model": "askash-main"
|
|
}
|
|
```
|
|
|
|
### Option 3: .opencode File (Project-specific)
|
|
|
|
Create `.opencode` in your project root:
|
|
|
|
```json
|
|
{
|
|
"llm": {
|
|
"provider": "openai",
|
|
"apiKey": "<REDACTED - see nxtgauge-litellm-secrets Secret, key LITELLM_MASTER_KEY>",
|
|
"baseUrl": "http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1",
|
|
"model": "askash-main"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Test Connection
|
|
|
|
```bash
|
|
# List models
|
|
curl http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1/models \
|
|
-H "Authorization: Bearer <REDACTED - see nxtgauge-litellm-secrets Secret, key LITELLM_MASTER_KEY>"
|
|
|
|
# Chat completion
|
|
curl http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1/chat/completions \
|
|
-H "Authorization: Bearer <REDACTED - see nxtgauge-litellm-secrets Secret, key LITELLM_MASTER_KEY>" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "askash-main",
|
|
"messages": [{"role": "user", "content": "Hello"}]
|
|
}'
|
|
```
|
|
|
|
## Port-Forward (If Running Locally)
|
|
|
|
If you need to access from your local machine:
|
|
|
|
```bash
|
|
kubectl port-forward svc/litellm 4000:4000 -n nxtgauge-ai
|
|
```
|
|
|
|
Then use: `http://localhost:4000/v1`
|
|
|
|
## Troubleshooting
|
|
|
|
| Issue | Solution |
|
|
|-------|----------|
|
|
| Connection refused | Run `kubectl port-forward svc/litellm 4000:4000 -n nxtgauge-ai` |
|
|
| 401 Unauthorized | Check API key is correct |
|
|
| 404 Not Found | Ensure URL ends with `/v1` |
|
|
| Timeout | Model may be loading; retry |
|
|
|
|
## Verify LiteLLM is Running
|
|
|
|
```bash
|
|
kubectl get pods -n nxtgauge-ai -l app=litellm
|
|
```
|
|
|
|
---
|
|
|
|
**Last Updated**: 2026-06-14
|
|
**Namespace**: nxtgauge-ai
|
|
**Service**: litellm:4000
|