7902b265a9
- Add comprehensive AI plans implementation documentation - Add LiteLLM gateway Kubernetes manifests - Update PostgreSQL and Forgejo deployment configs - Add build-from-binaries script
100 lines
2.5 KiB
Markdown
100 lines
2.5 KiB
Markdown
# LiteLLM Connection Details for OpenCode
|
|
|
|
## Quick Connect
|
|
|
|
| Setting | Value |
|
|
|---------|-------|
|
|
| **Base URL** | `http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1` |
|
|
| **API Key** | `sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9` |
|
|
|
|
## Available Models
|
|
|
|
- `askash-main` - Primary model (Ollama gemma3:270m)
|
|
- `askash-fast` - Faster response variant
|
|
- `coding-main` - Optimized for code tasks
|
|
|
|
## OpenCode Configuration
|
|
|
|
### Option 1: Environment Variables
|
|
|
|
```bash
|
|
export OPENAI_API_KEY="sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9"
|
|
export OPENAI_API_BASE="http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1"
|
|
export LLM_MODEL="askash-main"
|
|
```
|
|
|
|
### Option 2: Config File
|
|
|
|
Create `~/.config/opencode/config.json`:
|
|
|
|
```json
|
|
{
|
|
"provider": "openai",
|
|
"apiKey": "sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9",
|
|
"baseUrl": "http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1",
|
|
"model": "askash-main"
|
|
}
|
|
```
|
|
|
|
### Option 3: .opencode File (Project-specific)
|
|
|
|
Create `.opencode` in your project root:
|
|
|
|
```json
|
|
{
|
|
"llm": {
|
|
"provider": "openai",
|
|
"apiKey": "sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9",
|
|
"baseUrl": "http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1",
|
|
"model": "askash-main"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Test Connection
|
|
|
|
```bash
|
|
# List models
|
|
curl http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1/models \
|
|
-H "Authorization: Bearer sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9"
|
|
|
|
# Chat completion
|
|
curl http://litellm.nxtgauge-ai.svc.cluster.local:4000/v1/chat/completions \
|
|
-H "Authorization: Bearer sk-litellm-prod-1c66d63e701c32cd85922a62fd2e087469486a9b7a34d950423a8726d0aceec9" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "askash-main",
|
|
"messages": [{"role": "user", "content": "Hello"}]
|
|
}'
|
|
```
|
|
|
|
## Port-Forward (If Running Locally)
|
|
|
|
If you need to access from your local machine:
|
|
|
|
```bash
|
|
kubectl port-forward svc/litellm 4000:4000 -n nxtgauge-ai
|
|
```
|
|
|
|
Then use: `http://localhost:4000/v1`
|
|
|
|
## Troubleshooting
|
|
|
|
| Issue | Solution |
|
|
|-------|----------|
|
|
| Connection refused | Run `kubectl port-forward svc/litellm 4000:4000 -n nxtgauge-ai` |
|
|
| 401 Unauthorized | Check API key is correct |
|
|
| 404 Not Found | Ensure URL ends with `/v1` |
|
|
| Timeout | Model may be loading; retry |
|
|
|
|
## Verify LiteLLM is Running
|
|
|
|
```bash
|
|
kubectl get pods -n nxtgauge-ai -l app=litellm
|
|
```
|
|
|
|
---
|
|
|
|
**Last Updated**: 2026-06-14
|
|
**Namespace**: nxtgauge-ai
|
|
**Service**: litellm:4000
|