nxtgauge-gitops/docs/AI_PLANS_IMPLEMENTATION.md

AI Plans API Key Management for Nxtgauge

Overview

Each user gets a unique API key to track usage and enforce plan limits.

API Key Structure

User ID: user_12345
Plan: free | pro | enterprise
API Key: sk-nxtgauge-user_12345-abc123xyz

Implementation Approach

Since LiteLLM Community Edition has limited virtual key features, we'll implement a custom middleware/proxy approach:

Backend Implementation (Rust/Node.js)

1. User Registration → Generate API key
2. API Key Validation → Check against database
3. Usage Tracking → Increment counters per request
4. Rate Limiting → Enforce plan limits

Database Schema

-- Users table
CREATE TABLE users (
    id UUID PRIMARY KEY,
    email VARCHAR(255) UNIQUE,
    plan_type VARCHAR(50), -- 'free', 'pro', 'enterprise'
    api_key VARCHAR(255) UNIQUE,
    ai_credits_remaining INTEGER DEFAULT 100,
    monthly_usage_tokens INTEGER DEFAULT 0,
    created_at TIMESTAMP
);

-- AI Usage tracking
CREATE TABLE ai_usage (
    id UUID PRIMARY KEY,
    user_id UUID REFERENCES users(id),
    model VARCHAR(100), -- 'askash-fast', 'askash-main', etc.
    tokens_input INTEGER,
    tokens_output INTEGER,
    request_type VARCHAR(100), -- 'help', 'resume', 'jd', etc.
    created_at TIMESTAMP
);

-- API Keys table (for rotation)
CREATE TABLE api_keys (
    id UUID PRIMARY KEY,
    user_id UUID REFERENCES users(id),
    key_hash VARCHAR(255),
    is_active BOOLEAN DEFAULT true,
    created_at TIMESTAMP,
    expires_at TIMESTAMP
);

Plan Tiers

Plan	Price	Credits	Models Available	Rate Limit
Free	$0	100/month	askash-fast only	10 req/min
Pro	$9/mo	1000/month	All 4B models	60 req/min
Business	$29/mo	5000/month	All models incl 8B	120 req/min
Enterprise	Custom	Unlimited	All + Priority	Unlimited

API Key Generation (Example in Rust)

use uuid::Uuid;
use rand::{distributions::Alphanumeric, Rng};

pub fn generate_api_key(user_id: &str) -> String {
    let random_suffix: String = rand::thread_rng()
        .sample_iter(&Alphanumeric)
        .take(24)
        .map(char::from)
        .collect();
    
    format!("sk-nxtgauge-{}-{}", user_id, random_suffix)
}

// Example: sk-nxtgauge-user_12345-aBc3xYz9mNqP7rStUvWxYz12

API Middleware Flow

User Request (with API key)
    ↓
[Nginx/Traefik Ingress]
    ↓
[API Gateway - Validate Key]
    ↓
[Check Plan & Limits]
    ├─ Check credits remaining
    ├─ Check rate limit (Redis)
    └─ Check model access
    ↓
[Route to LiteLLM]
    ↓
[Track Usage]
    ├─ Decrement credits
    ├─ Log usage to DB
    └─ Update metrics
    ↓
[Return Response]

Model Access by Plan

free_tier:
  models:
    - askash-fast      # qwen3:4b
    - help-assistant   # qwen3:4b
    - messenger        # qwen3:4b
  max_tokens_per_request: 1000
  
pro_tier:
  models:
    - askash-fast
    - askash-main      # qwen3:8b
    - help-assistant
    - jd-generator     # qwen3:8b
    - profile-writer   # qwen3:8b
    - recommender      # qwen3:4b
  max_tokens_per_request: 4000
  
business_tier:
  models:
    - ALL_MODELS
  max_tokens_per_request: 8000
  
enterprise_tier:
  models:
    - ALL_MODELS
    - PRIORITY_QUEUE
  max_tokens_per_request: 32000

Cost Calculation (Per 1K tokens)

Since we're running local Ollama:

• Cost is compute-based, not API-based
• Track by GPU time or request duration
• Alternative: flat rate per request type

// Example pricing (based on compute cost)
const PRICING: &[(str, f64)] = &[
    ("askash-fast", 0.001),      // $0.001 per 1K tokens
    ("askash-main", 0.005),      // $0.005 per 1K tokens
    ("jd-generator", 0.008),     // $0.008 per 1K tokens
    ("profile-writer", 0.008),   // $0.008 per 1K tokens
];

Usage Endpoints for Frontend

// Get user's current usage
GET /api/v1/ai/usage
Headers: Authorization: Bearer sk-nxtgauge-user_12345-...

Response:
{
  "plan": "pro",
  "credits_remaining": 750,
  "credits_used_this_month": 250,
  "requests_today": 45,
  "rate_limit": {
    "requests_per_minute": 60,
    "current_window": "58/60"
  }
}

Implementation in Existing Backend

Add to your Rust backend:

1. Migration: Add api_key, ai_plan, ai_credits columns to users table
2. Middleware: Create AiAuthMiddleware to validate keys
3. Service: Create AiUsageService to track and limit
4. Endpoints:
   • POST /api/v1/ai/chat (with API key auth)
   • GET /api/v1/ai/usage
   • POST /api/v1/ai/upgrade (change plan)

Quick Start Commands

# Generate API key for user
curl -X POST https://api.nxtgauge.com/v1/ai/keys \
  -H "Authorization: Bearer $USER_JWT" \
  -d '{"plan": "pro"}'

# Use API key
curl https://llm.nxtgauge.com/v1/chat/completions \
  -H "Authorization: Bearer sk-nxtgauge-user_12345-abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "askash-main",
    "messages": [{"role": "user", "content": "Help with my resume"}]
  }'

Files to Create

1. /src/services/ai_usage.rs - Usage tracking service
2. /src/middleware/ai_auth.rs - API key validation
3. /src/models/ai_plan.rs - Plan definitions
4. Database migrations for API keys and usage tables

Next Steps

1. Choose: Build custom middleware OR use LiteLLM Enterprise
2. Create database migrations
3. Implement API key generation
4. Add usage tracking middleware
5. Create billing integration

LiteLLM Alternative

For simpler setup, LiteLLM Enterprise ($500/mo) provides:

• Built-in virtual keys
• Usage dashboards
• Team management
• Budget controls
• SSO/SAML

But custom implementation gives more control and lower cost.