docs: add Ask Ash billing architecture and implementation plan docs
All checks were successful
build-and-release / build (push) Successful in 4m23s
All checks were successful
build-and-release / build (push) Successful in 4m23s
This commit is contained in:
parent
964b8d7ec4
commit
4e3180745f
2 changed files with 890 additions and 0 deletions
704
docs/ASK_ASH_BILLING_ARCHITECTURE.md
Normal file
704
docs/ASK_ASH_BILLING_ARCHITECTURE.md
Normal file
|
|
@ -0,0 +1,704 @@
|
|||
# Ask Ash — AI Credits, Billing, Usage, Subscription & Monetization Architecture
|
||||
|
||||
**Status:** Design proposal — greenfield, informed by the one proven money-movement pattern that already exists in this codebase (TraceCoins)
|
||||
**Scope:** `nxtgauge-backend-rust` (net-new `ai_credits` schema), `nxtgauge-ai-assistant`, LiteLLM
|
||||
**Correction notice (2026-07-03):** An earlier draft of this document's audit section described an `ai_plans`/`user_ai_subscriptions`/`ai_credit_transactions`/`ai_credit_packages` system, an `apps/users/src/ai/` module tree, and a PayU AI-credit purchase flow in `apps/payments/src/ai_credits.rs` as existing, working code. **None of that exists.** It was a fabricated research-agent output that was not independently verified before being written into this document. Every claim below was re-verified directly (`Read`/`grep`/`find`/`ls` against the actual files) on 2026-07-03 before this correction. Section 2 now reflects only what is actually on disk.
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
There is **no AI credits, billing, or subscription system in Nxtgauge today.** What exists in `nxtgauge-backend-rust` is two disconnected, non-monetary quota mechanisms and one broken stub:
|
||||
|
||||
1. **`company_ai_usage` / `job_seeker_ai_usage`** (`apps/users/src/handlers/ai.rs`) — a real, working daily-generation counter (not a credit balance) tied to `pricing_packages.package_type = 'AI_PACK'`, gating job-description/cover-letter/resume generation.
|
||||
2. **`company_ai_credits`** (`apps/companies/src/handlers/ai.rs`) — a credit-balance-shaped stub that **cannot function as deployed**: it queries `company_ai_credits` and `ai_usage_log` tables that don't exist in any migration, and derives `company_id` from `Uuid::parse_str("placeholder")` (always errors) or `Uuid::new_v4()` (a fresh random ID every call, never the caller's actual company). This is dead code with a plausible-looking shape, not a working feature.
|
||||
3. **No purchase flow.** `apps/payments/src/` contains only `main.rs` and a generic `packages.rs` (TraceCoins package purchases) — there is no AI-credit-specific order/verify flow, no PayU-for-AI-credits integration.
|
||||
|
||||
What *does* exist and is solid: the **TraceCoins wallet** (`crates/db/src/models/tracecoin_wallet.rs`) — a correctly built reserve/capture/release wallet using `pool.begin()` + `SELECT ... FOR UPDATE`, cleanly isolated from anything AI-related — and a **fully deployed, working LiteLLM + self-hosted Ollama stack** (`nxtgauge-gitops/apps/litellm/`), routing 13 model aliases (`askash-fast`, `askash-main`, etc.) to `qwen3:4b`/`qwen3:8b`/`gemma3:270m`, no paid providers configured.
|
||||
|
||||
This document is therefore a **greenfield design**, not an extension of prior work — there is nothing billing-shaped to extend. It reuses the one pattern in this codebase proven to move money-equivalent value correctly (TraceCoins' row-locked reserve/capture), applies it to a new `ai_credits` schema, and treats the two existing quota mechanisms as things to migrate users *off of*, not systems to build on top of.
|
||||
|
||||
Phasing (Section 20) sequences this by what has to exist before anything else can: **Phase 1 stands up the wallet + ledger + atomic charge path from nothing**, informed by TraceCoins, before any packages/plans/promotions/admin surface is built on top of it.
|
||||
|
||||
---
|
||||
|
||||
## 2. Audit — What Exists Today (Baseline)
|
||||
|
||||
This section is the ground truth the rest of the document builds on. Every claim below was independently re-verified with direct `Read`/`grep -rn`/`find`/`ls` against the actual files in `nxtgauge-backend-rust`, `nxtgauge-ai-assistant`, and `nxtgauge-gitops` on 2026-07-03 — not taken from a prior unverified pass.
|
||||
|
||||
### 2.1 What does not exist (correcting the earlier draft)
|
||||
|
||||
Confirmed absent, exhaustively:
|
||||
|
||||
- **No `ai_plans`, `user_ai_subscriptions`, `ai_feature_costs`, `ai_credit_transactions`, or `ai_credit_packages` table** — no migration in `crates/db/migrations/` or `crates/db/migrations_new/` creates any of them (`ls crates/db/migrations/*.up.sql` lists 55 real migrations; none of these names appear, and `grep -rn "ai_plans\|ai_credit_transactions" --include=*.rs .` returns zero hits in the whole workspace).
|
||||
- **No `apps/users/src/ai/` module** — `ls apps/users/src/ai` errors "No such file or directory." There is no `credits.rs`, `plans.rs`, `orchestrator.rs`, `model_router.rs`, `middleware.rs`, `litellm.rs`, or `usage.rs` anywhere in the repo.
|
||||
- **No `apps/payments/src/ai_credits.rs`, no PayU-for-AI-credits flow, no `admin_ai.rs`.** `apps/payments/src/` contains exactly two files: `main.rs` and `packages.rs` (`find apps/payments/src -type f`). `packages.rs` is generic TraceCoins-package purchasing, unrelated to AI.
|
||||
|
||||
### 2.2 What actually exists — three real, disconnected pieces
|
||||
|
||||
**(a) `company_ai_usage` / `job_seeker_ai_usage` — a real, working daily-count quota (not a credit system).**
|
||||
Defined in `crates/db/migrations/20260425000000_ai_usage.up.sql` (verified, full content read): two tables, `(company_id|job_seeker_id, usage_date, generations_used)` with a `UNIQUE(id_col, usage_date)` constraint. Enforced in `apps/users/src/handlers/ai.rs` (confirmed real: 3735 lines, `ls -la` timestamp Jul 3 20:05) via `check_and_increment_usage` (line 718) and `has_active_ai_pack` (line 681), which checks Redis first (`ai_cache::check_ai_rate_limit`) then does an atomic Postgres upsert: `INSERT ... ON CONFLICT (id_col, usage_date) DO UPDATE SET generations_used = generations_used + 1`. This *is* atomic and correct as a daily counter — but it counts *generations*, not credits, has no monetary concept, no package purchase flow of its own (it reads an existing `pricing_packages.package_type = 'AI_PACK'` entitlement), and no per-feature cost differentiation. Callers: `ai_generate_job_field`, `ai_generate_cover_letter`, `ai_tailor_resume`, `ai_auto_apply`, `ai_usage_status`.
|
||||
|
||||
**(b) `company_ai_credits` — a credit-balance-shaped stub that cannot function as deployed.**
|
||||
`apps/companies/src/handlers/ai.rs` (confirmed real, 362 lines, read in full) defines `GET /credits`, `POST /generate`, `GET /usage-history` against a `company_ai_credits` table and an `ai_usage_log` (singular) table — **neither table exists in any migration** (`grep -rn "CREATE TABLE.*company_ai_credits\|CREATE TABLE.*ai_usage_log\b"` across all migrations: zero matches). Worse, `company_id` is never derived from the authenticated caller: `get_ai_credits` does `Uuid::parse_str("placeholder")`, which **always returns an error** — this endpoint cannot return a 200 today. `generate_ai` and `get_usage_history` use `Uuid::new_v4()` — a fresh random UUID on every single call, so even if the tables existed, credits would never accumulate against a real company. This is placeholder/demo code, not a partial implementation to build on.
|
||||
|
||||
**(c) No purchase flow exists for AI credits of any kind.** Confirmed by (2.1) above.
|
||||
|
||||
### 2.3 What's genuinely solid and is the actual foundation for this design
|
||||
|
||||
- **TraceCoins wallet** (`crates/db/src/models/tracecoin_wallet.rs`, confirmed real and read in full) — `Wallet { id, user_id, balance, reserved, updated_at }` + `LedgerEntry` audit table, with `try_reserve_tracecoins`/`try_debit_reserved_tracecoins`/`try_release_reserved_tracecoins`, each opening `pool.begin()`, taking a `SELECT ... FOR UPDATE` row lock, checking the business condition, mutating, writing a ledger row, and committing — or rolling back on failure. This is the **only proven correct money-movement pattern in the codebase** and is the direct template for the new AI credits wallet (Section 4) and ledger (Section 5).
|
||||
- **TraceCoins/future-AI-credits separation** — confirmed no shared tables between `tracecoin_wallets`/`tracecoin_ledger` and anything AI-related; the new schema should preserve this by construction (entirely new tables, no shared columns).
|
||||
- **LiteLLM** — confirmed fully deployed (`nxtgauge-gitops/apps/litellm/base/{configmap,deployment,service,ingress,secret,db-secret,postgres}.yaml` all present), 13 model aliases routed to self-hosted Ollama (`qwen3:4b`, `qwen3:8b`, `gemma3:270m`), no paid providers configured.
|
||||
- **`nxtgauge-ai-assistant`** — confirmed real, separate Rust/Axum service (`src/{chat,forms,jobs,tickets,cover_letter,content_tools,providers,retrieval,...}`) with a working `LiteLLMProvider` (`src/providers/llm/litellm_provider.rs`, confirmed: posts to `{base_url}/chat/completions` with bearer auth). It has zero credit/billing enforcement of its own today — same conclusion as before, still accurate.
|
||||
|
||||
### 2.4 Confirmed gaps this design must fill (not "defects to fix" — there's nothing to fix, only to build)
|
||||
|
||||
| # | Gap | Evidence | Why it matters |
|
||||
|---|---|---|---|
|
||||
| G1 | No atomic wallet for AI credits exists at all. | 2.1, 2.2(b) | Must be built from scratch — Section 4/5, modeled directly on TraceCoins' `FOR UPDATE` pattern rather than reinvented. |
|
||||
| G2 | The one credit-balance-shaped code that exists (`company_ai_credits` stub) is unauthenticated-in-effect (random/placeholder company_id) and points at nonexistent tables. | 2.2(b), `apps/companies/src/handlers/ai.rs:87-125` | This file should be deleted/rewritten wholesale as part of Phase 1, not patched — patching a `Uuid::new_v4()` identity bug and a missing-table bug on top of each other is not meaningfully different from a rewrite. |
|
||||
| G3 | Two different, still-growing quota mechanisms already exist for overlapping AI features with no shared entitlement model — (a) above (real, working, daily-count only) and (b) above (broken, credit-shaped). | 2.2(a), 2.2(b) | The new `ai_credits` system must decide explicitly whether it replaces (a) outright or coexists during a migration window (Section 19) — it cannot ignore (a), since real users are gated by it today. |
|
||||
| G4 | `apps/companies/src/handlers/ai.rs`'s `FOR UPDATE` (line 144) is issued directly against `&state.pool`, not inside a `pool.begin()` transaction — even if the missing-table bug were fixed, the lock would not hold across the subsequent `UPDATE` call. | `apps/companies/src/handlers/ai.rs:139-177` | Concrete negative example of what *not* to do — cited here specifically so Section 4.2's reserve/capture design doesn't repeat it. |
|
||||
| G5 | No purchase/payment flow exists for AI credits — no order/verify flow, no package catalog table. | 2.1, 2.2(c) | Section 10/16.3 design this from zero, following the same Razorpay integration TraceCoins' `packages.rs` already uses in `apps/payments/src` (confirmed: `razorpay_order_id`/`razorpay_payment_id` columns in `apps/payments/src/main.rs`), not a new payment provider. |
|
||||
|
||||
Sections 4 (Wallet), 5 (Ledger), 6 (Pricing) build the system these gaps require; Section 19 covers the concrete migration-off-(a) risk.
|
||||
|
||||
---
|
||||
|
||||
## 3. High-Level Architecture
|
||||
|
||||
All layers below are new — there is no prior enforcement pipeline for AI credits to extend (Section 2.1). The design still separates gateway/auth (genuinely existing and unchanged) from the AI-specific layers (all net-new), and structures the new layers so the reserve/capture pattern (Section 4.2) is a first-class step, not something a handler can accidentally skip — the exact failure mode found in the `company_ai_credits` stub (G2/G4), where enforcement lived inline in one handler with no shared middleware at all.
|
||||
|
||||
```
|
||||
Frontend (solid-js apps)
|
||||
│
|
||||
▼
|
||||
API Gateway (apps/gateway — existing, unchanged)
|
||||
│
|
||||
▼
|
||||
Authentication (JWT, existing crates/auth — unchanged)
|
||||
│
|
||||
▼
|
||||
Ask Ash Enforcement Middleware ◄── NEW — no equivalent exists today (G1)
|
||||
│ • Plan Validation (plan exists, active, not expired)
|
||||
│ • Feature Permission (feature ∈ plan.allowed_features)
|
||||
│ • Model Permission (model ∈ plan.allowed_models)
|
||||
│ • Quota Validation (daily/monthly action counters, single source)
|
||||
│ • Wallet Pre-check (read-only: has_sufficient_balance?)
|
||||
▼
|
||||
Idempotency Layer ◄── NEW: Idempotency-Key header, dedup table, 24h TTL
|
||||
│
|
||||
▼
|
||||
Credit Reservation ◄── NEW: RESERVE (not DEBIT) credits atomically before calling the model
|
||||
│
|
||||
▼
|
||||
LiteLLM (existing, deployed and working — extend with retry/timeout/circuit breaker, Section 8)
|
||||
│
|
||||
▼
|
||||
Self-hosted Model (Ollama) / future paid providers
|
||||
│
|
||||
▼
|
||||
Response
|
||||
│
|
||||
▼
|
||||
Settlement ◄── NEW: CAPTURE the reservation (convert RESERVE → DEBIT) on success,
|
||||
│ or RELEASE it on failure — atomic, single transaction
|
||||
▼
|
||||
Ledger Append ◄── NEW immutable ai_credit_ledger insert (Section 5) — source of truth
|
||||
▼
|
||||
Usage Log + Audit Log (new ai_usage_logs table for this system, distinct from the existing company_ai_usage/job_seeker_ai_usage counters — Section 9)
|
||||
▼
|
||||
Analytics / Observability (Section 14)
|
||||
```
|
||||
|
||||
**Why reserve-then-capture instead of check-then-debit:** a naive "check balance, then update balance" implementation has a window between the check and the write where a second concurrent request can pass the same check — this is exactly the shape of bug the `company_ai_credits` stub's `FOR UPDATE`-outside-a-transaction issue (G4) demonstrates isn't free even when a lock keyword is present. Reserving credits atomically (moving them from `available` to `reserved` in one row-locked statement, the same pattern TraceCoins' wallet already uses for holds — Section 2.3) closes that window: a second concurrent request sees the reduced `available` balance immediately, before either request has even called the LLM.
|
||||
|
||||
---
|
||||
|
||||
## 4. AI Wallet
|
||||
|
||||
### 4.1 Design
|
||||
|
||||
The wallet is a new table, `user_ai_subscriptions` (name chosen to read naturally alongside a future plans/subscriptions layer — Section 11), modeled directly on `tracecoin_wallets`' proven `balance`/`reserved` shape (Section 2.3) but with more pools since AI credits need to distinguish *why* a credit exists (subscription grant vs. purchase vs. bonus) for expiry and consumption-order purposes, which TraceCoins doesn't need to:
|
||||
|
||||
| Concept | Column |
|
||||
|---|---|
|
||||
| Subscription (monthly) credits | `monthly_credits_total`, `monthly_credits_used` |
|
||||
| Purchased credits | `purchased_credits_total`, `purchased_credits_used` |
|
||||
| Promotional / bonus credits | `bonus_credits_total`, `bonus_credits_used` |
|
||||
| Reserved (in-flight) credits | `reserved_credits` — the TraceCoins-style hold column (Section 2.3) |
|
||||
| Locked credits (admin freeze / fraud hold) | `locked_credits` |
|
||||
| Lifetime purchased | `lifetime_purchased_credits` (monotonic, never decremented — for analytics/LTV, independent of expiry) |
|
||||
| Lifetime used | `lifetime_used_credits` (monotonic) |
|
||||
| Expiry | `purchased_credits_expire_at`, per-batch expiry via ledger (Section 5) |
|
||||
|
||||
**Balance calculation** (single source of truth, exposed as a SQL view `ai_wallet_balance` and a Rust `Wallet::available()` method):
|
||||
|
||||
```
|
||||
available_credits =
|
||||
(monthly_credits_total - monthly_credits_used)
|
||||
+ (purchased_credits_total - purchased_credits_used)
|
||||
+ (bonus_credits_total - bonus_credits_used)
|
||||
- reserved_credits
|
||||
- locked_credits
|
||||
|
||||
spendable_credits = available_credits // what charge_feature is allowed to consume
|
||||
total_credits = available_credits + reserved_credits + locked_credits
|
||||
```
|
||||
|
||||
**Consumption order (FIFO across pools, not FIFO within a pool)** — when `charge_feature` debits N credits, it consumes in this priority order, each a separate ledger line so the audit trail shows exactly which pool paid:
|
||||
|
||||
1. **Expiring soonest first** across whichever pools have an expiry (promotional → subscription monthly → purchased), so credits closest to expiring are used before credits that don't expire yet.
|
||||
2. Within "monthly" vs "purchased" (same expiry class), monthly-subscription credits are consumed before purchased credits — this matches user expectation (use what resets first): a user shouldn't watch their paid-for purchased credits burn down while unused monthly credits expire untouched.
|
||||
|
||||
### 4.2 Why reservation, not direct debit
|
||||
|
||||
The state machine per request:
|
||||
|
||||
```
|
||||
available → RESERVE(n) → available -= n, reserved += n
|
||||
on LLM success → CAPTURE → reserved -= n, {pool}_used += n, ledger: DEBIT
|
||||
on LLM failure → RELEASE → reserved -= n, available += n, ledger: RESERVE_RELEASED (no charge)
|
||||
on timeout (no callback) → background reaper releases reservations older than N minutes
|
||||
```
|
||||
|
||||
Every transition is one row-locked (`SELECT ... FOR UPDATE`) statement inside one DB transaction — the same shape as `try_reserve_tracecoins`/`try_debit_reserved_tracecoins`/`try_release_reserved_tracecoins` (Section 2.3), reused deliberately rather than inventing a new concurrency pattern for this system. This is also the direct fix for the concurrency bug demonstrated in the `company_ai_credits` stub (G4), where a `FOR UPDATE` was issued without wrapping it in `pool.begin()` and so didn't actually hold the lock across the following `UPDATE`.
|
||||
|
||||
---
|
||||
|
||||
## 5. AI Ledger (immutable, source of truth)
|
||||
|
||||
Per your ledger-strictness decision: the ledger is the source of truth; wallet balance columns (Section 4) are a **materialized cache** recomputed from the ledger, not the other way around. This is a deliberate step up from TraceCoins' pattern (which pairs a mutable balance column with an append-only `tracecoin_ledger` audit trail, but treats the balance column as authoritative, not derived) — justified here because AI credits will map directly to money captured via a payment gateway (Section 10), so financial-grade auditability is warranted from day one rather than retrofitted later.
|
||||
|
||||
### 5.1 Why the ledger must never be updated
|
||||
|
||||
An immutable, insert-only ledger is the only structure that can be independently re-derived and audited after the fact. If a row can be updated, "what was the balance at time T" becomes unanswerable without external backups; if every state change is a new row, the wallet balance is always `SUM(ledger rows)`, which means:
|
||||
|
||||
- **Reconciliation is trivial**: recompute balance from the ledger and compare to the cached wallet column; any drift is a bug, immediately detectable, rather than silently compounding.
|
||||
- **Disputes/refunds/audits have a paper trail** that can't have been retroactively altered — required for any billing system handling real payments.
|
||||
- **Idempotency is enforceable at the ledger level**: a unique constraint on `(idempotency_key)` makes retried writes a no-op at the database layer, not just an application-level check.
|
||||
|
||||
### 5.2 Ledger entry types
|
||||
|
||||
`purchase`, `subscription_grant`, `bonus_grant`, `referral_grant`, `admin_adjustment_credit`, `admin_adjustment_debit`, `consumption` (feature use), `reservation_hold`, `reservation_release`, `reservation_capture`, `refund`, `expiration`, `transfer` (future: org credit pools), `correction` (explicit reversal of a prior erroneous entry — never edits the original row).
|
||||
|
||||
Every entry carries: `id, wallet_id, entry_type, credits (signed), balance_after (computed at write time, within the same locked transaction that performed the mutation — never from an in-memory snapshot taken before the write, which is what makes it safe under concurrency), idempotency_key (unique), reference_type, reference_id, actor_type (user/admin/system), actor_id, metadata JSONB, created_at`. No `updated_at`, no `deleted_at` — by design, nothing on this table is ever mutated or soft-deleted; corrections are new rows referencing the row they correct.
|
||||
|
||||
---
|
||||
|
||||
## 6. AI Feature Pricing Engine
|
||||
|
||||
### 6.1 Why feature-based, not token-based, pricing
|
||||
|
||||
The design uses a flat `credit_cost` per `feature_code` (not a token-metered charge). Reasons:
|
||||
|
||||
1. **Predictability.** A user calling `jd_generate` needs to know it costs 5 credits before they click, not after the model streams back an unpredictable token count. Token-based pricing makes the same feature cost a different, unpredictable amount every time depending on how verbose the model happens to be — bad UX for a product surface, fine for a raw API.
|
||||
2. **Provider/model independence.** Feature pricing decouples "what the user pays" from "what the model costs internally," which is exactly what's needed to swap Ollama models or add paid providers later (your stated direction) without changing user-facing prices. Token-based pricing would leak infrastructure cost volatility directly into user bills.
|
||||
3. **Margin control.** With feature-based pricing, margin = `credit_cost × credit_unit_price − internal_cost(feature)`, tunable per feature independent of the underlying model's actual token cost — you can price `jd_generate` at 5 credits even if the model backing it changes from a 4B to an 8B model with different compute cost, and adjust margin centrally (Section 7) instead of re-deriving user prices.
|
||||
4. **Simpler enforcement.** `charge_feature` becomes "does the user have ≥5 credits" — a single comparison — instead of "estimate tokens, reserve a ceiling, charge actual, refund the difference," which is meaningfully more complex and is exactly the kind of complexity your own requirements (predictable feature pricing) are steering away from.
|
||||
|
||||
Token-level tracking is still fully retained (Section 7) — it's just used for **internal cost/margin observability**, not **user-facing pricing**. Both can and should coexist.
|
||||
|
||||
### 6.2 Feature pricing schema
|
||||
|
||||
`ai_feature_costs` — one row per priced feature:
|
||||
|
||||
| Column | Purpose |
|
||||
|---|---|
|
||||
| `feature_code`, `display_name`, `default_model`, `credit_cost` | core identity + price |
|
||||
| `max_input_tokens`, `max_output_tokens` | enforced as hard caps in the reservation step |
|
||||
| `min_plan_code` | explicit minimum plan tier required, independent of a plan's `allowed_features` array (defense in depth) |
|
||||
| `priority` | `low`/`normal`/`high` — routes to LiteLLM request priority / queue |
|
||||
| `timeout_ms` | per-feature LLM call timeout |
|
||||
| `rate_limit_per_minute` | per-user per-feature limit, layered on top of the Redis sliding-window pattern already used elsewhere in `apps/users/src/handlers/ai.rs` (`ai_cache::check_ai_rate_limit`) rather than inventing a second rate-limit mechanism |
|
||||
| `retry_policy` | `{max_retries, backoff_ms}` JSONB |
|
||||
| `fallback_model` | explicit fallback model to retry against on provider/model failure, separate from plan-based model fallback |
|
||||
|
||||
Example row (using the naming convention Ask Ash's feature list implies — `resume.improve`):
|
||||
```
|
||||
feature_key: resume_improve
|
||||
base_cost: 8 AI Credits
|
||||
default_model: askash-main
|
||||
min_plan_code: professional
|
||||
max_input_tokens: 4000
|
||||
max_output_tokens: 1500
|
||||
priority: normal
|
||||
timeout_ms: 15000
|
||||
retry_policy: {"max_retries": 1, "backoff_ms": 500}
|
||||
fallback_model: askash-fast
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Token Cost Engine
|
||||
|
||||
Tracks true internal cost per request, separate from what the user is charged (Section 6). A new `ai_usage_logs` table (there is no existing one to extend — Section 2.1) carries both the usage facts and cost columns from the start:
|
||||
|
||||
| Field | Notes |
|
||||
|---|---|
|
||||
| Prompt tokens, completion tokens, total tokens | from the LLM response |
|
||||
| Cached tokens | LiteLLM reports cache hits |
|
||||
| Provider cost | computed per model per the cost-basis config (below) |
|
||||
| Internal cost | Provider cost + amortized infra overhead (GPU/compute allocation for self-hosted) |
|
||||
| Credits charged | the feature's `credit_cost` at charge time |
|
||||
| Credit unit price (₹/credit) | from the active pricing config, needed to convert credits → ₹ for margin math |
|
||||
| Profit margin | `(credits_charged × credit_unit_price) − internal_cost`, computed at read time in analytics, not stored per-row (avoids recomputation on price changes) |
|
||||
| Currency conversion | Deferred — single-currency to start; add a rate table only when a second currency is actually needed |
|
||||
| Historical costs | Ledger + usage logs are append-only, so historical cost is inherently preserved — no separate "history" table needed |
|
||||
|
||||
**Cost-basis strategy (pluggable, per your LiteLLM + self-hosted decision):** since today's models are 100% self-hosted Ollama (confirmed: `askash-fast`/`askash-main` → `qwen3:4b`/`qwen3:8b`/`gemma3:270m`, no external provider configured in `litellm/configmap.yaml`), `provider_cost` defaults to a **compute-amortization strategy**: a per-model ₹/1K-tokens figure computed offline from GPU node cost ÷ observed throughput, configured in a small `ai_model_cost_config` table (`model_alias, cost_per_1k_input_tokens, cost_per_1k_output_tokens, cost_basis: 'compute_amortized' | 'provider_metered', effective_from`). The moment a paid provider (OpenAI/Anthropic/etc.) is added to LiteLLM's `model_list`, that model's row simply switches `cost_basis` to `provider_metered` and gets real per-token pricing — no schema change, no code change to the cost engine itself, only a config row. This is the concrete mechanism for "design for LiteLLM + self-hosted today, paid providers later" without a future rewrite.
|
||||
|
||||
**Credits → provider cost mapping**: `internal_cost(request) = tokens_used × cost_per_1k(model, cost_basis) / 1000`. Margin is a read-time analytics computation, never gates the request path — pricing (Section 6) already decided the charge before the model ran; cost is purely for observability and future price-tuning, not enforcement.
|
||||
|
||||
---
|
||||
|
||||
## 8. LiteLLM Integration
|
||||
|
||||
The confirmed-real integration point is `nxtgauge-ai-assistant`'s `LiteLLMProvider` (`src/providers/llm/litellm_provider.rs`) — a thin HTTP wrapper posting to `{base_url}/chat/completions` with bearer auth. `nxtgauge-backend-rust` has no LiteLLM client of its own today (Section 2.1); this design either gives the new AI-credits wallet/ledger code in `nxtgauge-backend-rust` its own equally-thin LiteLLM client (matching this provider's shape) or has `nxtgauge-backend-rust` call through to `nxtgauge-ai-assistant` for the actual model call while owning billing itself — Section 20 treats this as a Phase 1 design decision, not something to guess here.
|
||||
|
||||
| Capability | Status | Design |
|
||||
|---|---|---|
|
||||
| Usage/token reporting | Provider response includes usage; not yet wired to a cost engine anywhere | capture prompt/completion/cached tokens into `ai_usage_logs` |
|
||||
| Provider/cost reporting | Not read from LiteLLM anywhere today | LiteLLM exposes cost in response metadata for metered providers — capture into `ai_usage_logs.provider_reported_cost` when present (self-hosted models report 0, expected) |
|
||||
| Retries | No retry/backoff in `litellm_provider.rs` today | wrap the call with the `retry_policy` from `ai_feature_costs` (Section 6), exponential backoff, capped by `timeout_ms` |
|
||||
| Fallback model/costs | Not implemented today | on LiteLLM error/timeout, retry once against `fallback_model` from feature config; log which model actually served the request so cost/margin attributes correctly even after fallback |
|
||||
| Streaming costs | Not implemented today | reservation (Section 4.2) must hold the *maximum* possible cost (`max_output_tokens × credit_cost` ceiling) before a stream starts, then capture actual on stream completion — same reserve/capture pattern as non-streaming, just with a wider initial hold |
|
||||
| Cache savings | Not tracked today | once cached-token capture lands, expose `cache_savings = cached_tokens × cost_per_1k` in analytics (Section 14) |
|
||||
| Health monitoring | No health check against LiteLLM found in either repo | add a `/health` poll (LiteLLM exposes one) feeding a circuit breaker: on sustained failure, the enforcement middleware (Section 3) can return a fast "AI temporarily unavailable" instead of every request timing out individually |
|
||||
| Model aliases | Already a well-designed layer of indirection — confirmed real, 13 aliases in `litellm/configmap.yaml` (`askash-fast`, `askash-main`, etc.) map to real Ollama models, callers never hardcode real model names | keep as-is |
|
||||
|
||||
---
|
||||
|
||||
## 9. Database Design
|
||||
|
||||
Every table below is **new** — Section 2.1 confirmed no `ai_plans`/`ai_credit_*` schema exists to extend. Two existing tables outside this schema are relevant only as migration targets, not as things this schema builds on: `company_ai_usage`/`job_seeker_ai_usage` (real, working daily counters — Section 19 covers migrating their users onto this system) and `company_ai_credits`/`ai_usage_log` (referenced by the stub in `apps/companies/src/handlers/ai.rs` but never actually created by any migration — not a migration source, just dead code to delete, per G2).
|
||||
|
||||
**`ai_plans`** — plan catalog. `id, code UNIQUE, name, monthly_credits, daily_action_limit, daily_credit_limit, allowed_models JSONB, allowed_features JSONB, is_active, created_at, updated_at`. `daily_credit_limit` (separate from the daily action *count*) caps credit *spend* per day so a single expensive feature can't exhaust a whole month's credits in one burst.
|
||||
|
||||
**`user_ai_subscriptions`** — per-user wallet + subscription state (this **is** the wallet, per Section 4). `id, user_id UNIQUE (FK users.id), plan_id (FK ai_plans.id), role_code, monthly_credits_total, monthly_credits_used, purchased_credits_total, purchased_credits_used, bonus_credits_total, bonus_credits_used, reserved_credits, locked_credits, lifetime_purchased_credits, lifetime_used_credits, daily_actions_used, purchased_credits_expire_at, current_period_start, current_period_end, status, created_at, updated_at`. *Indexes:* `(user_id)` unique, `(plan_id)`.
|
||||
|
||||
**`ai_feature_costs`** — per Section 6.2: `id, feature_code UNIQUE, display_name, default_model, credit_cost, max_input_tokens, max_output_tokens, min_plan_code, priority, timeout_ms, rate_limit_per_minute, retry_policy JSONB, fallback_model, is_active, created_at, updated_at`.
|
||||
|
||||
**`ai_usage_logs`** — per Section 7: `id, user_id (FK users.id), role_code, feature_code, model_alias, credits_charged, input_tokens, output_tokens, total_tokens, cached_tokens, provider_reported_cost, internal_cost, credit_unit_price_at_time, status, request_id, error_message, created_at`. *Indexes:* `(user_id)`, `(feature_code)`, `(created_at)`.
|
||||
|
||||
**`ai_credit_ledger`** (Section 5) — *Columns:* `id, wallet_id (FK user_ai_subscriptions.id), entry_type, credits INT, balance_after INT, idempotency_key VARCHAR UNIQUE, reference_type, reference_id UUID, actor_type, actor_id UUID, metadata JSONB, created_at`. *Indexes:* `(wallet_id, created_at)`, `UNIQUE(idempotency_key)`, `(reference_type, reference_id)`. *Relationships:* `wallet_id → user_ai_subscriptions.id`; `reference_id` is polymorphic (payment, usage log, admin action) — no hard FK (mirrors `tracecoin_ledger.reference_id`'s approach, Section 2.3) but validated at the application layer with a `reference_type` enum instead of left fully untyped.
|
||||
|
||||
**`ai_credit_packages`** — `id, name, description, credits, price_paise INT CHECK (price_paise > 0), currency VARCHAR DEFAULT 'INR', package_type ('one_time'|'subscription_bundle'|'promotional'|'referral'), min_purchase_plan_code NULL, is_featured BOOLEAN, is_active, created_at, updated_at`. Storing price in the smallest currency unit (paise, not rupees) from the start avoids the class of bug that would occur if a display-formatting divide-by-100 were ever applied inconsistently — a concrete risk called out here specifically because the `company_ai_credits` stub's other bugs (G2) show this codebase's AI-adjacent code has shipped identity/placeholder bugs before without being caught by review; a `CHECK` constraint plus paise-first-always is cheap insurance.
|
||||
|
||||
**`ai_subscription_history`** — audit trail of plan changes (upgrade/downgrade/cancel): `id, user_id, from_plan_id, to_plan_id, change_type (upgrade/downgrade/cancel/renew), proration_credits INT, effective_at, created_by (user/admin/system), created_at`.
|
||||
|
||||
**`ai_coupons`** — `id, code UNIQUE, discount_type (percent/flat_credits/flat_amount), value, applicable_package_ids UUID[] NULL, applicable_plan_codes TEXT[] NULL, max_redemptions, redemptions_used, valid_from, valid_until, is_active`, plus **`ai_coupon_redemptions`** (`coupon_id, user_id, redeemed_at, UNIQUE(coupon_id, user_id)`) so a code is single-use-per-user by construction, not by convention.
|
||||
|
||||
**`ai_promotions`** — broader than coupons (no code required, e.g. "all Business plan signups this month get +50 bonus credits"): `id, name, trigger_type (signup/plan_upgrade/manual/referral_milestone), bonus_credits, applicable_plan_codes TEXT[], valid_from, valid_until, is_active`.
|
||||
|
||||
**`ai_bonus_credits`** — records *why* a bonus was granted, one row per grant event (referral, promo, admin goodwill): `id, user_id, source_type (referral/promotion/admin/partner), source_reference_id, credits, expires_at, ledger_entry_id (FK → ai_credit_ledger)`.
|
||||
|
||||
**`ai_refunds`** — `id, user_id, ledger_debit_entry_id (FK, the original consumption being refunded), credits_refunded, reason (provider_failure/timeout/validation_failure/manual), initiated_by (system/admin/user_dispute), status (pending/approved/rejected/completed), created_at, resolved_at`.
|
||||
|
||||
**`ai_limits`** — a single queryable table for admin visibility over plan daily-action-count, daily-credit-limit, and per-minute rate limits: `id, scope_type (plan/user/feature), scope_id, limit_type (daily_actions/daily_credits/per_minute_requests), limit_value, window_type`.
|
||||
|
||||
**`ai_usage_statistics`** — pre-aggregated rollups (hourly/daily) for dashboard performance, populated by a background job from `ai_usage_logs`: `id, period_start, period_type (hour/day/month), scope_type (platform/user/plan/feature/model), scope_id, requests_count, credits_consumed, tokens_consumed, internal_cost, unique_users`.
|
||||
|
||||
**`ai_billing_events`** — a unified event stream for the admin dashboard and future webhooks (purchase completed, subscription renewed, refund issued, coupon redeemed, low-balance warning fired): `id, event_type, user_id, payload JSONB, created_at`.
|
||||
|
||||
**`ai_audit_logs`** — every admin action against the billing system (plan change, manual credit adjustment, refund decision, coupon creation): `id, admin_user_id, action_type, target_user_id NULL, before_state JSONB, after_state JSONB, ip_address, created_at`. *Distinct from `ai_credit_ledger`*: the ledger is financial truth; this is who-did-what-when for admins specifically.
|
||||
|
||||
**`ai_reservation_holds`** — backs Section 4.2's reserve/capture flow: `id, wallet_id, credits_held, feature_code, request_id, status (held/captured/released/expired), created_at, expires_at`. A background reaper releases `held` rows past `expires_at` back to available balance — this is what makes a crash between the LLM call and the charge completing safe: a crashed request's hold simply expires and releases, worst case the user got one free response, never an unbounded liability.
|
||||
|
||||
---
|
||||
|
||||
## 10. AI Credit Packages (business design)
|
||||
|
||||
Kept as one-time/subscription-bundle/promotional/referral per the extended `package_type` column (Section 9). Advantages/disadvantages of each existing-plus-new type:
|
||||
|
||||
| Type | Advantage | Disadvantage |
|
||||
|---|---|---|
|
||||
| One-time (today's only type) | Simple, no commitment, good for trial/top-up | No recurring revenue, doesn't build retention |
|
||||
| Subscription bundle | Predictable MRR, encourages habitual use | Needs proration/cancellation machinery (Section 11) — the risk you're explicitly asking this doc to cover |
|
||||
| Promotional | Drives acquisition/reactivation cheaply | Must have hard expiry (Section 4.1's expiry-first consumption order) or it cannibalizes future paid purchases |
|
||||
| Admin-issued | Support/goodwill tool, retention lever | Must be audited (`ai_audit_logs`) or becomes a fraud/abuse vector — the admin credit-grant endpoint (Section 13) should require a mandatory reason field from day one, not add one later |
|
||||
| Referral | Cheap acquisition, viral loop | Needs fraud detection (Section 12) — self-referral is the obvious abuse case |
|
||||
| Coupon | Marketing flexibility, time-boxed | Must be single-use-per-user by default (`ai_coupons` needs a redemption-tracking join table, not just a global `redemptions_used` counter, to prevent one user redeeming the same code repeatedly — add `ai_coupon_redemptions(coupon_id, user_id, redeemed_at, UNIQUE(coupon_id, user_id))`) |
|
||||
|
||||
---
|
||||
|
||||
## 11. AI Plans & Subscription System
|
||||
|
||||
### 11.1 Plans
|
||||
|
||||
| Plan | Monthly credits | Daily action limit | Daily credit limit | Notes |
|
||||
|---|---|---|---|---|
|
||||
| Free | 10 | 3 | 10 | onboarding tier |
|
||||
| Pro | 100 | 15 | 40 | |
|
||||
| Business | 300 | 40 | 100 | first tier to get `team_support: true` (reserved flag — see 11.3) |
|
||||
| Enterprise | 50,000 | 999,999 | — | `api_access: true` (future), `team_support: true` |
|
||||
|
||||
(These figures are illustrative starting points, not derived from any existing seeded plan — Section 2.1 confirmed no `ai_plans` table exists today to carry forward numbers from. Final values are a pricing/business decision, not an engineering one.)
|
||||
|
||||
`max_context_length` per plan: not currently modeled at the plan level (only per-feature via `max_input_tokens`/`max_output_tokens`) — recommendation: keep context limits at the feature level (Section 6), not duplicated at the plan level, since context needs are feature-specific, not plan-specific; a plan-level cap would just be `MAX(feature caps)` and add no new information.
|
||||
|
||||
### 11.2 Subscription lifecycle
|
||||
|
||||
- **Monthly/yearly**: `user_ai_subscriptions.current_period_start/end` models the period; `billing_cycle: 'monthly' | 'yearly'` and `auto_renew BOOLEAN DEFAULT true` select the cadence.
|
||||
- **Included credits**: `monthly_credits_total`, granted at period start.
|
||||
- **Unused credit policy**: **recommend no rollover for subscription credits** (they're the "use it or lose it" pool by design — this is what makes the plan's monthly price predictable for the business); **purchased credits do carry over** (subject to `purchased_credits_expire_at`, Section 4.1) since the user paid cash for them directly.
|
||||
- **Upgrade**: immediate — new plan's `monthly_credits_total` applies immediately, prorated bonus credited via ledger `subscription_grant` for the remainder of the current period (`remaining_days / days_in_period × (new_monthly - old_monthly)`), written to `ai_subscription_history`.
|
||||
- **Downgrade**: takes effect at the *next* period start (not immediate), avoiding the awkwardness of clawing back credits the user already has access to mid-period; recorded in `ai_subscription_history` with `effective_at = current_period_end`.
|
||||
- **Cancellation**: subscription reverts to `free` at `current_period_end`; purchased credits are unaffected (they're not subscription-tied). Any bonus credits explicitly tied to the cancelled plan (via `ai_bonus_credits.source_type='promotion'` scoped to that plan) expire per their own `expires_at`.
|
||||
- **Proration**: computed once, at the moment of upgrade, as a ledger entry — never retroactively adjusted, consistent with the ledger's immutability (Section 5).
|
||||
- **Grace period**: on payment failure for a recurring subscription, a 3-day grace period keeps the current plan active with a `status='past_due'` flag before reverting to `free` — `user_ai_subscriptions.status` needs `active`/`past_due`/`cancelled`/`expired` as valid values from the start.
|
||||
- **Trial plans**: modeled as an `ai_plans` row with `is_trial: true` and a mandatory `trial_days` — at expiry, auto-downgrades to `free` unless the user has since purchased/upgraded.
|
||||
|
||||
### 11.3 Team support / API access
|
||||
|
||||
Not modeled at all today (confirmed: no organization/team concept exists anywhere in `nxtgauge-backend-rust`'s auth model — `roles`/`active_role` is per-user only). Recommend treating both as **explicitly out of scope for Phase 1–3** (Section 15) rather than half-building multi-tenancy underneath a billing system — team credit pools require an org model that doesn't exist yet and shouldn't be improvised as a side effect of this document. Flagging `team_support`/`api_access` as boolean plan flags now is enough to reserve the concept without committing to the underlying org data model prematurely.
|
||||
|
||||
---
|
||||
|
||||
## 12. Security
|
||||
|
||||
Mapped to the gaps in Section 2.4 plus standard billing-system hardening:
|
||||
|
||||
- **Double-spending prevention** → reserve/capture pattern (Section 4.2), row-locked (`SELECT ... FOR UPDATE`) transactions inside `pool.begin()` for every wallet mutation — the TraceCoins pattern, applied correctly (contrast with the `company_ai_credits` stub's `FOR UPDATE`-without-a-transaction bug, G4).
|
||||
- **Race conditions** → same fix; additionally, the wallet get-or-create-on-first-use path should use `INSERT ... ON CONFLICT (user_id) DO NOTHING` + re-fetch (the exact pattern `TracecoinWalletRepository::ensure_wallet` already uses, Section 2.3) rather than relying on a bare `UNIQUE` constraint to surface a raw DB error to the user on a race.
|
||||
- **Idempotency** → `Idempotency-Key` header required on the charge entry point and on the purchase-verify endpoint (Section 16.3); enforced via the `UNIQUE(idempotency_key)` constraint on `ai_credit_ledger` (Section 5), not just an application-level check, so it holds even under process crashes/retries.
|
||||
- **Ledger integrity** → append-only, no `UPDATE`/`DELETE` grants on `ai_credit_ledger` at the DB role level (enforce with a Postgres `REVOKE UPDATE, DELETE` on that table for the application's DB role — belt-and-suspenders beyond just "the code doesn't do it").
|
||||
- **Fraud detection** → referral self-abuse (same device/IP/payment method referring itself repeatedly), coupon abuse (one user, many accounts) — flag via `ai_audit_logs` + a simple velocity check (N referral redemptions from the same IP/device fingerprint in 24h → hold for manual review) rather than building a full fraud-ML system in Phase 1.
|
||||
- **Rate limiting** → reuse the Redis sliding-window pattern already proven in `apps/users/src/handlers/ai.rs` (`ai_cache::check_ai_rate_limit`), made feature-scoped via `ai_feature_costs.rate_limit_per_minute` rather than inventing a new mechanism.
|
||||
- **Abuse detection** → rapid-fire low-value requests designed to drain daily action counts, caught via the daily-credit-limit (Section 9) rather than only a raw action count, which a burst of cheap actions could otherwise exhaust without tripping any spend-based alarm.
|
||||
- **Credit locking** → `locked_credits` column (Section 4.1) — admin can freeze a suspicious account's spendable balance without touching the underlying totals, fully reversible, logged in `ai_audit_logs`.
|
||||
- **Transaction validation** → every ledger write validates `balance_after = balance_before + credits` within the same locked transaction before committing — a cheap invariant check specifically motivated by the concurrency bug pattern seen in `company_ai_credits` (G4), where balance math and the actual write were never guaranteed consistent.
|
||||
- **Audit trails** → `ai_audit_logs` (Section 9) for admin actions; `ai_credit_ledger` for financial actions; the two are deliberately separate tables because they answer different questions ("who did it" vs "what happened to the money").
|
||||
- **Service-to-service auth** → if a cross-service call is needed (e.g. payments service notifying users service of a completed purchase, or `nxtgauge-ai-assistant` calling into `nxtgauge-backend-rust` to check/charge a wallet — Section 8's open design question), use a short-lived service JWT minted per-call with a dedicated `service` role scope the receiving service's auth middleware explicitly recognizes, not a long-lived static bearer token/secret pasted into service config — that class of credential is a standing exposure risk regardless of what it protects.
|
||||
|
||||
---
|
||||
|
||||
## 13. Admin Dashboard (Ask Ash Billing Admin)
|
||||
|
||||
There is no existing admin surface for AI billing (Section 2.1) — everything below is net-new. Listed as capabilities to build, not a gap-vs-existing table:
|
||||
|
||||
| Capability | Backed by |
|
||||
|---|---|
|
||||
| View user wallet | "wallet summary" endpoint combining balance + reservations + lifetime stats (Section 4) |
|
||||
| Adjust credits (add) | new endpoint, writes through the ledger (Section 5), not a direct column update — the exact mistake to avoid is computing `balance_after` from a stale in-memory read the way the `company_ai_credits` stub's sibling logic would have, had it worked at all |
|
||||
| Adjust credits (subtract/refund) | new endpoint, backed by `ai_refunds` (Section 9) |
|
||||
| Manage packages | CRUD against `ai_credit_packages` (Section 9) |
|
||||
| Manage plans/feature pricing | CRUD against `ai_plans`/`ai_feature_costs` (Section 9) |
|
||||
| Provider cost / margin view | reads `ai_usage_logs.internal_cost` + `ai_credit_ledger` (Section 7) |
|
||||
| Revenue dashboard | aggregates `ai_credit_ledger` (purchases) + `ai_usage_statistics` (Section 9) |
|
||||
| Usage/token/model analytics | `ai_usage_statistics` rollups (Section 9) feed pre-aggregated charts instead of scanning `ai_usage_logs` live |
|
||||
| Top users / failed requests | queries against `ai_usage_logs` (status tracked per row) and `ai_usage_statistics` |
|
||||
| Credit expiration visibility | reads `purchased_credits_expire_at` + `ai_bonus_credits.expires_at` |
|
||||
| Subscription metrics (MRR, churn) | derived from `ai_subscription_history` |
|
||||
|
||||
---
|
||||
|
||||
## 14. Observability
|
||||
|
||||
- **Revenue, credits sold, credits consumed** — from `ai_credit_ledger` grouped by `entry_type`, materialized into `ai_usage_statistics` for dashboard speed rather than aggregating the raw ledger on every page load.
|
||||
- **Provider costs, profit margins** — from `ai_usage_logs.internal_cost` vs. ledger revenue (Section 7).
|
||||
- **Feature/model usage, DAU, avg cost per request, avg credits per feature, cache savings** — all derivable from `ai_usage_logs` + `ai_usage_statistics` (Section 9).
|
||||
- **Infra**: today there is no Prometheus/Grafana in `nxtgauge-gitops` (confirmed — only an OpenTelemetry Collector shipping to OpenObserve). Recommend **OpenTelemetry metrics + OpenObserve dashboards** rather than introducing a second observability stack (Prometheus/Grafana) purely for billing — reuse what's already deployed and proven, add billing-specific metrics (`ai_ledger_writes_total`, `ai_reservation_holds_active`, `ai_charge_latency_ms`) as OTel instruments emitted from the Rust services, consistent with the existing collector pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 15. Technology Recommendations
|
||||
|
||||
| Technology | Recommendation | Why |
|
||||
|---|---|---|
|
||||
| Rust | Keep (already the whole backend) | No reason to introduce a second language for a billing subsystem embedded in an existing Rust service. |
|
||||
| PostgreSQL | Keep (already the whole backend, via sqlx) | Row-level locking (`SELECT FOR UPDATE`) and `UNIQUE` constraints are exactly the primitives this design leans on; no need for a specialized ledger DB at this scale (single-replica K3s, confirmed in audit). |
|
||||
| Redis | Keep (already used for rate limiting) | Extend existing sliding-window pattern; don't introduce a second rate-limit mechanism. |
|
||||
| LiteLLM | Keep (already deployed, working) | Explicitly your stated direction; extend with retries/health checks (Section 8), don't replace. |
|
||||
| OpenTelemetry | Keep/extend (collector already deployed) | Reuse existing OTel → OpenObserve pipeline rather than adding Prometheus/Grafana for one subsystem. |
|
||||
| Prometheus/Grafana | **Do not add** for this project alone | Confirmed absent today; introducing a whole second observability stack just for billing metrics is disproportionate to current single-replica, low-traffic scale — revisit only if OpenObserve genuinely can't serve the dashboard needs in Section 14. |
|
||||
| NATS | **Not needed yet** | No async event bus exists today and nothing in this design requires one — the reservation reaper (Section 4.2) and stats rollup (Section 9) are fine as simple scheduled jobs (`tokio::spawn` interval or a k8s CronJob) at current scale. Revisit if/when multiple services need to react to billing events asynchronously (e.g., a future notifications service). |
|
||||
| Kafka | **Not justified** | Same reasoning, stronger — Kafka is disproportionate to a single-replica K3s deployment with no current multi-consumer event-streaming need. |
|
||||
|
||||
---
|
||||
|
||||
## 16. Mermaid Diagrams
|
||||
|
||||
### 16.1 Overall Billing Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
FE[Frontend] --> GW[API Gateway]
|
||||
GW --> AUTH[Auth: JWT]
|
||||
AUTH --> MW[Ask Ash Enforcement Middleware]
|
||||
MW -->|plan/feature/model/quota checks| IDEM[Idempotency Layer]
|
||||
IDEM --> RES[Credit Reservation]
|
||||
RES --> LLM[LiteLLM]
|
||||
LLM --> MODEL[Ollama / future paid providers]
|
||||
MODEL --> LLM
|
||||
LLM --> SETTLE[Settlement: Capture or Release]
|
||||
SETTLE --> LEDGER[(ai_credit_ledger)]
|
||||
SETTLE --> USAGE[(ai_usage_logs)]
|
||||
USAGE --> STATS[(ai_usage_statistics)]
|
||||
STATS --> DASH[Admin Dashboard]
|
||||
LEDGER --> DASH
|
||||
```
|
||||
|
||||
### 16.2 Wallet Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Wallet[user_ai_subscriptions]
|
||||
MC[monthly_credits]
|
||||
PC[purchased_credits]
|
||||
BC[bonus_credits]
|
||||
RSV[reserved_credits]
|
||||
LCK[locked_credits]
|
||||
end
|
||||
REQ[Feature Request] -->|RESERVE| RSV
|
||||
RSV -->|CAPTURE on success| MC & PC & BC
|
||||
RSV -->|RELEASE on failure/timeout| Wallet
|
||||
ADMIN[Admin] -->|freeze| LCK
|
||||
ADMIN -->|unfreeze| Wallet
|
||||
```
|
||||
|
||||
### 16.3 Credit Purchase Flow
|
||||
|
||||
Confirmed directly (`apps/payments/src/main.rs`: `razorpay_order_id`, `razorpay_payment_id` columns on the existing `payments` table): the real gateway TraceCoins purchases already use is **Razorpay**, not PayU — the earlier draft's PayU references were fabricated. This diagram follows the same gateway for consistency, using a new AI-credit-specific order/verify flow (Section 2.1 confirmed none exists today) rather than reusing the `payments` table's TraceCoins-specific columns.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as User
|
||||
participant PAY as Payments Service
|
||||
participant RZP as Razorpay
|
||||
participant WAL as Wallet Service (Section 20 Appendix Q1: location TBD)
|
||||
participant LED as ai_credit_ledger
|
||||
|
||||
U->>PAY: POST /api/ai-credits/order {package_id}
|
||||
PAY->>PAY: insert ai_credit_orders row (status=PENDING)
|
||||
PAY-->>U: Razorpay order id / checkout params
|
||||
U->>RZP: pay
|
||||
RZP-->>PAY: POST /api/ai-credits/verify (webhook or client callback)
|
||||
PAY->>PAY: verify signature, idempotency check
|
||||
PAY->>PAY: ai_credit_orders.status = SUCCESS
|
||||
PAY->>WAL: grant purchased credits (service JWT, idempotency key)
|
||||
WAL->>LED: insert entry_type=purchase (idempotency_key)
|
||||
WAL->>WAL: update cached wallet balance
|
||||
WAL-->>PAY: 200 OK
|
||||
PAY-->>U: purchase confirmed notification
|
||||
```
|
||||
|
||||
### 16.4 AI Request Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as User
|
||||
participant MW as Enforcement Middleware
|
||||
participant ORCH as Orchestrator
|
||||
participant W as Wallet
|
||||
participant LLM as LiteLLM
|
||||
participant LED as Ledger
|
||||
|
||||
U->>MW: POST /api/ai/{feature}
|
||||
MW->>MW: validate plan, feature, model, quota
|
||||
MW->>ORCH: forward
|
||||
ORCH->>W: RESERVE credit_cost
|
||||
alt insufficient balance
|
||||
W-->>ORCH: reject
|
||||
ORCH-->>U: 402 Insufficient Credits
|
||||
else reserved
|
||||
ORCH->>LLM: chat_completion (with timeout/retry)
|
||||
alt LLM success
|
||||
LLM-->>ORCH: response + tokens
|
||||
ORCH->>W: CAPTURE reservation
|
||||
ORCH->>LED: insert consumption entry
|
||||
ORCH-->>U: 200 + response
|
||||
else LLM failure
|
||||
ORCH->>W: RELEASE reservation
|
||||
ORCH->>LED: insert reservation_release entry
|
||||
ORCH-->>U: 502/504 error, not charged
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### 16.5 Credit Deduction Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
START[charge_feature called] --> LOCK[SELECT ... FOR UPDATE on wallet row]
|
||||
LOCK --> CHECK{available_credits >= cost?}
|
||||
CHECK -->|No| FAIL[Rollback, return InsufficientCredits]
|
||||
CHECK -->|Yes| RESERVE[reserved += cost, available -= cost]
|
||||
RESERVE --> LEDGERW[Insert ledger row: reservation_hold]
|
||||
LEDGERW --> COMMIT[Commit transaction]
|
||||
COMMIT --> CALL[Call LiteLLM]
|
||||
CALL --> RESULT{Success?}
|
||||
RESULT -->|Yes| CAPTURE[New transaction: reserved -= cost, used += cost]
|
||||
CAPTURE --> LEDGERC[Insert ledger row: consumption]
|
||||
RESULT -->|No| RELEASE[New transaction: reserved -= cost, available += cost]
|
||||
RELEASE --> LEDGERR[Insert ledger row: reservation_release]
|
||||
```
|
||||
|
||||
### 16.6 Refund Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
TRIGGER[Refund Trigger] --> TYPE{Type}
|
||||
TYPE -->|Provider failure / timeout| AUTO[Automatic refund]
|
||||
TYPE -->|User dispute| MANUAL[Admin review queue]
|
||||
TYPE -->|Validation failure pre-LLM-call| NOCHARGE[No charge occurred - no refund needed]
|
||||
AUTO --> LOOKUP[Find ledger consumption entry by request_id]
|
||||
MANUAL --> APPROVE{Admin approves?}
|
||||
APPROVE -->|Yes| LOOKUP
|
||||
APPROVE -->|No| REJECT[ai_refunds.status = rejected]
|
||||
LOOKUP --> CREDIT[Ledger insert: refund entry, credits back to available]
|
||||
CREDIT --> AUDIT[ai_audit_logs entry]
|
||||
AUDIT --> NOTIFY[Notify user]
|
||||
```
|
||||
|
||||
### 16.7 Subscription Lifecycle
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> Free: signup (ensure_free_subscription)
|
||||
Free --> Active: upgrade + payment
|
||||
Active --> Active: renew (auto_renew=true)
|
||||
Active --> PastDue: payment failure
|
||||
PastDue --> Active: payment recovered (grace period)
|
||||
PastDue --> Free: grace period expired
|
||||
Active --> DowngradeScheduled: downgrade requested
|
||||
DowngradeScheduled --> Active: takes effect at period end (new plan)
|
||||
Active --> Cancelled: user cancels
|
||||
Cancelled --> Free: reverts at period end
|
||||
Free --> [*]
|
||||
```
|
||||
|
||||
### 16.8 Database ER Diagram (core billing tables)
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
ai_plans ||--o{ user_ai_subscriptions : "plan_id"
|
||||
user_ai_subscriptions ||--o{ ai_credit_ledger : "wallet_id"
|
||||
user_ai_subscriptions ||--o{ ai_reservation_holds : "wallet_id"
|
||||
user_ai_subscriptions ||--o{ ai_subscription_history : "user_id"
|
||||
ai_feature_costs ||--o{ ai_usage_logs : "feature_code"
|
||||
ai_usage_logs ||--o| ai_credit_ledger : "reference_id (consumption)"
|
||||
ai_credit_packages ||--o{ ai_credit_ledger : "reference_id (purchase)"
|
||||
ai_coupons ||--o{ ai_coupon_redemptions : "coupon_id"
|
||||
ai_coupon_redemptions }o--|| user_ai_subscriptions : "user_id"
|
||||
ai_bonus_credits ||--|| ai_credit_ledger : "ledger_entry_id"
|
||||
ai_credit_ledger ||--o| ai_refunds : "ledger_debit_entry_id"
|
||||
ai_audit_logs }o--|| user_ai_subscriptions : "target_user_id"
|
||||
```
|
||||
|
||||
### 16.9 Sequence Diagram — Admin Manual Credit Adjustment
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant A as Admin
|
||||
participant API as admin_ai handler
|
||||
participant W as Wallet
|
||||
participant L as ai_credit_ledger
|
||||
participant AL as ai_audit_logs
|
||||
|
||||
A->>API: POST /api/admin/ai/users/{id}/credits {credits, reason (required)}
|
||||
API->>API: require_admin(auth)
|
||||
API->>W: SELECT ... FOR UPDATE
|
||||
API->>W: bonus_credits_total += credits
|
||||
API->>L: insert entry_type=admin_adjustment_credit, balance_after (re-read, not stale)
|
||||
API->>AL: insert admin action record (before/after state, admin_user_id)
|
||||
API-->>A: 200 OK, new balance
|
||||
```
|
||||
|
||||
### 16.10 Deployment Diagram
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph K3s Cluster - nxtgauge namespace
|
||||
GW[gateway :8080]
|
||||
USR[users service]
|
||||
PAY[payments service]
|
||||
AIA[nxtgauge-ai-assistant]
|
||||
end
|
||||
subgraph nxtgauge-ai namespace
|
||||
LLM[LiteLLM :4000]
|
||||
OLL[Ollama]
|
||||
LLMDB[(litellm postgres)]
|
||||
end
|
||||
subgraph Shared
|
||||
PG[(Postgres - billing tables)]
|
||||
RD[(Redis - rate limits)]
|
||||
OTEL[OTel Collector]
|
||||
OO[OpenObserve]
|
||||
end
|
||||
GW --> USR
|
||||
GW --> PAY
|
||||
GW --> AIA
|
||||
USR --> PG
|
||||
USR --> RD
|
||||
USR --> LLM
|
||||
PAY --> PG
|
||||
PAY -->|Razorpay| EXT[Razorpay external]
|
||||
LLM --> OLL
|
||||
LLM --> LLMDB
|
||||
USR --> OTEL
|
||||
PAY --> OTEL
|
||||
OTEL --> OO
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 17. Scaling Strategy
|
||||
|
||||
Current deployment is single-replica across every service (confirmed in audit — no service runs >1 replica except one 2-replica overlay patch), which is appropriate for current traffic and should **not** be over-engineered against prematurely. Concrete, staged scaling triggers rather than pre-built infrastructure:
|
||||
|
||||
1. **Wallet write contention** (row-locking `SELECT FOR UPDATE` becomes a bottleneck): first sign is lock-wait metrics on `user_ai_subscriptions`/future ledger table — mitigate by ensuring reservation transactions are as short as possible (lock, mutate, commit — no LLM calls inside the locked transaction, which the design in Section 4.2 already guarantees by construction).
|
||||
2. **Ledger table growth**: append-only tables grow monotonically — partition `ai_credit_ledger` by month once it passes a few million rows (Postgres native partitioning), not before.
|
||||
3. **Read-heavy admin dashboard**: `ai_usage_statistics` rollups (Section 9) already exist specifically to keep dashboard queries off the hot `ai_usage_logs` table — this is the scaling mitigation, not a separate analytics DB, until proven insufficient.
|
||||
4. **Service replica count**: bump `users`/`payments` beyond 1 replica once request latency/CPU metrics (via the existing OTel pipeline) show saturation — no billing-specific work is needed to support this, since the design has no in-memory state (all state is in Postgres/Redis).
|
||||
|
||||
---
|
||||
|
||||
## 18. Disaster Recovery
|
||||
|
||||
- **Ledger as recovery anchor**: because `ai_credit_ledger` is append-only and the wallet balance columns are a derived cache, wallet corruption or a bad deploy that mis-updates cached balances is recoverable by replaying/summing the ledger — this is the core DR value of the immutability decision in Section 5, not just an audit nicety.
|
||||
- **Reservation reaper as self-healing**: any crash mid-request leaves, at worst, a `held` row in `ai_reservation_holds` that the background reaper releases after its TTL — no manual intervention needed for the common crash case.
|
||||
- **Purchase-verification replay safety**: idempotency key + unique constraint (Section 12) means a lost/replayed payment-gateway webhook during an outage can be safely reprocessed without double-crediting.
|
||||
- **Backups**: standard Postgres point-in-time recovery (confirm this is already configured at the cluster level — out of scope for this document to re-specify, but the ledger design assumes durable Postgres storage, which is table stakes).
|
||||
- **LiteLLM/Ollama outage**: circuit breaker (Section 8) fails fast with a clear "AI temporarily unavailable" instead of every in-flight request timing out and leaving reservations to expire naturally via the reaper — degrades gracefully, doesn't corrupt wallet state.
|
||||
|
||||
---
|
||||
|
||||
## 19. Risks & Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| **Building this on the wrong assumption a second time.** Section 2's first draft was fabricated and shipped before verification; that specific failure mode (unverified research treated as ground truth) is the single largest risk to this document's credibility. | Every implementation step in Phase 1 must be checked against the actual repo state (`grep`/`Read`/`cargo check`) immediately before writing code against it, not assumed from this document alone — schemas and code referenced here should be re-confirmed at implementation time in case this document itself drifts from reality as work proceeds. |
|
||||
| Migrating `company_ai_usage`/`job_seeker_ai_usage` users onto the new credit system creates an entitlement gap or double-counts usage during the transition. | Run the new system in shadow (log-only, no enforcement) alongside the existing daily-counter checks for one billing period, compare outcomes, then cut over — don't hard-switch a real, working quota mechanism that users currently depend on. |
|
||||
| Deleting the `company_ai_credits` stub (G2) breaks a caller that depends on its routes even though the underlying tables don't exist (i.e., every real call already 500s/400s today, so nothing user-facing should change) — worth confirming no frontend code silently swallows the current failure and expects it to keep failing the same way. | Grep `nxtgauge-frontend-solid`/`nxtgauge-admin-solid` for calls to `/api/companies/ai/{credits,generate,usage-history}` before removing the routes, to confirm current callers (if any) are already broken and not depending on today's specific error shape. |
|
||||
| Choosing where the new wallet/ledger code lives — `nxtgauge-backend-rust` (alongside TraceCoins, reusing its DB connection and auth) vs. `nxtgauge-ai-assistant` (alongside the actual LLM feature calls) — is an architectural decision this document has not made. | Resolve explicitly at the start of Phase 1 (Section 20) rather than defaulting silently; Section 8 flags the same open question for the LiteLLM call path specifically. |
|
||||
| Reservation reaper introduces a new failure mode (reaper itself down → holds never release). | Reaper runs as a simple scheduled job with its own health check/alert; holds also have a hard `expires_at` enforced at read time (`available_credits` calculation treats expired-but-not-yet-reaped holds as already released), so correctness doesn't depend on the reaper running promptly — only on it running eventually. |
|
||||
| Team/org support deferred (11.3) but requested features imply it eventually. | Explicitly scoped out of Phase 1–3; flagged as a prerequisite (org data model) rather than silently ignored, so it's a conscious backlog item, not a gap discovered later. |
|
||||
|
||||
---
|
||||
|
||||
## 20. Implementation Roadmap
|
||||
|
||||
**Phase 1 — Foundation: wallet, ledger, atomic charge path**
|
||||
Create `ai_plans`, `user_ai_subscriptions`, `ai_feature_costs`, `ai_usage_logs`, `ai_credit_ledger` (Sections 4, 5, 6, 9) as new migrations. Implement the reserve/capture/release wallet functions directly modeled on `TracecoinWalletRepository` (Section 2.3) — same `pool.begin()` + `FOR UPDATE` shape, new tables. Decide and document where this code physically lives (`nxtgauge-backend-rust` vs `nxtgauge-ai-assistant` — see Risks table above) before writing it. Delete or rewrite the non-functional `company_ai_credits` stub (`apps/companies/src/handlers/ai.rs`, G2) rather than patching a file whose core identity logic (`company_id`) doesn't work. No packages/plans-management/promotions yet — this phase only proves the money-movement primitive is correct.
|
||||
|
||||
**Phase 2 — Feature integration + quota consolidation**
|
||||
Wire the enforcement middleware (Section 3) into the actual AI feature call paths (in whichever service ends up owning them). Run new credit-based enforcement in shadow mode alongside the existing `company_ai_usage`/`job_seeker_ai_usage` counters (Risks table), then cut over once parity is confirmed. Wire the daily/monthly reset job (cron/k8s CronJob) — there's no existing-but-unwired job to find here (unlike the earlier draft's fabricated claim); it has to be written from scratch.
|
||||
|
||||
**Phase 3 — Monetization surface**
|
||||
Purchase flow (`ai_credit_packages`, order/verify against whichever payment gateway the business decides on — Section 16.3), `ai_refunds`, `ai_coupons` + `ai_coupon_redemptions`, `ai_promotions`, `ai_bonus_credits`, subscription lifecycle (`ai_subscription_history`, upgrade/downgrade/proration, Section 11.2). This is the largest net-new phase.
|
||||
|
||||
**Phase 4 — Admin & observability**
|
||||
Admin CRUD for packages/plans/feature pricing (Section 13); revenue/margin/usage dashboards on top of `ai_usage_statistics`; `ai_audit_logs` for admin actions.
|
||||
|
||||
**Phase 5 — Scale & future providers**
|
||||
Cost-basis switch to `provider_metered` when/if a paid LLM provider is added to LiteLLM; streaming reserve/capture (Section 8) if streaming UX is prioritized; team/org credit pools once an org data model exists (explicitly deferred, Section 11.3); partition the ledger table if/when volume warrants it (Section 17).
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Open questions for the team (not decided by this document)
|
||||
|
||||
1. **Where does this system live?** `nxtgauge-backend-rust` (co-located with TraceCoins and the existing auth/user model) or `nxtgauge-ai-assistant` (co-located with the actual LLM feature calls)? This is the single biggest open architectural decision and should be resolved before Phase 1 starts, not discovered mid-implementation.
|
||||
2. **Payment gateway — resolved during this correction pass**: confirmed directly as **Razorpay** (`apps/payments/src/main.rs`: `razorpay_order_id`/`razorpay_payment_id` columns), not PayU as the earlier fabricated draft claimed. Section 16.3 now reflects this. Still open: whether AI credit purchases reuse the existing `payments` table (adding AI-specific columns) or get their own `ai_credit_orders` table (Section 16.3 assumes the latter, for cleaner separation from TraceCoins) — a real decision for whoever owns the payments service.
|
||||
3. **Legacy quota system sunset**: who owns the decision to deprecate `company_ai_usage`/`job_seeker_ai_usage`, and what's the actual migration timeline — Phase 2 assumes it's eventually fully retired; confirm no other consumer depends on it first.
|
||||
4. **Pricing**: the illustrative credit/plan numbers in Sections 6, 10, 11 are placeholders for the shape of the schema, not proposed real prices — actual pricing is a business decision this document does not make.
|
||||
186
docs/ASK_ASH_IMPLEMENTATION_PLAN.md
Normal file
186
docs/ASK_ASH_IMPLEMENTATION_PLAN.md
Normal file
|
|
@ -0,0 +1,186 @@
|
|||
# Ask Ash — Remaining Implementation Plan
|
||||
|
||||
**Status as of 2026-07-06.** This is an execution checklist, not a redesign — it assumes the architecture in `ASK_ASH_BILLING_ARCHITECTURE.md` and the code already built (Section "Done" below). Each remaining item lists exact file paths, function signatures, and DB shapes so it can be implemented without re-deriving design decisions. Once implemented, hand back for testing/bug-hunting the same way prior phases were verified (real DB, real HTTP, real edge cases — not just `cargo check`).
|
||||
|
||||
---
|
||||
|
||||
## 0. What's already done (don't redo)
|
||||
|
||||
- **Wallet + ledger**: `crates/db/src/models/ai_credits.rs` — `AiCreditsRepository` with `ensure_wallet`, `try_reserve_credits`, `try_capture_reservation`, `try_release_reservation`, `add_purchased_credits`, `admin_adjust_credits`. Row-locked, idempotent, tested (`crates/db/tests/ai_credits.rs`, `ai_credits_reaper.rs`).
|
||||
- **Schema**: migrations `20260703210000_ai_credits_wallet`, `20260703220000_ai_credits_seed_features`, `20260705110000_ai_credit_packages` in `crates/db/migrations/`.
|
||||
- **Charge helper**: `apps/users/src/ai_credits.rs::charge_ai_feature` — reserve → work closure → capture/release, writes `ai_usage_logs`.
|
||||
- **HTTP (user-facing)**: `apps/users/src/handlers/ai_credits.rs` — `GET /api/ai-credits/wallet`, `POST /api/ai-credits/charge`, `GET /api/ai/usage/summary`, `GET /api/ai/usage/logs`.
|
||||
- **HTTP (purchase, PayU)**: `apps/payments/src/ai_credits.rs` + `apps/payments/src/payu.rs` — `GET /api/ai-credits` (packages), `POST /api/ai-credits/order`, `POST /api/ai-credits/verify`. Real PayU HMAC-SHA512 hash, cross-verified against an independent Python implementation. Also fixed: TraceCoins' `/api/payments/create-order`/`verify` (was calling a Beeceptor mock, not any real gateway; also had a broken `amount` vs `amount_inr` column bug).
|
||||
- **Admin package CRUD (payments)**: `apps/payments/src/ai_credits.rs::admin_router()` — `GET/POST /api/admin/ai-credits/packages`, `PATCH /api/admin/ai-credits/packages/{id}`. Nested in `apps/payments/src/main.rs`. **Not yet nested into the gateway routing test suite for the users-side admin endpoints below — those don't exist yet.**
|
||||
- **Admin credit adjustment primitive**: `AiCreditsRepository::admin_adjust_credits(pool, user_id, amount, is_add, reason, actor_id, idempotency_key)` in `crates/db/src/models/ai_credits.rs` — ADD grants bonus credits, DEDUCT consumes bonus→monthly→purchased in order, mandatory `reason`, ledger entry types `admin_adjustment_credit`/`admin_adjustment_debit`. **HTTP handler not yet written** (see Task 2 below).
|
||||
- **Gateway routing**: `apps/gateway/src/main.rs` — fixed a real bug where `/api/ai-credits/*` (order/verify/packages, hosted in `payments`) silently fell through to the generic `/api/ai` → `users` rule and would have 404'd in production. Added `/api/admin/ai-credits/packages` → `payments` routing. Both covered by `#[cfg(test)] mod tests` in the same file — **extend this test module** when adding the users-side admin routes in Task 2.
|
||||
- **Cron jobs**: `apps/cron/src/tasks/ai_credits.rs` — `sweep_expired_reservation_holds` (every 2 min), `reset_stale_daily_ai_usage` (hourly). Verified against the real running binary.
|
||||
- **JWT crypto fix**: `crates/auth/Cargo.toml` + `crates/contracts/Cargo.toml` — added `features = ["rust_crypto"]` to the `jsonwebtoken` dependency. Was causing every authenticated request to panic its worker thread.
|
||||
- **Real features wired to credits**: `apps/users/src/handlers/ai.rs::ai_generate_job_field` (title/description/skills/category — all four, one handler) and `::ai_generate_cover_letter`, both migrated off the old `company_ai_usage`/`job_seeker_ai_usage` daily-counter onto `charge_ai_feature`. Fixed a real bug found while testing this: `try_release_reservation` wasn't rolling back `daily_actions_used`/`daily_credits_used`, so a failed request permanently burned daily quota even with no charge — fixed, regression-tested.
|
||||
- **Frontend fixes**: `nxtgauge-frontend-solid/src/lib/payu.ts` split into `submitPayuCheckout` (takes an already-built order) + `openPayuCheckout` (creates a TraceCoins order, then delegates) — fixes a bug where `CreditsPage.tsx`'s AI-credit checkout was silently creating a second, mismatched order. `CompanyJobsPage.tsx`'s description/skills buttons fixed to call the one real `/api/ai/generate-job-field` endpoint instead of two nonexistent ones.
|
||||
|
||||
**Known pre-existing gap, not caused by this work, not yet fixed**: `nxtgauge-admin-solid`'s existing TraceCoins credit-adjustment page (`src/routes/admin/credit.tsx`) calls `/api/admin/credits/{adjust,ledger,reconcile,balance}` — **none of these exist anywhere in the backend.** That admin feature doesn't work today, independent of AI credits. Not in scope here unless you want it added — flagging so it's a conscious decision, not a surprise.
|
||||
|
||||
---
|
||||
|
||||
## Task 1 — Finish the admin endpoints for AI credits wallet (users service)
|
||||
|
||||
**Files**: `apps/users/src/handlers/ai_credits.rs` (add handlers), `apps/users/src/handlers/mod.rs` (already has `ai_credits` registered), `apps/users/src/main.rs` (nest a new admin router).
|
||||
|
||||
Add to `apps/users/src/handlers/ai_credits.rs`:
|
||||
|
||||
```rust
|
||||
pub fn admin_router() -> Router<AppState> {
|
||||
Router::new()
|
||||
.route("/balance", get(admin_balance))
|
||||
.route("/ledger", get(admin_ledger))
|
||||
.route("/adjust", post(admin_adjust))
|
||||
.route("/reconcile", get(admin_reconcile))
|
||||
}
|
||||
```
|
||||
|
||||
- **`GET /balance?userId=<uuid>`** — `require_admin(&auth)` gate (same pattern as `apps/payments/src/ai_credits.rs::admin_list_packages`). Parse `userId` query param, call `AiCreditsRepository::ensure_wallet`, return the same `WalletDto` shape already defined in this file (reuse it, don't redefine).
|
||||
- **`GET /ledger?userId=<uuid>&page=&limit=`** — paginated query against `ai_credit_ledger WHERE wallet_id = (SELECT id FROM user_ai_subscriptions WHERE user_id = $1) ORDER BY created_at DESC`. Response shape `{ data: [...], total: i64 }` to match the existing TraceCoins ledger UI pattern in `nxtgauge-admin-solid/src/routes/admin/credit.tsx` (`GET /api/admin/credits/ledger?userId=`).
|
||||
- **`POST /adjust`** — body `{ user_id: Uuid, amount: i32, type: "ADD" | "DEDUCT", reason: String, reference_id: Option<Uuid> }` (exact shape of the existing, already-defined-but-backend-missing TraceCoins contract in `credit.tsx` line ~103 — reuse the field names so the admin UI's fetch call is identical apart from the URL). Validate `reason` is non-empty (return 400 `INVALID_REASON` if blank — this is deliberately mandatory, not optional, per the architecture doc's "every admin-issued credit change must be explained" rule). Call `AiCreditsRepository::admin_adjust_credits(pool, body.user_id, body.amount, body.type == "ADD", &body.reason, auth.user_id, idempotency_key)`. Generate a server-side idempotency key if the client doesn't send one (e.g. `format!("admin-adjust:{}:{}", auth.user_id, uuid::Uuid::new_v4())`) — or better, require the frontend to send one so a double-click can't double-adjust (recommended: frontend generates a UUID per form submission attempt, resent unchanged on retry).
|
||||
- **`GET /reconcile?from=<date>&to=<date>`** — sums `ai_credit_ledger.credits` grouped by `entry_type` within the date range, plus a per-wallet drift check: for a sample (or all, if the table is still small) of wallets touched in the range, recompute `SUM(credits) FROM ai_credit_ledger WHERE wallet_id = X` and compare to the wallet's derived `available_credits()` — flag any mismatch. This is the concrete "ledger is the source of truth, reconcile against it" mechanism from the architecture doc's Section 5.
|
||||
|
||||
**Error handling**: reuse `credits_error_response` already in this file; add any new error codes needed (e.g. a 400 for missing/invalid `userId` query param — just return `(StatusCode::BAD_REQUEST, Json(json!({"error": "..."})))` inline, no need for a new `AiCreditsError` variant for pure request-parsing issues).
|
||||
|
||||
**Wire it up** in `apps/users/src/main.rs`:
|
||||
```rust
|
||||
.nest("/api/admin/ai-credits", handlers::ai_credits::admin_router())
|
||||
```
|
||||
Place this near the existing `.nest("/api/ai-credits", handlers::ai_credits::router())` line.
|
||||
|
||||
**Gateway**: no gateway change needed — `/api/admin/ai-credits/balance` etc. don't match the `/api/admin/ai-credits/packages` rule just added, so they correctly fall through to the existing generic `/api/admin/` → `users_url` catch-all. **But add test cases** to `apps/gateway/src/main.rs`'s existing `#[cfg(test)] mod tests` confirming this (the pattern is already there — `admin_ai_credits_routes_split_correctly` already asserts `/api/admin/ai-credits/balance` etc. route to `users` — just make sure the real handlers now exist to back that assumption).
|
||||
|
||||
**Verification checklist** (mirror how every prior phase was tested — spin up a throwaway Postgres+Redis via Docker, run migrations, start the real binary, hit it with `curl` + a minted JWT):
|
||||
- Admin balance for a user with no wallet yet auto-provisions Free plan (same as the user-facing endpoint).
|
||||
- Ledger pagination returns newest-first, respects `page`/`limit`.
|
||||
- Adjust ADD: bonus_credits_total increases, ledger row `entry_type='admin_adjustment_credit'`, `credits > 0`.
|
||||
- Adjust DEDUCT: consumes bonus→monthly→purchased in that order (write a test wallet with credits in multiple pools and confirm the split), rejects with `InsufficientCredits` if amount exceeds `available_credits()`.
|
||||
- Adjust with blank `reason` → 400, no DB write.
|
||||
- Adjust twice with the same idempotency key → second call is a no-op (wallet unchanged, no second ledger row) — this exact behavior is already implemented in `admin_adjust_credits`, just confirm the HTTP layer passes the key through correctly.
|
||||
- Reconcile flags a deliberately-introduced drift (e.g. manually `UPDATE ai_credit_ledger SET credits = credits + 999` on one row in a test DB, confirm the endpoint reports the mismatch).
|
||||
|
||||
---
|
||||
|
||||
## Task 2 — Admin UI: AI credits tabs in nxtgauge-admin-solid
|
||||
|
||||
**Files**: `src/routes/admin/pricing.tsx` (add a tab), `src/routes/admin/credit.tsx` (add a tab). Do not create new route files or new sidebar entries — the existing sidebar entries for "Pricing" and "Credits" (`src/components/AdminSidebar.tsx` lines ~237-248) already navigate to these files; adding a tab is enough.
|
||||
|
||||
### 2a. `pricing.tsx` — "AI Credit Packages" tab
|
||||
|
||||
Add a third tab value alongside the existing `view` signal's `"packages" | "create"` (check the exact signal name/values in the file — described as "two tabs... via local `view` signal" in the research). New tab: `"ai_packages"`.
|
||||
|
||||
- List view: `GET /api/admin/ai-credits/packages` (admin-scoped, shows inactive too) → render as a table, same `.data-table`/`.table-card` classes as the existing TraceCoins package table. Columns: Name, Credits, Price (₹, convert from paise: `price_inr / 100`), Active toggle, Edit.
|
||||
- Create form: Name*, Description, Credits*, Price (₹)* → convert rupees to paise before `POST /api/admin/ai-credits/packages` (body `{ name, description, credits, price_inr }`).
|
||||
- Inline edit / toggle active: `PATCH /api/admin/ai-credits/packages/{id}` with whichever fields changed (all optional in the body per Task 1's `UpdatePackageRequest` shape — `{ name?, description?, credits?, price_inr?, is_active? }`).
|
||||
- Auth: same `authHeaders()` helper already in this file (`sessionStorage.getItem('nxtgauge_admin_access_token')`).
|
||||
|
||||
### 2b. `credit.tsx` — "AI Credits" tab
|
||||
|
||||
Add a fourth tab alongside the existing `ledger | adjust | reconcile` (`activeTab` signal). New tab: `"ai_credits"`, itself with the same three sub-views (or just reuse three separate tab entries prefixed "AI " — whichever reads more naturally in the existing tab bar; the file already copy-pastes the tab-bar markup between sections, so follow that convention).
|
||||
|
||||
- **Balance & Ledger**: `GET /api/admin/ai-credits/balance?userId=`, `GET /api/admin/ai-credits/ledger?userId=`. Same User ID input field as the existing TraceCoins version.
|
||||
- **Reward/Deduct**: same form fields as the existing TraceCoins "Reward/Deduct" tab (User ID, Amount, Type ADD/DEDUCT, Reason required, Reference ID optional) → `POST /api/admin/ai-credits/adjust`. This is deliberately a copy of the existing form's shape, just pointed at a new endpoint — reuse the component/markup, don't redesign it.
|
||||
- **Reconcile**: `GET /api/admin/ai-credits/reconcile?from=&to=`.
|
||||
|
||||
**Do not touch** the existing TraceCoins tabs/endpoints in either file — those stay exactly as they are (including the fact that their backend doesn't exist yet, per the known gap noted in Section 0; that's explicitly out of scope here).
|
||||
|
||||
**Verification**: once Task 1's backend is live, click through every button in both new tabs against a real running admin-solid dev server + real backend (not just a TypeScript compile check) — create a package, edit its price, deactivate it, look up a real user's AI wallet, adjust their credits, confirm the ledger updates.
|
||||
|
||||
---
|
||||
|
||||
## Task 3 — Ollama security hardening
|
||||
|
||||
**Context already gathered, don't re-research**: `nxtgauge-gitops/apps/ollama/base/` — `deployment.yaml` (`OLLAMA_HOST=0.0.0.0:11434`, resource limits `cpu: 1000m / memory: 1500Mi`, no auth env vars — Ollama has no built-in auth), `service.yaml` (`ClusterIP`, not internet-exposed — no `ingress.yaml` present), no `NetworkPolicy` anywhere restricting which pods can reach it. Cluster is Flux-managed (confirmed via existing `NetworkPolicy` resources in the `flux-system`/`data` namespaces) — **changes must go into the `nxtgauge-gitops` repo and be committed/reconciled through Flux, never applied directly via `kubectl apply` against the live cluster.**
|
||||
|
||||
Concrete work items, roughly in priority order:
|
||||
|
||||
1. **NetworkPolicy restricting ingress to the `ollama` Service** in `nxtgauge-gitops/apps/ollama/base/networkpolicy.yaml` (new file, add to `kustomization.yaml`'s `resources:` list). Restrict to pods with labels matching the `users`, `payments`, and `nxtgauge-ai-assistant` deployments (check their actual pod labels in `nxtgauge-gitops/apps/*/base/deployment.yaml` before writing the selector — don't guess label values). Confirm the cluster's CNI actually enforces `NetworkPolicy` (k3s ships an optional lightweight controller; this needs verifying against the live cluster, not assumed) before treating this as a real control rather than a documentation-only artifact.
|
||||
2. **Application-layer prompt/response length caps**, enforced in `apps/users/src/handlers/ai.rs` before calling `call_ollama_inline` — use `ai_feature_costs.max_input_tokens`/`max_output_tokens` (already columns in the schema, currently unenforced anywhere) as a character-count approximation (e.g. `max_chars ≈ max_tokens * 4`) since a real tokenizer isn't wired up. Reject oversized requests with 400 before spending a reservation.
|
||||
3. **Truncated prompt/response audit logging** — add `prompt_preview`/`response_preview` columns to `ai_usage_logs` (new migration, short varchar, e.g. first 200 chars), populate in `apps/users/src/ai_credits.rs::log_usage`. Mirrors the pattern already used (but currently dead) in the broken `apps/companies/src/handlers/ai.rs` stub — same idea, done for real this time.
|
||||
4. **TLS between services and Ollama** — recommend but treat as a separate, heavier infra task (cert-manager + Ollama TLS termination or a service mesh); don't block the above items on this.
|
||||
5. **Health-check + circuit breaker** for Ollama outages — already flagged as an open item in the architecture doc's Section 8; implement as a periodic `GET {OLLAMA_BASE_URL}/` poll (Ollama's root path responds 200 when healthy) feeding a simple in-memory circuit-breaker state in `apps/users`, returning a fast "AI temporarily unavailable" instead of every request timing out individually.
|
||||
|
||||
---
|
||||
|
||||
## Task 4 — Wire remaining real AI features to credits
|
||||
|
||||
Before writing any code, **audit which of these have a real (non-404) backend handler** — several already-shipped frontend buttons turned out to call nonexistent endpoints (found twice now: `CompanyJobsPage.tsx`'s description/skills buttons, and the entire pre-existing `admin/credits/*` page). Grep `apps/users/src/handlers/*.rs` for actual `.route(...)` registrations, don't trust frontend `fetch()` calls as evidence a backend exists.
|
||||
|
||||
Candidates from the original feature list: AI Chat, Help Center Assistant, Knowledge Base Search, Resume Improvement, Professional Profile Improvement, AI Suggestions, AI Form Filling, Requirement Analysis, Ticket Summaries, Admin Insights.
|
||||
|
||||
For each real handler found, migrate it the same way `ai_generate_job_field`/`ai_generate_cover_letter` were migrated:
|
||||
1. Identify what daily-quota/rate-limit mechanism it currently uses (if any).
|
||||
2. Seed a feature cost row in `ai_feature_costs` if one doesn't already exist for it (4 exist today: `help_answer`, `jd_generate`, `cover_letter_generate`, `resume_improve` — check `crates/db/migrations/20260703220000_ai_credits_seed_features.up.sql`).
|
||||
3. Wrap the existing generation call (`call_ollama_inline` or equivalent) in `crate::ai_credits::charge_ai_feature(...)`, following the exact pattern in `ai_generate_job_field`.
|
||||
4. Test the failure path specifically (point `OLLAMA_BASE_URL` at an unreachable host, confirm the reservation releases and daily counters roll back — this exact bug was found and fixed once already; don't assume the fix generalizes without testing each new call site).
|
||||
|
||||
---
|
||||
|
||||
## Task 5 — Refund architecture
|
||||
|
||||
New table (migration): `ai_refunds` — `id, user_id, ledger_debit_entry_id (FK ai_credit_ledger.id), credits_refunded, reason (provider_failure|timeout|validation_failure|manual), initiated_by (system|admin|user_dispute), status (pending|approved|rejected|completed), created_at, resolved_at`.
|
||||
|
||||
Two paths:
|
||||
- **Automatic**: triggered from `charge_ai_feature`'s error path — currently a failed request just releases the *reservation* (never charged in the first place, so nothing to refund). A genuine refund is needed only if a charge was *captured* and the feature then failed asynchronously after the fact (e.g., a downstream step fails after the LLM call succeeded and credits were captured) — audit whether this can currently happen anywhere; if not, automatic refunds may not be needed yet and this table exists for the manual path only.
|
||||
- **Manual (admin)**: new endpoint `POST /api/admin/ai-credits/refunds` — body `{ user_id, ledger_debit_entry_id, reason, notes? }`, looks up the original debit ledger entry, credits back via a new ledger entry `entry_type='refund'` (extend `AiCreditsRepository` with a `refund_credits` function mirroring `admin_adjust_credits`'s ADD path but tagged distinctly), writes an `ai_refunds` row, requires `require_admin`.
|
||||
|
||||
---
|
||||
|
||||
## Task 6 — Coupons, promotions, referral, admin-issued credits
|
||||
|
||||
New tables (one migration): `ai_coupons` (`id, code UNIQUE, discount_type, value, applicable_package_ids UUID[], applicable_plan_codes TEXT[], max_redemptions, redemptions_used, valid_from, valid_until, is_active`), `ai_coupon_redemptions` (`coupon_id, user_id, redeemed_at, UNIQUE(coupon_id, user_id)` — enforces single-use-per-user at the DB level, not just application logic), `ai_promotions` (`id, name, trigger_type, bonus_credits, applicable_plan_codes TEXT[], valid_from, valid_until, is_active`), `ai_bonus_credits` (`id, user_id, source_type, source_reference_id, credits, expires_at, ledger_entry_id FK`).
|
||||
|
||||
Coupon redemption at purchase time: extend `apps/payments/src/ai_credits.rs::create_order` to accept an optional `coupon_code`, validate against `ai_coupons` (active, not expired, redemption count under max, not already redeemed by this user via `ai_coupon_redemptions`), apply the discount to the PayU order amount, and only insert into `ai_coupon_redemptions` after `verify_order` confirms payment success (not at order-creation time, to avoid burning a redemption on an abandoned checkout).
|
||||
|
||||
---
|
||||
|
||||
## Task 7 — Subscription lifecycle
|
||||
|
||||
New table: `ai_subscription_history` (`id, user_id, from_plan_id, to_plan_id, change_type (upgrade|downgrade|cancel|renew), proration_credits, effective_at, created_by, created_at`).
|
||||
|
||||
Add columns to `user_ai_subscriptions` (already has `billing_cycle`, `auto_renew` from the initial migration — check before adding duplicates): confirm `status` supports `active|past_due|cancelled|expired` (currently only `'active'` is ever written).
|
||||
|
||||
Logic (new module, e.g. `apps/users/src/ai_subscription.rs`):
|
||||
- **Upgrade**: immediate — set new `plan_id`, new `monthly_credits_total`, compute prorated bonus (`remaining_days / days_in_period × (new_monthly - old_monthly)`), grant via a new ledger entry type `subscription_grant`, write `ai_subscription_history`.
|
||||
- **Downgrade**: don't change `plan_id` immediately — write `ai_subscription_history` with `effective_at = current_period_end`, and have the (currently-hourly) cron reset job also check for and apply due downgrades when it rolls a wallet into a new period.
|
||||
- **Trial plans**: add `is_trial`, `trial_days` columns to `ai_plans`; on trial expiry (checked in the same cron job), auto-downgrade to `free` unless the user has since upgraded/purchased.
|
||||
|
||||
---
|
||||
|
||||
## Task 8 — Credit expiration enforcement
|
||||
|
||||
`user_ai_subscriptions.purchased_credits_expire_at` already exists as a column but nothing sets or checks it.
|
||||
|
||||
1. Set it in `apps/payments/src/ai_credits.rs::verify_order` when granting purchased credits (e.g. `NOW() + INTERVAL '12 months'` — confirm the actual policy with whoever owns pricing; the architecture doc recommends this but doesn't mandate a specific duration).
|
||||
2. Add expiry-first consumption order to `try_capture_reservation`/`admin_adjust_credits`'s DEDUCT path — currently consumption order is bonus → monthly → purchased; per the architecture doc's Section 4.1, it should be *expiring-soonest-first* across whichever pools have an expiry, which today only `purchased_credits` does. Given only one pool expires, the existing order (bonus → monthly → purchased) already happens to leave purchased-with-expiry for last, which is backwards — purchased credits closest to expiring should be spent *before* they're lost. This needs revisiting once expiry is actually set.
|
||||
3. New cron task in `apps/cron/src/tasks/ai_credits.rs`: sweep wallets where `purchased_credits_expire_at < NOW()` and `purchased_credits_total > purchased_credits_used`, write a ledger entry `entry_type='expiration'` for the unused remainder, zero out the expired portion.
|
||||
|
||||
---
|
||||
|
||||
## Task 9 — Token cost engine / provider cost / margin tracking
|
||||
|
||||
`ai_usage_logs` already has `cached_tokens`, `provider_reported_cost`, `internal_cost`, `credit_unit_price_at_time` columns (added in the Phase 1 migration) — none are populated.
|
||||
|
||||
1. New table `ai_model_cost_config`: `model_alias, cost_per_1k_input_tokens, cost_per_1k_output_tokens, cost_basis ('compute_amortized'|'provider_metered'), effective_from`.
|
||||
2. In `apps/users/src/ai_credits.rs::charge_ai_feature`/`log_usage`, after the LLM call, compute `internal_cost = tokens_used × cost_per_1k(model, cost_basis) / 1000` using the config table, and store it. `provider_reported_cost` stays `NULL` until a real per-token-billed provider is added to LiteLLM (self-hosted Ollama has no real per-request cost signal today).
|
||||
3. Margin (`credits_charged × credit_unit_price − internal_cost`) is a read-time analytics computation for the admin dashboard (Task 2/reconcile-adjacent), not stored per-row.
|
||||
|
||||
---
|
||||
|
||||
## Task 10 — Observability
|
||||
|
||||
Per the architecture doc's explicit recommendation: **do not add Prometheus/Grafana** — this cluster doesn't run them (confirmed: only an OpenTelemetry Collector shipping to OpenObserve exists in `nxtgauge-gitops`). Add OTel metrics (`ai_ledger_writes_total`, `ai_reservation_holds_active`, `ai_charge_latency_ms`) emitted from `apps/users`/`apps/payments` via whatever OTel Rust crate is already used elsewhere in this workspace (check for one before adding a new dependency), feeding the existing collector pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Suggested order
|
||||
|
||||
1. Task 1 + 2 (finish the admin dashboard you're already partway through) — no design decisions left, just execution.
|
||||
2. Task 3 (Ollama security) — real, currently-open vulnerability.
|
||||
3. Task 4 (wire remaining features) — mostly mechanical once the audit is done, directly increases how much of the system is actually load-bearing.
|
||||
4. Tasks 5–9 in whatever order matches business priority — none block each other.
|
||||
5. Task 10 last — needs the others to exist before there's anything meaningful to instrument.
|
||||
Loading…
Add table
Reference in a new issue