nxtgauge-backend-rust/docs/LIVE_SERVER_RUNBOOK.md

Nxtgauge — Live Server Runbook

Steps that require a running database and/or server to complete.
Run these on the server where Postgres + Redis are already up.

---

✅ Completed — Database migrations (applied to prod, 2026-08-13)

All four DB tasks below are done and confirmed against the live production DB.

Migration	What	Status
20260627030000_wallet_full.up.sql	Re-enabled (was mistakenly .skip-ed); tracecoin_holds + tracecoin_buckets now live	✅ Applied
20260703210000_ai_credits_wallet.up.sql	Rewrote as idempotent (IF NOT EXISTS); ai_reservation_holds + ai_credit_ledger now live	✅ Applied
20260813000000_fix_ai_credits_timestamp_types.up.sql	9 TIMESTAMP → TIMESTAMPTZ columns across AI tables; data intact and UTC-correct	✅ Applied
nxtgauge_test DB	Dedicated test database created, schema mirrored from prod, wired into Forgejo CI via TEST_DATABASE_URL secret	✅ Done

/wallet/me/holds returns real data. All 8 backend integration tests pass in CI.

---

⏳ Pending — DB migrations for AI credit purchases (2026-08-14)

Three new migrations need to be applied to prod. Run in order (sqlx-migrate handles this, but listed explicitly so you can verify):

# On the live server, inside nxtgauge-backend-rust:
git pull origin high-performance
sqlx migrate run --database-url "$DATABASE_URL"

That single command applies all three in chronological order. Verify after:

sqlx migrate info --database-url "$DATABASE_URL"

All three should show applied.

Migration	What	Why
20260814000000_fix_ai_credit_packages_pricing_and_orders.up.sql	Fixes prices (99→9900 paise etc.) + creates ai_credit_orders with coupon_code + discount_applied columns	AI credit purchase flow was completely broken: no orders table, wrong prices
20260814010000_ai_credit_packages_role_scoping.up.sql	Adds applicable_roles TEXT[] column to ai_credit_packages	Admin UI now supports restricting packages to specific roles
20260814020000_ai_coupons.up.sql	Creates ai_coupons + ai_coupon_redemptions tables	Backend create_order/verify_order reference these tables

After applying, redeploy the payments service:

systemctl restart nxtgauge-payments

---

1. Deploy backend changes

cd nxtgauge-backend-rust
cargo build --release

Then restart the affected services:

# Restart all profession services (wallet/me/holds route)
systemctl restart nxtgauge-photographers nxtgauge-developers nxtgauge-tutors \
  nxtgauge-makeup_artists nxtgauge-fitness_trainers nxtgauge-catering_services \
  nxtgauge-video_editors nxtgauge-graphic_designers nxtgauge-social_media_managers

# Restart job_seekers (printpdf upgrade)
systemctl restart nxtgauge-job_seekers

---

2. Run security checks (no DB needed)

cd nxtgauge-backend-rust

# CVE scan
cargo audit

# Policy check (licenses, bans, CVE with acknowledged ignores)
cargo deny check advisories bans

Both should exit 0. If a new CVE appears, add it to deny.toml under [advisories] ignore with a comment explaining the risk.

---

3. Set up schemathesis API fuzzing

Schemathesis fuzzes your real API from an OpenAPI spec. It needs:

• The server running (with DB connected)
• An OpenAPI spec (openapi.json)

3a. Install schemathesis

pip install schemathesis
# or with uv:
uv tool install schemathesis

3b. Generate the OpenAPI spec

Note: The axum services don't yet have utoipa annotations. Until then, use the manually maintained spec in docs/openapi.wallet-holds.json or skip to 3c.

To add utoipa: add utoipa and utoipa-axum to each service's Cargo.toml, annotate handlers with #[utoipa::path], and mount a /openapi.json endpoint.

3c. Run the fuzzer

schemathesis run openapi.json \
  --base-url http://localhost:3000 \
  --auth "Bearer <admin_or_test_jwt>" \
  --checks all \
  --hypothesis-settings max_examples=200

# Quick smoke-check:
schemathesis run openapi.json \
  --base-url http://localhost:3000 \
  --auth "Bearer <jwt>" \
  --checks not_a_server_error

---

4. Run Playwright end-to-end tests

Playwright tests live in nxtgauge-frontend-solid/tests/. Need full stack running.

cd nxtgauge-frontend-solid
npx playwright install --with-deps chromium   # one-time

export PLAYWRIGHT_BASE_URL=https://nxtgauge.com
npx playwright test
npx playwright test --grep @smoke             # smoke only
npx playwright show-report                   # view results

Note: A few e2e test files still have localhost:3001 URLs — pending cleanup, update to the staging/prod URL before running against live.

---

5. Backend integration tests (CI — already wired)

The nxtgauge_test DB and Forgejo CI secret are already set up. To run locally:

export TEST_DATABASE_URL=postgres://user:pass@localhost:5432/nxtgauge_test
cd nxtgauge-backend-rust
cargo test --test ai_credits -- --test-threads=1

---

6. Frontend linter

cd nxtgauge-frontend-solid
npm run lint

---

7. TypeScript check

cd nxtgauge-frontend-solid
npx tsc --noEmit --skipLibCheck

Should exit 0.

---

Summary

Task	Needs DB	Needs server	Status
DB migrations	✅	✅	✅ Done (prod)
Build + deploy backend	—	—	Pending next release
cargo audit / cargo deny	—	—	Run anytime
Schemathesis fuzzing	✅	✅	Pending OpenAPI spec
Playwright e2e	✅	✅	Pending URL cleanup
Backend integration tests	✅	—	✅ Green in CI
npm run lint	—	—	✅ Passing
tsc --noEmit	—	—	Run anytime