nxtgauge-backend-rust/docs/LIVE_SERVER_RUNBOOK.md

Nxtgauge — Live Server Runbook

Steps that require a running database and/or server to complete.
Run these on the server where Postgres + Redis are already up.

---

✅ Completed — Database migrations (applied to prod, 2026-08-13)

All four DB tasks below are done and confirmed against the live production DB.

Migration	What	Status
20260627030000_wallet_full.up.sql	Re-enabled (was mistakenly .skip-ed); tracecoin_holds + tracecoin_buckets now live	✅ Applied
20260703210000_ai_credits_wallet.up.sql	Rewrote as idempotent (IF NOT EXISTS); ai_reservation_holds + ai_credit_ledger now live	✅ Applied
20260813000000_fix_ai_credits_timestamp_types.up.sql	9 TIMESTAMP → TIMESTAMPTZ columns across AI tables; data intact and UTC-correct	✅ Applied
nxtgauge_test DB	Dedicated test database created, schema mirrored from prod, wired into Forgejo CI via TEST_DATABASE_URL secret	✅ Done

/wallet/me/holds returns real data. All 8 backend integration tests pass in CI.

---

1. Deploy backend changes

cd nxtgauge-backend-rust
cargo build --release

Then restart the affected services:

# Restart all profession services (wallet/me/holds route)
systemctl restart nxtgauge-photographers nxtgauge-developers nxtgauge-tutors \
  nxtgauge-makeup_artists nxtgauge-fitness_trainers nxtgauge-catering_services \
  nxtgauge-video_editors nxtgauge-graphic_designers nxtgauge-social_media_managers

# Restart job_seekers (printpdf upgrade)
systemctl restart nxtgauge-job_seekers

---

2. Run security checks (no DB needed)

cd nxtgauge-backend-rust

# CVE scan
cargo audit

# Policy check (licenses, bans, CVE with acknowledged ignores)
cargo deny check advisories bans

Both should exit 0. If a new CVE appears, add it to deny.toml under [advisories] ignore with a comment explaining the risk.

---

3. Set up schemathesis API fuzzing

Schemathesis fuzzes your real API from an OpenAPI spec. It needs:

• The server running (with DB connected)
• An OpenAPI spec (openapi.json)

3a. Install schemathesis

pip install schemathesis
# or with uv:
uv tool install schemathesis

3b. Generate the OpenAPI spec

Note: The axum services don't yet have utoipa annotations. Until then, use the manually maintained spec in docs/openapi.wallet-holds.json or skip to 3c.

To add utoipa: add utoipa and utoipa-axum to each service's Cargo.toml, annotate handlers with #[utoipa::path], and mount a /openapi.json endpoint.

3c. Run the fuzzer

schemathesis run openapi.json \
  --base-url http://localhost:3000 \
  --auth "Bearer <admin_or_test_jwt>" \
  --checks all \
  --hypothesis-settings max_examples=200

# Quick smoke-check:
schemathesis run openapi.json \
  --base-url http://localhost:3000 \
  --auth "Bearer <jwt>" \
  --checks not_a_server_error

---

4. Run Playwright end-to-end tests

Playwright tests live in nxtgauge-frontend-solid/tests/. Need full stack running.

cd nxtgauge-frontend-solid
npx playwright install --with-deps chromium   # one-time

export PLAYWRIGHT_BASE_URL=https://nxtgauge.com
npx playwright test
npx playwright test --grep @smoke             # smoke only
npx playwright show-report                   # view results

Note: A few e2e test files still have localhost:3001 URLs — pending cleanup, update to the staging/prod URL before running against live.

---

5. Backend integration tests (CI — already wired)

The nxtgauge_test DB and Forgejo CI secret are already set up. To run locally:

export TEST_DATABASE_URL=postgres://user:pass@localhost:5432/nxtgauge_test
cd nxtgauge-backend-rust
cargo test --test ai_credits -- --test-threads=1

---

6. Frontend linter

cd nxtgauge-frontend-solid
npm run lint

---

7. TypeScript check

cd nxtgauge-frontend-solid
npx tsc --noEmit --skipLibCheck

Should exit 0.

---

Summary

Task	Needs DB	Needs server	Status
DB migrations	✅	✅	✅ Done (prod)
Build + deploy backend	—	—	Pending next release
cargo audit / cargo deny	—	—	Run anytime
Schemathesis fuzzing	✅	✅	Pending OpenAPI spec
Playwright e2e	✅	✅	Pending URL cleanup
Backend integration tests	✅	—	✅ Green in CI
npm run lint	—	—	✅ Passing
tsc --noEmit	—	—	Run anytime