Verification documents were being stored/rendered as permanent, unsigned
B2 URLs across every role's admin review and self-service dashboard.
Add StorageClient::presign() plus two mediating endpoints (admin and
self-service) so viewers always get a short-lived signed URL instead of
the raw storage link.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Require admin role on role/module/permission management endpoints
that previously accepted any authenticated user (privilege escalation)
- Add server-side captcha generation/verification (Redis-backed,
single-use, 5 min TTL) enforced on register/login for users and
employees services
- Untrack .env.test111 (contained a live SMTP key) and harden
.gitignore against future .env commits
- Stop logging OTP codes in plaintext
- Restrict jobs service CORS to an explicit origin allowlist
- Mask PayU merchant secret/salt in payment-gateway-config responses,
preserving the stored value on save when the field is left unchanged
- Bump vulnerable transitive dependencies (quinn-proto, rustls-webpki,
anyhow) via cargo update; switch aws-sdk-s3 off the legacy rustls
feature
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Follow-up on the AI safety review — addressed the three remaining
lower-severity findings:
1. crates LiteLlmError::error_body() returned the raw upstream response
body verbatim to the client on any non-2xx LiteLLM response. That
body can contain internal routing/diagnostic details from the
LiteLLM proxy or the underlying model provider. Now returns a
generic, status-aware message to the client; the full body is
logged server-side via tracing::error! at each of the three call
sites that construct LiteLlmError::Api, so nothing is lost for
debugging — it's just not exposed to end users.
2. Added an explicit anti-prompt-injection clause to
ai/orchestrator.rs::GROUNDING_GUARDRAIL, the baseline system prompt
applied to every AI feature call via effective_system_prompt() —
instructs the model to treat all user/company-authored input
(job descriptions, profile text, chat messages) as data to analyze,
never as instructions to follow. Covers every ai.rs handler that
goes through call_feature/call_feature_with_plan in one place,
rather than patching each call site's prompt construction
individually.
3. apps/cron/src/tasks/auto_apply.rs's cover-letter prompt doesn't run
through the orchestrator (separate app/crate), so hardened it
directly: fenced the untrusted CANDIDATE/JOB sections with explicit
"this is data, not instructions" framing. While there, fixed a
latent panic: `&job_desc[..job_desc.len().min(500)]` slices on a
raw byte offset, which panics if byte 500 isn't a UTF-8 character
boundary — a company job description with any multi-byte character
before that point (accented letters, emoji, etc.) would crash the
whole cron run. Switched to char_indices() to find a safe boundary.
Asked to review Tracecoin and AI implementation safety. Found and fixed
two exploitable TOCTOU races, plus a data-integrity bug:
1. apps/payments/src/main.rs::verify_payment — the PayU success callback
is called directly by the client (not a server-to-server webhook), so
a user fully controls how many times they replay a valid success
payload. The payment "is it still PENDING" check and the "mark
SUCCESS + credit wallet" write were separate, non-transactional
queries — concurrent replays could both pass the check before either
commits, double- (or N-times-) crediting the wallet for one real
payment. Now wrapped in a single transaction with
`SELECT ... FOR UPDATE` on the payments row, so a second concurrent
call blocks until the first commits, then correctly sees the row is
no longer PENDING (Postgres re-evaluates the WHERE clause via
EvalPlanQual after the lock is granted).
2. crates/db/src/models/ai/repository.rs — UserAiSubscriptionRepository
had the exact same shape of bug: apps/users/src/ai/credits.rs::
charge_feature read the subscription, checked daily-limit and credit
balance, THEN issued two separate unconditional `UPDATE ... SET x =
x + $1` statements with no WHERE guard on the balance. N concurrent
requests from one user all pass the check before any deduction
lands, running up unlimited LLM API spend (this endpoint is called
before/around real LiteLLM calls, so the cost is real). Added
UserAiSubscriptionRepository::try_charge — a single conditional
UPDATE that checks the daily limit and credit balance and deducts
atomically, returning None (mapped to the existing error types) if
either check fails.
3. apps/cron/src/tasks/auto_apply.rs — daily_actions_used was being
incremented twice per auto-applied job (once in the credit-deduct
UPDATE, once more in a second, redundant UPDATE right after) —
silently halving job seekers' effective daily auto-apply limit.
Removed the redundant second UPDATE.
Also added non-negative CHECK constraints directly to the live
database (tracecoin_wallets.balance/reserved,
user_ai_subscriptions.daily_actions_used/monthly_credits_used/
purchased_credits_used) as defense in depth — belt-and-suspenders in
case a future code path reintroduces a similar bug.
Found while investigating "notifications/emails not working on approve
or job posting":
1. Real bug: apps/companies/src/handlers/mod.rs::view_contact (company
viewing an applicant's contact info) inserted into notifications
using column name `notification_type`, which has never existed —
the column is `type`. This INSERT has been failing outright every
time a company views a contact.
2. Root cause for approvals specifically: verifications/approval_requests
never existed until earlier this session (see
20260718200000_create_verifications_and_approvals) — every admin
approve/reject action was failing at the DB layer before it ever
reached the notification/email code, so nothing in this area could
have worked regardless of the email/notification logic itself.
3. Observability gap: every `state.mail.send_*_email(...)` call site
silently discarded its Result (`let _ = ...`), so if the SMTP/
Zeptomail provider is unconfigured (crates/email::Mailer already
logs a clear warning at startup for that, but callers gave no
per-send signal) or a send fails for any other reason, there was no
way to see it happen. Added `tracing::error!` logging on failure
for every job/approval-related email: job submitted, job approved,
job rejected, requirement approved, profile approval
approved/rejected, requirement submitted. Doesn't change delivery —
if the environment has no EMAIL_PROVIDER/SMTP_*/ZEPTOMAIL_*
configured, sends still fail, but that failure is now visible in
logs instead of silent.
In-app notifications for approvals were already schema-correct
(job/profile/requirement approve+reject all insert into notifications
with the right columns) — the two real defects were #1 and #2 above.
Finished the sweep of every table referenced by live Rust code but not
created by any active migration:
- audit_logs / audit_log_changes — backs wallet::audit() (crates/wallet),
called from every admin Tracecoin balance adjustment
(apps/payments/src/admin.rs::adjust_credits). actor_type defaulted since
the only caller doesn't supply it.
- tax_rules — backs the admin tax-rule CRUD in apps/payments/src/admin.rs.
init-db.sql has a differently-named version (title/percentage/
applicable_to); schema here matches the live code's actual columns
(name/tax_rate/applies_to).
- reviews — backs the professional review/rating system
(apps/users/src/handlers/reviews.rs).
Also fixed a real bug found while building the reviews schema:
admin_create_review's customer-lookup subquery selected
`lead_requests.user_id`, a column that has never existed (it's
`customer_user_id`) — would have failed on the very first review
creation attempt now that lead_requests actually exists.
Checked every other previously-flagged table (permissions, orders,
external_roles, internal_roles, kb_*, notification_*, order_items,
portfolio_images, smtp_configs, dashboard_widgets,
verification_documents) against actual Rust usage — none of them are
referenced by a real SQL query (grep hits were all Rust variable/type
names), so nothing to fix there. Full migration-chain sweep confirms
zero remaining "references a table before it's created" issues.
customer_profiles already had custom_data but never got a `status`
column, so — same root cause already fixed for job_seeker_profiles —
the generic profile save/get/submit handlers failed outright for the
CUSTOMER role, and the admin final-approval path couldn't write a
verified status either.
- Migration: add customer_profiles.status (default 'DRAFT').
- Special-case CUSTOMER in the generic profile.rs handlers (get_profile,
save_profile, fetch_saved_profile, set_profile_status), mirroring the
existing JOB_SEEKER special-case, storing basic-tab fields under
custom_data.basic_info.
- Re-enable the CUSTOMER branch in activate_profile_after_final_approval
now that the status column exists.
- Add the missing POST /api/customers/profile/documents upload endpoint
— the frontend's document upload (required: Aadhar/Government ID)
targets this exact path for CUSTOMER and previously 404'd since no
such route was ever registered. Mirrors the B2-upload-only pattern
used by the profession apps' shared upload_document handler.
Verified separately: requirement posting (POST /api/customers/requirements)
and requirement submission-for-verification (POST
/api/customers/requirements/:id/submit) already work correctly — both
use the `leads` table (an active migration, despite the model's
"Requirement" naming) and properly create a verification record
(case_type REQUIREMENT_APPROVAL) that lands in admin Verification
Management via the existing approve_requirement/reject_requirement
handlers. No changes needed there.
- Company approval wrote profile status to 'ACTIVE' (company_profiles'
own pre-verification default) using an id column that never matched
any row, so create_job's APPROVED check always rejected newly
approved companies. Match on user_id for user_id-keyed tables and
write the canonical 'APPROVED' status.
- job_seeker_profiles was missing columns the job-seeker app has
always queried (full_name, location, summary, skills,
active_application_count, status), and the job_applications /
job_seeker_documents tables it depends on were never migrated in —
job seeker profile save/submit and job applications failed outright
with "column/relation does not exist".
- Renamed the job_seeker first_name/last_name split to full_name to
match what the frontend has always sent.
- Special-cased JOB_SEEKER in the generic profile.rs handlers (mirrors
the existing COMPANY special-case) so the shared ProfilePage save/
submit flow, which was routed through a user_role_profile_id-based
path job_seeker_profiles never had, now persists correctly.
- Fixed apply_to_job's company notification query joining a
nonexistent "companies" table instead of company_profiles.
- Fixed auto-apply cron's company status filter to match the
corrected 'APPROVED' status.
Adds a DB-trigger-generated reference_number (NXT-{TYPE}-{YY}-{000001}) to
verifications, support_tickets, payments, job_applications, lead_requests,
and users, replacing raw UUIDs shown to customers/admins. Also fixes
verification-status endpoint to return uploaded documents (previously
omitted, so documents never appeared after submission), and adds a
reference-number lookup endpoint for the AI support assistant.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The askash-main model at llm.nxtgauge.com takes 60-120s to generate
job descriptions; previous 60s timeout caused all JD generations to fail.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- companies/admin: add proper POST /jobs/{id}/approve and /jobs/{id}/reject
endpoints (sets status=LIVE, not direct SQL bypass)
- customers: fix list_requests to query by customer_user_id (not professional),
add optional lead_id filter; fix debit to use professional_user_id
- payments: switch razorpay_order_id column to payu_txnid (PayU migration)
- users/auth: fix role resolution to not inject phantom roles for professionals
- contracts/profession_shared: fix my_requests SQL to join leads+users instead
of nonexistent requirements table
- db/job_seeker: fix INSERT VALUES placeholder count (add missing $10)
- storage: add MOCK_STORAGE=true mode for local dev without real B2 creds
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
register() now generates a per-account LiteLLM key (best-effort, non-blocking)
and stores it on the user. New internal endpoint GET /internal/users/{id}/llm-key
lets other services fetch (or lazily backfill) an account's key, authenticated
via the existing X-AI-Service-Key shared secret.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both still on stale bad digests after the previous retrigger. Old replicas
continue serving traffic throughout, so no outage.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Restarting these deployments to pick up the corrected B2 secret revealed
they'd been running healthy old pods on top of a bad digest in the
deployment spec all along (silent from an earlier rebuild wave) — the
restart exposed it via ImagePullBackOff/ErrImagePull.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The profession-specific tables (photographer_profiles, tutor_profiles, etc.)
were extended with a separate user_role_profile_id FK column and a NOT NULL
user_id column, but every generic profile query still treated the shared
user_role_profiles.id value as if it were the profession table's own `id`
PK, and never supplied user_id at all. Result: PATCH /api/profile 500'd
with "null value in column user_id violates not-null constraint" on every
first-time save for any of the 10 professional roles — profile saves were
completely broken, which cascades into submit-for-verification always
running off an empty fallback payload with no real data or documents.
Fixes get_profile, save_profile, set_profile_status, and
fetch_saved_profile_by_urp_id to query/write user_role_profile_id (and
populate user_id on insert) instead of assuming id.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GET /api/admin/verifications returned a bare JSON array, but every
consumer (Verification Management, Approval Management, and the e2e
test's own mock) expects {items: [...]}. Array.isArray(payload?.items)
was always false against a bare array, so both admin screens have been
showing "No verification requests found" regardless of what's actually
in the queue.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
extract_documents() recognized a stale set of document keys that no longer
matched what the frontend actually uploads (portfolio_ownership_proof,
professional_certifications, qualification_proof, tax_document), so every
non-COMPANY role's verification case was created with an empty documents
array. Also extracts role_key_to_display/role_to_table into a shared
role_meta module — verifications.rs and approvals.rs were each missing
UGC_CONTENT_CREATOR from their inline copies, so that role's rejections
and final approvals were silently no-ops.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Roles assigned at registration are immediately active for login + session.
Document/profile approval (handled separately by user_role_profiles in
onboarding/verifications) is the correct gate for review workflows —
do not gate raw role assignment on that flow.
Previously defaulted to PENDING for non-demo accounts, which get filtered
out by get_user_role_keys (WHERE status = 'APPROVED') and produced JWTs
with empty roles, causing the 'role is not assigned' UX bug after login.
- Add #![allow(dead_code)] pragma to all main.rs files
- Remove unused imports from users handlers (ai_cache, AiCreditPackageRepository, AiCreditTransactionRepository)
- Make LiteLLMChatMessage and LiteLLMChoice public with public fields
- Fix remaining unused variables with cargo fix
- Add missing pub visibility modifiers to litellm structs
All packages now compile with ZERO warnings and errors!
- Fix match arm type errors in ai.rs: wrap bare String returns in (String, bool)
tuples to match expected return type (response_text, _ollama_used)
- Add Deserialize trait to LiteLLMChatMessage for deserialization
- Add missing fields to GenerateFieldResponse constructors
- Remove body.user_id reference from form extraction (field doesn't exist)
- Add get_llm_base_url() and get_llm_model() helper functions
users package now compiles successfully (only warnings remain).
- Restore deleted module declarations (tutor, ugc_content_creator, user,
user_role_profile, verification, video_editor) in db/models/mod.rs
- Add mod ai; to users/src/main.rs (fixes crate::ai import)
- Add pub mod ai_auto; to handlers/mod.rs
- Add rust_decimal dependency to payments/Cargo.toml
- Fix missing PgPool import in payments/src/ai_credits.rs
- Make LiteLLMUsage fields public in users/src/litellm.rs
- Add get_llm_base_url() and get_llm_model() helper functions
- Remove leading underscore from variables that are used
Partial fix for high-performance branch build issues.
Delete legacy code that used old company_ai_usage/job_seeker_ai_usage tables:
- Remove has_active_ai_pack() - old AI_PACK pricing package check
- Remove check_and_increment_usage() - legacy daily quota tracking
- Remove BASE_AI_LIMIT, get_ai_limit_for_package constants/functions
- Remove legacy queries from ai_auto_apply() and ai_usage_status()
- Update auto_apply.rs to use user_ai_subscriptions.daily_actions_used
instead of job_seeker_ai_usage table
- Inline apply_scheduled_downgrades() and expire_trials() in cron tasks
to remove dependency on users crate internal modules
The new system uses user_ai_subscriptions with:
- daily_actions_used / daily_credits_used counters
- monthly_credits_total / monthly_credits_used
- purchased_credits_total / purchased_credits_used
All AI billing now flows through the wallet/ledger system with
LiteLLM integration (Tasks 1-10).
Critical: ai_access_middleware was wired via from_fn_with_state((), ...)
- passing the unit type as state - and pulled AppState from request
extensions, which nothing ever populated. Every request through
/api/ai/* and /api/ai/auto/* returned 500 INCOMPLETE_CONTEXT. Fixed by
extracting State<AppState> properly and passing the real state at both
call sites; removed the redundant, identically-broken inner middleware
layer inside ai_router().
Security: ai_addon_purchase (/api/ai/addons/purchase, /api/ai/credits/buy)
and ai_plan_upgrade (/api/ai/plans/upgrade) granted AI credits / plan
upgrades (including enterprise) with zero payment verification - any
authenticated user could mint unlimited free credits, and the frontend
already called this directly. Disabled both until wired to a real
payment flow.
Quality: added a grounding/anti-hallucination system prompt applied to
every AI feature call (orchestrator::call_feature /
call_feature_with_plan, plus the handful of call sites that bypass the
orchestrator). Verified against the live model that it reduces but does
not eliminate fabrication on harder reasoning tasks - even the larger
model invents facts not present in the input on some prompts. This is a
real limitation of the two locally-hosted models, not something a
system prompt alone fully solves; flagged for follow-up (e.g. a
verification pass or deterministic checks for high-stakes decisions
like auto-apply).
Also fixed Persona/Pillar keyword detection using naive substring
matching (e.g. "team" matching inside "esteemed", "lead" matching
inside "leadership") - added a word-boundary-aware contains_word()
helper and applied it to all keyword classifiers in this file.
The frontend (lib/payu.ts, payu-return.tsx) and admin config UI had
already switched to PayU, but the payments service was still calling
Razorpay's order API and verifying Razorpay HMAC signatures -
payments were broken end to end. Rewrites apps/payments to PayU's
hash-based hosted-checkout flow (SHA-512 request/response hash,
key+salt from admin config or PAYU_MERCHANT_KEY/PAYU_SALT), for both
the tracecoin wallet purchase flow and the AI credits flow. The AI
credits frontend checkout was also fabricating a fake payment_id and
signature client-side instead of ever opening a real PayU checkout -
fixed to use the same real flow as tracecoin purchases.
Also fixes the ai_create_ticket endpoint on the users service, which
never validated the X-AI-Service-Key header despite the client
sending one - anyone could create tickets under an arbitrary user_id.
- Add AI plans, credits, model routing, LiteLLM client, and orchestrator services
- Add AI management endpoints, auto-apply/auto-request handlers, and log endpoints
- Add cron jobs for daily action reset and monthly credit reset
- Add AI credit purchase flow in payments service
- Add ai_credit_packages migration with seed data
- Update Dockerfile build tooling across services
User-facing AI endpoints:
- GET /api/ai/usage/v2 - extended with addon_balance, renewal_date
- POST /api/ai/addons/purchase - purchase addon packs
- POST /api/ai/plans/upgrade - upgrade AI plans
Admin AI endpoints:
- GET /api/admin/ai/stats - AI usage statistics
- GET /api/admin/ai/users - paginated user AI usage list
- GET /api/admin/ai/plans - list AI plans
Files:
- apps/users/src/handlers/admin_ai.rs (new)
- Add get_llm_base_url() and get_llm_model() helper functions
- Update call_ollama_inline to route to LiteLLM when LLM_PROVIDER=litellm
- Add auto-apply cron task for background job matching
- Auto-apply matches job seekers to new jobs based on skills
- Generate cover letters via LiteLLM for each application
- Auto-verifies emails for accounts ending with @demo.com
- Auto-approves COMPANY role for demo accounts
- Skips email verification and OTP for demo accounts
- Auto-approves profile verification for demo accounts
- Allows login without email verification for demo accounts
This enables payment gateway companies to login directly and view packages.
DB:
- Add niche_tags column to ugc_content_creator_profiles (was blocking UGC service)
- Add turnaround_days and fix user_role_profile_id NOT NULL for UGC
- leads/lead_requests tables (already created in session 1)
Code:
- Add UGC_CONTENT_CREATOR to is_professional_role() to auto-create user_role_profiles
- Fix onboarding INSERT to include user_id for photographer_profiles
- Fix send_lead_request_ai to use correct customer_user_id (was self-notifying)
- Add PATCH /api/leads/:id support + mount leads at /api/* for gateway compatibility
- Fix admin_list_cases query (WHERE was using wrong params)
- Fix admin_get_case query (was using list query instead of fetch-by-id)
- Add GET /api/me in profile.rs (moved from onboarding)
- Add KB articles by ID route /api/kb/articles/id/{id}
- Rewrite reviews handlers to match actual reviews table schema
- Add public reviews router GET /api/reviews
Gateway:
- Add /api/reviews route to users service
- payments/src/main.rs: fail-fast on BEECEPTOR_URL and DATABASE_URL
- gateway/src/main.rs: fail-fast on all service URLs and CORS URLs
- users/src/handlers/ai.rs: fail-fast on LEADS_SERVICE_URL
- leads/src/main.rs: fail-fast on OLLAMA_BASE_URL and OLLAMA_CHAT_MODEL
- storage/Cargo.toml: replace rustls-aws-lc with rustls for aws-config/aws-sdk-s3
- Add AI credit management endpoints for companies
- Add AI usage history tracking
- Add AI content generation with Ollama integration
- Add Ollama client for generating job descriptions, resume analysis, and cover letters
- Integrate AI router into companies service
- Add missing AI intents: generate_cover_letter, improve_resume, request_view_contact, auto_apply_job, unknown
- Add is_internal_admin helper to prevent admin/super_admin users from using user-facing AI flows
- Add admin guards to: ai_generate_job_field, ai_generate_cover_letter, ai_tailor_resume, ai_auto_apply, ai_auto_respond_to_lead
- Add professional approval check in ai_auto_respond_to_lead - must be APPROVED status
- Add tracecoin balance check before contact reveal (requires 30 tracecoins)
- Add KB escalation: when no articles found, suggest creating support ticket
- Add explicit unknown intent handler with helpful message
