feat: add LiteLLM support and auto-apply cron job

- Add get_llm_base_url() and get_llm_model() helper functions
- Update call_ollama_inline to route to LiteLLM when LLM_PROVIDER=litellm
- Add auto-apply cron task for background job matching
- Auto-apply matches job seekers to new jobs based on skills
- Generate cover letters via LiteLLM for each application
This commit is contained in:
Tracewebstudio Dev 2026-06-14 18:00:38 +02:00
parent ac3fef72a1
commit a4af5c2c06
7 changed files with 780 additions and 40 deletions

View file

@ -0,0 +1,328 @@
version: 1
plan:
name: nxtgauge-full-e2e-test
max_concurrency: 6
max_consecutive_failures: 3
max_cycles: 10
auto_reject_retries: 1
tasks:
- id: track-1-professions
title: Test all 10 profession services
prompt: |
You are testing the Nxtgauge backend API. All tests are via direct HTTP API calls.
BASE_URL: http://localhost:9100
GATEWAY is at :9100 (routes to all services)
Individual service ports:
photographers: 9107, makeup_artists: 9109, tutors: 9108, developers: 9110,
video_editors: 9111, graphic_designers: 9112, social_media_managers: 9113,
fitness_trainers: 9114, catering_services: 9115, ugc_content_creators: 9117
Test accounts (use test_mode=true to bypass OTP):
Professional: bug2_test_1781086381@test.com / Test1234! (PHOTOGRAPHER role active)
Customer: cu_1781041378@test.com / Test1234! (CUSTOMER role active)
Open lead ID: ba18c522-b3cd-43a7-a269-90cf981c173e (PHOTOGRAPHER, status OPEN)
FOR EACH of the 10 profession services below, test:
1. Health check: GET http://localhost:<PORT>/health or via gateway
2. Login to get a fresh token (use test_mode=true register, then login)
3. GET /api/<profession>/profile/me — does profile endpoint work?
4. GET /api/<profession>/marketplace — does marketplace browse work?
5. Send a lead request via POST /api/<profession>/leads/request with body {"requirement_id":"ba18c522-b3cd-43a7-a269-90cf981c173e","message":"interested"}
Professions to test:
- photographers (9107)
- makeup_artists (9109) — route is /api/makeup-artists
- tutors (9108)
- developers (9110) — route is /api/developers
- video_editors (9111) — route is /api/video-editors
- graphic_designers (9112) — route is /api/graphic-designers
- social_media_managers (9113) — route is /api/social-media-managers
- fitness_trainers (9114) — route is /api/fitness-trainers
- catering_services (9115) — route is /api/catering-services
- ugc_content_creators (9117) — route is /api/ugc-content-creators
NOTE: The "professional" user (bug2_test_1781086381@test.com) only has PHOTOGRAPHER role. For other professions, register a fresh user, register the role, switch to that role, then test.
Write results to /Users/ashwin/workspace/e2e-results/track-1-professions.md with:
- For each profession: service port, health status, profile result, marketplace result, lead_request result, any errors
- Summary of which services are fully functional vs partially working vs broken
assigned_to: general
verify_skip_reason: "Downstream compilation task reads all result files directly; no independent verification needed for raw API test output."
max_retries: 1
- id: track-2-leads
title: Test leads service and lead request accept/reject flow
prompt: |
You are testing the Nxtgauge backend API. All tests are via direct HTTP API calls.
Gateway: http://localhost:9100
Leads service: http://localhost:9118 (or via gateway /api/leads)
Customers service: http://localhost:9105
Test accounts (use test_mode=true):
Customer: cu_1781041378@test.com / Test1234!
Photographer: bug2_test_1781086381@test.com / Test1234! (has PHOTOGRAPHER approved profile)
Test the following flows:
1. LEADS SERVICE CRUD:
- GET /api/leads — list all leads (or http://localhost:9118/leads)
- POST /api/leads — create a new lead {"title":"Test Lead 2","description":"API test","location":"Delhi","profession_key":"PHOTOGRAPHER"}
- GET /api/leads/:id — get specific lead
- PATCH /api/leads/:id — update the lead title
2. CUSTOMER VIEW REQUESTS (the /requests endpoint we fixed):
- Login as customer, get token
- GET /api/customers/requests — should return HTTP 200 with {"data":[]}
- POST a lead request as the photographer, then check GET /api/customers/requests again
3. CUSTOMER ACCEPT/REJECT LEAD REQUEST:
- First, check what the approve/reject endpoints look like
- Try POST /api/customers/requests/:lead_request_id/approve
- Try POST /api/customers/requests/:lead_request_id/reject
- Note: these may fail with 404 or 500 — document what happens
4. PROFESSIONAL LEAD REQUEST FLOW:
- Login as photographer, get fresh token
- GET /api/photographers/marketplace — should show OPEN leads
- POST /api/photographers/leads/request — send a new lead request
- GET /api/photographers/leads/requests/me — list my sent requests
- DELETE /api/photographers/leads/requests/:id — cancel a pending request
Write results to /Users/ashwin/workspace/e2e-results/track-2-leads.md with:
- Each endpoint tested, HTTP status, response body
- Any errors, 4xx/5xx codes, and what they mean
- Overall status of leads + lead request flow
assigned_to: general
verify_skip_reason: "Downstream compilation task reads all result files directly; no independent verification needed for raw API test output."
max_retries: 1
- id: track-3-payments
title: Test payments and tracecoin flows
prompt: |
You are testing the Nxtgauge backend API. All tests are via direct HTTP API calls.
Gateway: http://localhost:9100
Payments service: http://localhost:9116
Test accounts: bug2_test_1781086381@test.com / Test1234!
DATABASE: postgresql://nxtgauge:nxtgauge_dev@localhost:5432/nxtgauge_db
Test these flows:
1. TRACEOCOIN WALLET:
- GET /api/packages — list all pricing packages (via gateway or users service)
- GET /api/photographers/wallet/me — check photographer wallet (may fail with 404 if not set up)
- Check DB: psql "postgresql://nxtgauge:nxtgauge_dev@localhost:5432/nxtgauge_db" -c "SELECT * FROM tracecoin_wallets LIMIT 5;"
2. TRACEOCOIN PURCHASE (if payments service is running on 9116):
- GET http://localhost:9116/health — is payments service up?
- GET http://localhost:9116/api/packages — list packages
- Try to initiate a tracecoin purchase (check the payments service routes)
3. PAYMENTS SERVICE ROUTES:
- Document what routes the payments service exposes
- Try GET /api/payments/health
- Try GET /api/payments/packages
- Check what the Beeceptor mock URL is (if used)
4. TRACEOCOIN LEDGER:
- Check DB: psql "postgresql://nxtgauge:nxtgauge_dev@localhost:5432/nxtgauge_db" -c "SELECT * FROM tracecoin_ledger LIMIT 10;"
- Check the schema of tracecoin_ledger table
5. INVOICES:
- Check DB: psql ... -c "SELECT * FROM invoices LIMIT 5;"
- Check invoices table schema
Write results to /Users/ashwin/workspace/e2e-results/track-3-payments.md with:
- Payments service status (running or not)
- All endpoints tested with HTTP status and response
- Tracecoin wallet balances found in DB
- Any payment flow errors
assigned_to: general
verify_skip_reason: "Downstream compilation task reads all result files directly; no independent verification needed for raw API test output."
max_retries: 1
- id: track-4-admin
title: Test admin approval and verification workflow
prompt: |
You are testing the Nxtgauge backend API. All tests are via direct HTTP API calls.
Gateway: http://localhost:9100
Test admin account: cu_1781041378@test.com / Test1234! (or use admin credentials if available)
NOTE: The admin dashboard is a separate Next.js app. Test the backend API endpoints only.
Test these flows:
1. ADMIN VERIFICATIONS ENDPOINT:
- GET /api/admin/verifications — list verification cases (requires admin auth)
- GET /api/admin/verifications?status=PENDING — filter by status
- What does this return? 200? 403? 401?
2. ADMIN APPROVALS ENDPOINT:
- GET /api/admin/approvals — list approval cases
- What does it return?
3. APPROVE A JOB (job posting approval):
- First create a job as a company (use co_1781041378@test.com / Test1234! if available, or register new)
- Submit the job (status goes to PENDING_APPROVAL)
- Try to approve it via admin endpoint: POST /api/admin/jobs/:job_id/approve or similar
- Document what admin job approval endpoints exist
4. APPROVE A REQUIREMENT (customer lead approval):
- Create a customer requirement
- Submit it (status goes to PENDING_APPROVAL)
- Try to approve via admin endpoint
5. ADMIN APPROVAL FLOW:
- Check what routes exist under /api/admin/ for approvals
- GET /api/admin/approval-cases if it exists
- Try to find and call the actual approval endpoints
6. VERIFICATION MANAGEMENT:
- GET /api/admin/verifications — list all verifications
- Try to approve/reject a verification
Write results to /Users/ashwin/workspace/e2e-results/track-4-admin.md with:
- Each admin endpoint tested, HTTP status, response
- Whether verification/approval workflow is functional end-to-end
- Missing or broken admin endpoints
assigned_to: general
verify_skip_reason: "Downstream compilation task reads all result files directly; no independent verification needed for raw API test output."
max_retries: 1
- id: track-5-support
title: Test support tickets and email notifications
prompt: |
You are testing the Nxtgauge backend API. All tests are via direct HTTP API calls.
Gateway: http://localhost:9100
Test accounts: bug2_test_1781086381@test.com / Test1234! (or register fresh users with test_mode=true)
Test these flows:
1. SUPPORT TICKETS (user-facing):
- POST /api/support/tickets — create a support ticket
Body: {"subject":"Test ticket from API","message":"This is a test","category":"GENERAL"}
- GET /api/support/tickets — list my tickets
- GET /api/support/tickets/:id — get specific ticket
- POST /api/support/tickets/:id/messages — add a message to a ticket
2. SUPPORT TICKETS (admin):
- GET /api/admin/support-cases — list all support cases (admin only)
- Try to reply to a ticket as admin
3. EMAIL NOTIFICATIONS (check if emails are being sent):
- Create a ticket → check DB: psql "postgresql://nxtgauge:nxtgauge_dev@localhost:5432/nxtgauge_db" -c "SELECT * FROM email_logs ORDER BY created_at DESC LIMIT 10;"
- Submit a job → check email_logs
- Submit a requirement → check email_logs
- Submit a job application → check email_logs
4. NOTIFICATIONS:
- GET /api/me/notifications — list user notifications
- Check notification table: psql "postgresql://nxtgauge:nxtgauge_dev@localhost:5432/nxtgauge_db" -c "SELECT * FROM notifications ORDER BY created_at DESC LIMIT 20;"
5. KNOWLEDGE BASE:
- GET /api/kb/articles — list KB articles
- GET /api/kb/articles/:id — get specific article
- GET /api/kb/categories — list categories
Write results to /Users/ashwin/workspace/e2e-results/track-5-support.md with:
- Support ticket creation status (success/fail/error)
- Email logs found in DB (were emails sent?)
- Notifications generated
- KB articles accessible
- Any broken endpoints
assigned_to: general
verify_skip_reason: "Downstream compilation task reads all result files directly; no independent verification needed for raw API test output."
max_retries: 1
- id: track-6-reviews-portfolio
title: Test reviews, portfolio, and onboarding flows
prompt: |
You are testing the Nxtgauge backend API. All tests are via direct HTTP API calls.
Gateway: http://localhost:9100
Test accounts: register fresh users with test_mode=true as needed
DATABASE: postgresql://nxtgauge:nxtgauge_dev@localhost:5432/nxtgauge_db
Test these flows:
1. REVIEWS:
- POST /api/reviews — submit a review
Body: {"entity_type":"COMPANY","entity_id":"<company_uuid>","rating":5,"comment":"Great service!"}
(You'll need to find a valid company_id from the DB: psql ... -c "SELECT id FROM company_profiles LIMIT 3;")
- GET /api/reviews — list reviews
- GET /api/reviews?entity_type=COMPANY&entity_id=<id> — reviews for a company
- Admin: GET /api/admin/reviews — admin review list
2. PORTFOLIO (photographer):
- Login as photographer (bug2_test_1781086381@test.com / Test1234!)
- POST /api/photographers/portfolio/me — create portfolio item
Body: {"title":"Test Portfolio","description":"Test description","image_url":"https://example.com/test.jpg"}
- GET /api/photographers/portfolio/me — list portfolio items
- PATCH /api/photographers/portfolio/me/:id — update portfolio item
- DELETE /api/photographers/portfolio/me/:id — delete portfolio item
3. SERVICES (photographer):
- POST /api/photographers/services/me — create a service
Body: {"name":"Wedding Photography Package","description":"Full day coverage","price_inr":50000}
- GET /api/photographers/services/me — list services
- PATCH /api/photographers/services/me/:id — update service
- DELETE /api/photographers/services/me/:id — delete service
4. ONBOARDING STATE:
- GET /api/onboarding/state — get onboarding state for a user
- POST /api/onboarding/submit — submit onboarding data
- GET /api/me — check if role and profile status is shown
5. RUNTIME CONFIG:
- GET /api/runtime-config — get runtime config for a role
- GET /api/runtime-config?role=PHOTOGRAPHER — get specific role config
Write results to /Users/ashwin/workspace/e2e-results/track-6-reviews-portfolio.md with:
- Each endpoint tested, HTTP status, response
- DB schema for reviews and portfolio tables
- Any errors found
assigned_to: general
verify_skip_reason: "Downstream compilation task reads all result files directly; no independent verification needed for raw API test output."
max_retries: 1
- id: track-7-compile-report
title: Compile comprehensive test report
role: verify-as-task
prompt: |
Read all the test result files created by the other tracks and compile a comprehensive report.
Read these files:
- /Users/ashwin/workspace/e2e-results/track-1-professions.md
- /Users/ashwin/workspace/e2e-results/track-2-leads.md
- /Users/ashwin/workspace/e2e-results/track-3-payments.md
- /Users/ashwin/workspace/e2e-results/track-4-admin.md
- /Users/ashwin/workspace/e2e-results/track-5-support.md
- /Users/ashwin/workspace/e2e-results/track-6-reviews-portfolio.md
- Also read: /Users/ashwin/workspace/E2E_TEST_REPORT.md (the previous report from Session 1)
Compile everything into a single comprehensive report at:
/Users/ashwin/workspace/e2e-results/COMPLETE_E2E_REPORT.md
The report should include:
1. Executive summary: overall system health (X/Y endpoints working)
2. Track-by-track results with PASS/FAIL for each endpoint tested
3. All bugs found this round (new issues not in the previous report)
4. All bugs from the previous report and their current status (fixed/still broken)
5. Recommendations and priority list of what to fix next
6. Test data created during this round
Format: Markdown report. Be thorough — this is the canonical test record.
assigned_to: general
depends_on:
- track-1-professions
- track-2-leads
- track-3-payments
- track-4-admin
- track-5-support
- track-6-reviews-portfolio
max_retries: 1

1
Cargo.lock generated
View file

@ -941,6 +941,7 @@ dependencies = [
"chrono",
"db",
"email",
"reqwest",
"serde",
"serde_json",
"sqlx",

View file

@ -18,3 +18,4 @@ serde.workspace = true
serde_json.workspace = true
chrono.workspace = true
uuid.workspace = true
reqwest.workspace = true

View file

@ -75,6 +75,19 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
}
});
// Spawn Hourly Auto-Apply task for job seekers
let p_auto_apply_sys = pool.clone();
tokio::spawn(async move {
let mut interval = time::interval(Duration::from_secs(60 * 60));
loop {
interval.tick().await;
tracing::info!("Running Auto-Apply Task...");
if let Err(e) = tasks::auto_apply::run_auto_apply(&p_auto_apply_sys).await {
tracing::error!("Auto-Apply Task Failed: {}", e);
}
}
});
// Keep main thread alive
tokio::signal::ctrl_c().await?;
tracing::info!("Shutting down cron engine.");

View file

@ -0,0 +1,318 @@
use chrono::{Duration, Utc};
use reqwest::Client;
use serde::{Deserialize, Serialize};
use sqlx::PgPool;
use uuid::Uuid;
#[derive(Debug, Deserialize)]
struct AutoApplyConfig {
litellm_base_url: String,
litellm_api_key: String,
litellm_model: String,
max_applications_per_run: usize,
max_daily_applications: usize,
}
impl AutoApplyConfig {
fn from_env() -> Self {
Self {
litellm_base_url: std::env::var("LITELLM_BASE_URL")
.unwrap_or_else(|_| "https://llm.nxtgauge.com/v1".to_string()),
litellm_api_key: std::env::var("LITELLM_API_KEY").unwrap_or_default(),
litellm_model: std::env::var("LITELLM_MODEL")
.unwrap_or_else(|_| "askash-main".to_string()),
max_applications_per_run: std::env::var("AUTO_APPLY_MAX_PER_RUN")
.unwrap_or_else(|_| "5".to_string())
.parse()
.unwrap_or(5),
max_daily_applications: std::env::var("AUTO_APPLY_MAX_DAILY")
.unwrap_or_else(|_| "20".to_string())
.parse()
.unwrap_or(20),
}
}
}
#[derive(Debug, Serialize)]
struct ChatMessage {
role: String,
content: String,
}
#[derive(Debug, Serialize)]
struct ChatCompletionRequest {
model: String,
messages: Vec<ChatMessage>,
temperature: f32,
max_tokens: i32,
}
#[derive(Debug, Deserialize)]
struct ChatCompletionResponse {
choices: Vec<Choice>,
}
#[derive(Debug, Deserialize)]
struct Choice {
message: Message,
}
#[derive(Debug, Deserialize)]
struct Message {
content: String,
}
async fn generate_cover_letter(
client: &Client,
config: &AutoApplyConfig,
seeker_name: &str,
experience: i32,
skills: &[String],
summary: Option<&str>,
job_title: &str,
job_desc: &str,
) -> Result<String, Box<dyn std::error::Error + Send + Sync>> {
let url = format!("{}/chat/completions", config.litellm_base_url.trim_end_matches('/'));
let prompt = format!(
"Write a brief, professional cover letter (max 200 words).\n\n\
IMPORTANT: Do NOT include phone number, email, or any contact information.\n\
Only use the information provided below.\n\n\
CANDIDATE: Name: {}, Experience: {} years, Skills: {}, Summary: {}\n\
JOB: Title: {}, Description: {}\n\n\
Cover Letter:",
seeker_name,
experience,
skills.join(", "),
summary.unwrap_or(""),
job_title,
job_desc
);
let payload = ChatCompletionRequest {
model: config.litellm_model.clone(),
messages: vec![
ChatMessage {
role: "system".to_string(),
content: "You are a professional cover letter writer.".to_string(),
},
ChatMessage {
role: "user".to_string(),
content: prompt,
},
],
temperature: 0.2,
max_tokens: 500,
};
let res = client
.post(&url)
.header("Authorization", format!("Bearer {}", config.litellm_api_key))
.header("Content-Type", "application/json")
.json(&payload)
.send()
.await?;
if !res.status().is_success() {
return Ok("I am excited to apply for this position.".to_string());
}
let body: ChatCompletionResponse = res.json().await?;
if let Some(choice) = body.choices.first() {
Ok(choice.message.content.trim().to_string())
} else {
Ok("I am excited to apply for this position.".to_string())
}
}
#[derive(Debug, sqlx::FromRow)]
struct JobSeekerWithAi {
user_id: Uuid,
profile_id: Uuid,
full_name: String,
skills: Vec<String>,
experience_years: i32,
summary: Option<String>,
daily_limit: i32,
used_today: i32,
}
#[derive(Debug, sqlx::FromRow)]
struct NewJob {
id: Uuid,
title: String,
description: String,
skills: Vec<String>,
}
pub async fn run_auto_apply(pool: &PgPool) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let config = AutoApplyConfig::from_env();
if config.litellm_api_key.is_empty() {
tracing::warn!("Auto-apply skipped: LITELLM_API_KEY not configured");
return Ok(());
}
tracing::info!("Starting auto-apply job...");
let client = Client::new();
let cutoff_time = Utc::now() - Duration::hours(24);
let seekers: Vec<JobSeekerWithAi> = sqlx::query_as(
r#"
SELECT
u.id as user_id,
jsp.id as profile_id,
COALESCE(CONCAT(jsp.first_name, ' ', jsp.last_name), 'Candidate') as full_name,
COALESCE(jsp.skills, ARRAY[]::text[]) as skills,
COALESCE(jsp.experience_years, 0) as experience_years,
jsp.summary,
COALESCE((
SELECT p.monthly_action_limit / 30
FROM ai_entitlements e
JOIN ai_plans p ON p.id = e.plan_id
WHERE e.user_id = u.id AND e.role = 'JOB_SEEKER' AND e.status = 'active'
ORDER BY e.valid_from DESC
LIMIT 1
), 10) as daily_limit,
COALESCE((
SELECT generations_used
FROM job_seeker_ai_usage jau
WHERE jau.job_seeker_id = jsp.id
AND usage_date = CURRENT_DATE
), 0) as used_today
FROM users u
INNER JOIN job_seeker_profiles jsp ON jsp.user_id = u.id
WHERE u.status = 'ACTIVE'
AND jsp.status IN ('APPROVED', 'PENDING_REVIEW')
AND jsp.skills IS NOT NULL
AND array_length(jsp.skills, 1) > 0
AND u.email_verified = true
"#
)
.fetch_all(pool)
.await?;
if seekers.is_empty() {
tracing::info!("No job seekers with AI packs found for auto-apply");
return Ok(());
}
tracing::info!("Found {} job seekers eligible for auto-apply", seekers.len());
let mut total_applications = 0;
for seeker in seekers {
let remaining_today = seeker.daily_limit - seeker.used_today;
if remaining_today <= 0 {
tracing::debug!("User {} has no remaining AI applications today", seeker.user_id);
continue;
}
let new_jobs: Vec<NewJob> = sqlx::query_as(
r#"
SELECT j.id, j.title, j.description, COALESCE(j.skills, ARRAY[]::text[]) as skills
FROM jobs j
INNER JOIN companies c ON c.id = j.company_id
WHERE j.status = 'LIVE'
AND j.created_at > $1
AND c.status = 'APPROVED'
AND j.id NOT IN (
SELECT job_id FROM job_applications WHERE applicant_user_id = $2
)
AND j.skills && $3::text[]
ORDER BY j.created_at DESC
LIMIT $4
"#
)
.bind(cutoff_time)
.bind(seeker.user_id)
.bind(&seeker.skills)
.bind(std::cmp::min(remaining_today as i32, config.max_applications_per_run as i32))
.fetch_all(pool)
.await?;
if new_jobs.is_empty() {
continue;
}
tracing::info!(
"User {} has {} matching new jobs, applying to {}",
seeker.user_id,
new_jobs.len(),
new_jobs.len().min(remaining_today as usize)
);
for job in new_jobs.into_iter().take(remaining_today as usize) {
let cover_letter = match generate_cover_letter(
&client,
&config,
&seeker.full_name,
seeker.experience_years,
&seeker.skills,
seeker.summary.as_deref(),
&job.title,
&job.description,
)
.await
{
Ok(cl) => cl,
Err(e) => {
tracing::warn!("Failed to generate cover letter: {}", e);
"I am excited to apply for this position.".to_string()
}
};
let result = sqlx::query(
r#"
INSERT INTO job_applications (job_id, applicant_user_id, cover_letter, applied_via_ai)
VALUES ($1, $2, $3, true)
ON CONFLICT (job_id, applicant_user_id) DO NOTHING
"#
)
.bind(job.id)
.bind(seeker.user_id)
.bind(&cover_letter)
.execute(pool)
.await;
match result {
Ok(r) => {
if r.rows_affected() > 0 {
total_applications += 1;
sqlx::query(
r#"
INSERT INTO job_seeker_ai_usage (job_seeker_id, usage_date, generations_used)
VALUES ($1, CURRENT_DATE, 1)
ON CONFLICT (job_seeker_id, usage_date)
DO UPDATE SET generations_used = job_seeker_ai_usage.generations_used + 1
"#
)
.bind(seeker.profile_id)
.execute(pool)
.await
.ok();
sqlx::query(
"UPDATE job_seeker_profiles SET active_application_count = active_application_count + 1 WHERE id = $1"
)
.bind(seeker.profile_id)
.execute(pool)
.await
.ok();
tracing::info!("Auto-applied user {} to job {}", seeker.user_id, job.id);
}
}
Err(e) => {
tracing::error!("Failed to insert application: {}", e);
}
}
}
}
tracing::info!("Auto-apply completed. Total applications: {}", total_applications);
Ok(())
}

View file

@ -2,3 +2,4 @@ pub mod leads;
pub mod requirements;
pub mod jobs;
pub mod reminders;
pub mod auto_apply;

View file

@ -51,32 +51,7 @@ struct OllamaGenerateResponse {
async fn call_ollama(_state: &AppState, model: &str, prompt: &str) -> Result<String, String> {
let base_url = std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string());
let url = format!("{}/api/generate", base_url);
let req = OllamaGenerateRequest {
model: model.to_string(),
prompt: prompt.to_string(),
stream: false,
};
let client = reqwest::Client::new();
let response = client
.post(&url)
.json(&req)
.send()
.await
.map_err(|e| format!("ollama request failed: {}", e))?;
if !response.status().is_success() {
return Err(format!("ollama returned status: {}", response.status()));
}
let result: OllamaGenerateResponse = response
.json()
.await
.map_err(|e| format!("failed to parse ollama response: {}", e))?;
Ok(result.response)
call_ollama_inline(&base_url, model, prompt).await
}
// ── Phase 1: Strict keyword fast-path for intent classification ────────────────
@ -318,7 +293,39 @@ fn is_internal_admin(auth: &AuthUser) -> bool {
|| auth.claims.roles.contains(&"SUPER_ADMIN".to_string())
}
async fn call_ollama_inline(base_url: &str, model: &str, prompt: &str) -> Result<String, String> {
fn get_llm_base_url() -> String {
let llm_provider = std::env::var("LLM_PROVIDER").unwrap_or_else(|_| "ollama".to_string());
if llm_provider == "litellm" {
std::env::var("LITELLM_BASE_URL")
.unwrap_or_else(|_| "https://llm.nxtgauge.com/v1".to_string())
} else {
std::env::var("OLLAMA_BASE_URL")
.unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string())
}
}
fn get_llm_model() -> String {
let llm_provider = std::env::var("LLM_PROVIDER").unwrap_or_else(|_| "ollama".to_string());
if llm_provider == "litellm" {
std::env::var("LITELLM_MODEL")
.unwrap_or_else(|_| "askash-main".to_string())
} else {
std::env::var("OLLAMA_CHAT_MODEL")
.unwrap_or_else(|_| "gemma3:270m".to_string())
}
}
async fn call_ollama_inline(_base_url: &str, _model: &str, prompt: &str) -> Result<String, String> {
let llm_provider = std::env::var("LLM_PROVIDER").unwrap_or_else(|_| "ollama".to_string());
if llm_provider == "litellm" {
let base_url = get_llm_base_url();
let model = get_llm_model();
return call_litellm_inline(&base_url, &model, prompt).await;
}
let base_url = get_llm_base_url();
let model = get_llm_model();
let url = format!("{}/api/generate", base_url);
let req = OllamaGenerateRequest {
model: model.to_string(),
@ -346,6 +353,77 @@ async fn call_ollama_inline(base_url: &str, model: &str, prompt: &str) -> Result
Ok(result.response)
}
#[derive(Debug, Serialize)]
struct LiteLLMChatMessage {
role: String,
content: String,
}
#[derive(Debug, Serialize)]
struct LiteLLMRequest {
model: String,
messages: Vec<LiteLLMChatMessage>,
temperature: f32,
max_tokens: i32,
}
#[derive(Debug, Deserialize)]
struct LiteLLMResponse {
choices: Vec<LiteLLMChoice>,
}
#[derive(Debug, Deserialize)]
struct LiteLLMChoice {
message: LiteLLMMessage,
}
#[derive(Debug, Deserialize)]
struct LiteLLMMessage {
content: String,
}
async fn call_litellm_inline(base_url: &str, model: &str, prompt: &str) -> Result<String, String> {
let api_key = std::env::var("LITELLM_API_KEY").unwrap_or_default();
let url = format!("{}/chat/completions", base_url.trim_end_matches('/'));
let payload = LiteLLMRequest {
model: model.to_string(),
messages: vec![
LiteLLMChatMessage {
role: "user".to_string(),
content: prompt.to_string(),
},
],
temperature: 0.2,
max_tokens: 2048,
};
let client = reqwest::Client::new();
let response = client
.post(&url)
.header("Authorization", format!("Bearer {}", api_key))
.header("Content-Type", "application/json")
.json(&payload)
.send()
.await
.map_err(|e| format!("litellm request failed: {}", e))?;
if !response.status().is_success() {
return Err(format!("litellm returned status: {}", response.status()));
}
let result: LiteLLMResponse = response
.json()
.await
.map_err(|e| format!("failed to parse litellm response: {}", e))?;
if let Some(choice) = result.choices.first() {
Ok(choice.message.content.trim().to_string())
} else {
Err("litellm returned empty response".to_string())
}
}
async fn ai_chat_message(
State(state): State<AppState>,
Json(body): Json<OllamaChatRequest>,
@ -355,13 +433,13 @@ async fn ai_chat_message(
return (status, Json(payload)).into_response();
}
let ollama_base = std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string());
let model = std::env::var("OLLAMA_CHAT_MODEL").unwrap_or_else(|_| "gemma3:270m".to_string());
let ollama_base = get_llm_base_url();
let model = get_llm_model();
let default_conversation = Uuid::new_v4().to_string();
let conversation_id = body.conversation_id.unwrap_or_else(|| default_conversation);
// ── Phase 1: Strict keyword fast-path (skips Ollama when unambiguous) ─────
// ── Phase 1: Strict keyword fast-path (skips LLM when unambiguous) ─────
let (intent, confidence) = match classify_strict_keywords(&body.message) {
Some((kw_intent, kw_conf)) => (kw_intent.to_string(), kw_conf),
None => classify_intent(&body.message, &ollama_base, &model).await,
@ -606,8 +684,8 @@ async fn ai_extract_form(
State(_state): State<AppState>,
Json(body): Json<FormExtractBody>,
) -> impl IntoResponse {
let ollama_base = std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string());
let model = std::env::var("OLLAMA_CHAT_MODEL").unwrap_or_else(|_| "gemma3:270m".to_string());
let ollama_base = get_llm_base_url();
let model = get_llm_model();
let form_type = body.form_type.unwrap_or_else(|| "generic".to_string());
@ -834,8 +912,8 @@ async fn ai_generate_job_field(
}
};
let ollama_base = std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string());
let model = std::env::var("OLLAMA_CHAT_MODEL").unwrap_or_else(|_| "gemma3:270m".to_string());
let ollama_base = get_llm_base_url();
let model = get_llm_model();
let field_prompt = match body.field.as_str() {
"title" => format!(
@ -949,8 +1027,8 @@ async fn ai_generate_cover_letter(
}
};
let ollama_base = std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string());
let model = std::env::var("OLLAMA_CHAT_MODEL").unwrap_or_else(|_| "gemma3:270m".to_string());
let ollama_base = get_llm_base_url();
let model = get_llm_model();
let notes = body.additional_notes.as_deref().unwrap_or("");
let skills_str = skills.join(", ");
@ -1062,8 +1140,8 @@ async fn ai_tailor_resume(
}
};
let ollama_base = std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string());
let model = std::env::var("OLLAMA_CHAT_MODEL").unwrap_or_else(|_| "gemma3:270m".to_string());
let ollama_base = get_llm_base_url();
let model = get_llm_model();
let existing_resume = body.resume_text.as_deref().unwrap_or("Not provided");
let skills_str = skills.join(", ");
@ -1186,8 +1264,8 @@ async fn ai_auto_apply(
return (StatusCode::TOO_MANY_REQUESTS, Json(serde_json::json!({ "error": format!("Only {} generations left today", remaining) }))).into_response();
}
let ollama_base = std::env::var("OLLAMA_BASE_URL").unwrap_or_else(|_| "http://ollama.nxtgauge-ai.svc.cluster.local:11434".to_string());
let model = std::env::var("OLLAMA_CHAT_MODEL").unwrap_or_else(|_| "gemma3:270m".to_string());
let ollama_base = get_llm_base_url();
let model = get_llm_model();
let skills_str = skills.join(", ");
let mut created = 0;