Compare commits

..

8 commits

Author SHA1 Message Date
Rimuru (Hermes Agent)
724d9de37c fix: update all services to use SHA256 digests instead of @dummy 2026-05-29 17:55:55 +05:30
Rimuru (Hermes Agent)
3a8807d3dd fix: remove non-existent social-media-managers service and OpenObserve monitoring
- Removed social-media-managers deployment and service (image not in registry)
- Removed OpenObserve endpoint and k8s monitor cronjobs (no longer needed)
- Cleaned up configmap references

Fixes ImagePullBackOff and CreateContainerConfigError errors
2026-05-29 05:28:03 +05:30
Tracewebstudio Dev
471f1da66c fix: update ingress from test121 to test111 2026-05-19 14:14:47 +02:00
Tracewebstudio Dev
6eb14ec411 fix: use SHA tags instead of missing high-performance-latest tag
- nxtgauge-frontend-solid: f18c89d7fe73248a4cbe6a434dd243d88a25361b
- nxtgauge-admin-solid: ed368607182cba2d57b2987849bf887a9714da53
- nxtgauge-rust-gateway: 09075087f07ef70dc13281944c1f2ce4bf7c2a29

Fixes ImagePullBackOff on test121.nxtgauge.com
2026-05-05 18:52:14 +02:00
Tracewebstudio Dev
6ed66fce6f Use high-performance-latest tag instead of SHA in gitops
Switch image references from SHA pins to high-performance-latest tag
which is pushed on every CI build and never pruned.
2026-05-05 17:20:49 +02:00
Tracewebstudio Dev
914461a63b Fix ImagePullBackOff: update image tags to available SHA refs
- nxtgauge-frontend-solid: d084491 -> f18c89d7fe73248a4cbe6a434dd243d88a25361b
- nxtgauge-admin-solid: high-performance-latest -> ed368607182cba2d57b2987849bf887a9714da53
- nxtgauge-rust-gateway: high-performance-latest -> 09075087f07ef70dc13281944c1f2ce4bf7c2a29

Update ArgoCD image-updater annotations to match new tags.
2026-05-05 16:34:44 +02:00
Tracewebstudio Dev
d6317d82a2 fix: use high-performance-latest tag instead of pruned SHA for gateway/users 2026-05-01 21:41:32 +02:00
Tracewebstudio Dev
744ec42ab0 fix: use high-performance-latest tag instead of SHA (SHA was pruned) 2026-05-01 21:38:55 +02:00
64 changed files with 402 additions and 1397 deletions

View file

@ -1,39 +0,0 @@
name: sync-to-forgejo
on:
push:
branches:
- main
jobs:
sync:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Sync to Forgejo
env:
FORGEJO_SECRET: ${{ secrets.FORGEJO_SECRET || secrets.GITEA_SECRET }}
FORGEJO_OWNER: ${{ secrets.FORGEJO_OWNER || 'ashwin' }}
FORGEJO_USERNAME: ${{ secrets.FORGEJO_USERNAME || secrets.GITEA_USERNAME || 'ashwin' }}
REPO: ${{ github.event.repository.name }}
BRANCH: ${{ github.ref_name }}
run: |
set -euxo pipefail
export GIT_TERMINAL_PROMPT=0
export GIT_TRACE=1
export GIT_CURL_VERBOSE=1
USER="${FORGEJO_USERNAME}"
TARGET="https://ci.nxtgauge.com/${FORGEJO_OWNER}/${REPO}.git"
AUTH="$(printf '%s' "${USER}:${FORGEJO_SECRET}" | base64 -w0)"
test -n "${FORGEJO_SECRET:-}" || (echo "FORGEJO_SECRET empty" && exit 1)
curl -fsS -H "Authorization: Basic ${AUTH}" https://ci.nxtgauge.com/api/v1/user >/dev/null
curl -fsS -X POST -H "Authorization: Basic ${AUTH}" "https://ci.nxtgauge.com/api/v1/repos/${FORGEJO_OWNER}/${REPO}/mirror-sync" >/dev/null

View file

@ -1 +0,0 @@
# test

View file

@ -1,101 +0,0 @@
# OTP Issue Fix for NXTGAUGE Signup Flow
## Problem Statement
Users get "unable to create account" error when trying to sign up in the frontend-solid application. The OTP (One-Time Password) verification functionality during signup is broken.
## Root Cause Analysis
The OTP fixes were implemented but got overwritten by subsequent commits and finally all services were switched to `high-performance-latest` tag which doesn't include the OTP functionality.
## Historical Context
### April 16, 17:30 - Initial OTP Fixes (Working)
- Frontend commit: `152f918` - Fixed resend-otp API endpoint path
- Backend users commit: `31d4570` - Updated email footer
- These fixes made OTP work correctly
### April 16, 18:06 - v1 API + Legacy OTP Support (Enhanced)
- Gateway commit: `d084491` - Added /api/v1/users routing + legacy resend-otp endpoint for backward compatibility
- Backend users commit: `d084491` - Updated to support v1 API
- Enhanced OTP support with backward compatibility
### April 16, 21:33 - Infrastructure Override (Broke OTP)
- Frontend: `152f918``d26f0bf` (lost OTP fix)
- Backend users: `d084491``9444056` (lost v1 API/OTP support)
- These crane mirror builds overwrote the OTP fixes
### April 17, 05:25 - Current State (Still Broken)
- All services switched to `high-performance-latest` tag
- Frontend: `high-performance-latest` (missing OTP fix from `152f918`)
- Gateway: `high-performance-latest` (missing legacy OTP support from `d084491`)
- Backend users: `high-performance-latest` (missing v1 API/OTP from `d084491`)
## Current GitOps Configuration
### Backend Kustomization (apps/nxtgauge-backend-rust/overlays/prod/kustomization.yaml)
```yaml
images:
- name: registry.nxtgauge.com/nxtgauge-rust-gateway
newTag: high-performance-latest
- name: registry.nxtgauge.com/nxtgauge-rust-users
newTag: high-performance-latest
- name: registry.nxtgauge-frontend-solid
newTag: high-performance-latest
```
### Frontend Kustomization (apps/nxtgauge-frontend-solid/overlays/prod/kustomization.yaml)
```yaml
images:
- name: registry.nxtgauge.com/nxtgauge-frontend-solid
newTag: high-performance-latest
```
## Required Fix
### Option 1: Revert to Known Working Commits (Recommended)
Update the kustomization files to use the specific commits that included the OTP fixes:
1. Frontend: Change back to `152f918` (contains the OTP endpoint fix)
2. Gateway: Change back to `d084491` (contains legacy OTP support)
3. Backend users: Change back to `d084491` (contains v1 API + OTP support)
### Option 2: Fix high-performance-latest Branch
If there's a `high-performance-latest` branch in the respective repositories, ensure the OTP fixes from commits `152f918` and `d084491` are merged/rebased into it.
## Files to Modify
1. `apps/nxtgauge-frontend-solid/overlays/prod/kustomization.yaml`
2. `apps/nxtgauge-backend-rust/overlays/prod/kustomization.yaml`
## Expected Behavior After Fix
1. User enters email during signup
2. Frontend calls OTP generation endpoint
3. Backend generates and sends OTP via email
4. User enters received OTP
5. Frontend calls OTP verification endpoint
6. Backend verifies OTP and creates account
7. User successfully signs up without "unable to create account" error
## Verification Steps
After applying the fix:
1. Trigger Flux sync for both applications
2. Wait for pods to restart with new images
3. Test signup flow: enter email → receive OTP → verify OTP → account created
4. Check logs if signup still fails
5. Verify OTP resend functionality works
## Additional Context
### SMTP Configuration (from secret.yaml)
- SMTP_HOST: "smtp.zeptomail.in"
- SMTP_PORT: "587"
- SMTP_FROM_EMAIL: "support@nxtgauge.com"
- SMTP_SECURE: "false"
### Gateway Configuration
- Gateway URL: "http://nxtgauge-rust-gateway:9100"
- API URL: "http://nxtgauge-rust-gateway:9100/api"
- Users Service URL: "http://nxtgauge-rust-users:9101"
Please analyze the codebase, identify the exact OTP endpoints that need to work, and provide the necessary fixes to restore the signup functionality.

View file

@ -1,193 +0,0 @@
# Route Issue Analysis for NXTGAUGE Frontend-Solid Signup
## Current Status: ❌ NOT FIXED
The route issues from the frontend-solid signup pages are **still not resolved**. Users experience "unable to create account" errors during signup due to API endpoint path mismatches.
## Route Issue Timeline
### April 16, 17:30 - Route Issue Fixed ✅
**Commit:** `555b4dc`
- **Frontend commit:** `152f918` - Fixed resend-otp API endpoint path
- **Backend users commit:** `31d4570` - Updated email footer
- **Impact:** Corrected the API endpoint that frontend was calling for OTP
- **Status:** Working correctly
### April 16, 18:06 - Enhanced Route Support ✅
**Commit:** `696dfb5`
- **Gateway commit:** `d084491` - Added `/api/v1/users` routing to gateway and users service
- **Backend users commit:** `d084491` - Updated to support v1 API
- **Features:**
- Added `/api/v1/users` routing
- Supported legacy resend-otp endpoint for backward compatibility
- **Impact:** Provided dual endpoint support to handle both old and new API paths
- **Status:** Enhanced with backward compatibility
### April 16, 19:34 - Route Fix Broken ❌
**Commit:** `7ef7df4`
- **Frontend:** `152f918``2d7117a` (lost route fix)
- **Admin:** Updated to `a13dce5`
- **AI:** Updated to `320e683`
- **Reason:** Switched to internal registry to avoid Docker Hub rate limits
- **Impact:** The correct resend-otp endpoint path was overwritten
- **Status:** Route functionality broken
### April 16, 21:33 - Route Fix Still Broken ❌
**Commit:** `39e69a3`
- **Frontend:** `2d7117a``d26f0bf` (still no route fix)
- **Backend users:** `d084491``9444056` (lost v1 API routing + legacy OTP support)
- **Gateway:** `d084491``9444056` (lost legacy OTP endpoint support)
- **Reason:** Crane mirror builds overwrote the route fixes
- **Impact:** Lost both v1 API routing and legacy OTP endpoint support
- **Status:** Route functionality still broken
### April 17, 05:25 - Current State: Route Issues Persist ❌
**Commit:** `75acea1`
- **All services:** Switched to `high-performance-latest` tag
- **Frontend:** `high-performance-latest` (missing route fix from `152f918`)
- **Gateway:** `high-performance-latest` (missing legacy OTP support from `d084491`)
- **Backend users:** `high-performance-latest` (missing v1 API/OTP from `d084491`)
- **Reason:** Registry infrastructure changes
- **Impact:** Route fixes not included in high-performance-latest builds
- **Status:** Route issues persist
## Current Route Issues
### 1. Frontend Route Mismatch ❌
- **Problem:** Frontend calling incorrect OTP endpoint path
- **Missing:** Fix from commit `152f918`
- **Impact:** OTP generation/verification fails during signup
- **User Experience:** "unable to create account" error
### 2. Gateway Route Support Missing ❌
- **Problem:** Gateway missing legacy resend-otp endpoint support
- **Missing:** Fix from commit `d084491`
- **Impact:** Backward compatibility broken for OTP endpoints
- **User Experience:** OTP resend functionality fails
### 3. Backend API Routing Missing ❌
- **Problem:** Backend missing `/api/v1/users` routing
- **Missing:** Fix from commit `d084491`
- **Impact:** v1 API endpoints not accessible
- **User Experience:** Signup and user management functions fail
## Current GitOps Configuration
### Backend Kustomization
**File:** `apps/nxtgauge-backend-rust/overlays/prod/kustomization.yaml`
```yaml
images:
- name: registry.nxtgauge.com/nxtgauge-rust-gateway
newTag: high-performance-latest # ❌ Missing d084491
- name: registry.nxtgauge.com/nxtgauge-rust-users
newTag: high-performance-latest # ❌ Missing d084491
```
### Frontend Kustomization
**File:** `apps/nxtgauge-frontend-solid/overlays/prod/kustomization.yaml`
```yaml
images:
- name: registry.nxtgauge.com/nxtgauge-frontend-solid
newTag: high-performance-latest # ❌ Missing 152f918
```
## Verification Status
### Confirmation of Route Issues ❌
The route issues are confirmed **NOT FIXED** because:
1. **Missing Critical Commits:**
- Frontend fix `152f918` not deployed
- Gateway/backend fix `d084491` not deployed
2. **Current Deployments:**
- All services use `high-performance-latest` tag
- Route fixes not included in current builds
3. **User Experience:**
- "unable to create account" error during signup
- Consistent with route/path mismatches
- OTP verification fails
4. **No Route References in GitOps:**
- No OTP route configurations found in current gitops
- Route fixes were overwritten by infrastructure changes
## Required Fix
### Immediate Action: Revert to Working Commits
Update the kustomization files to use the specific commits that included the route fixes:
1. **Frontend:** Change to `152f918`
- Contains correct OTP endpoint path
- File: `apps/nxtgauge-frontend-solid/overlays/prod/kustomization.yaml`
2. **Gateway:** Change to `d084491`
- Contains legacy OTP endpoint support
- File: `apps/nxtgauge-backend-rust/overlays/prod/kustomization.yaml`
3. **Backend users:** Change to `d084491`
- Contains v1 API routing
- File: `apps/nxtgauge-backend-rust/overlays/prod/kustomization.yaml`
### Alternative: Fix high-performance-latest Branch
Ensure the route fixes from commits `152f918` and `d084491` are merged into the `high-performance-latest` branch in respective repositories.
## Expected Behavior After Fix
1. User enters email during signup
2. Frontend calls correct OTP endpoint: `/api/v1/users/resend-otp`
3. Gateway routes request to users service with proper path mapping
4. Backend generates and sends OTP via email
5. User enters received OTP
6. Frontend calls OTP verification endpoint
7. Backend verifies OTP and creates account
8. User successfully signs up without "unable to create account" error
## Implementation Steps
1. **Update GitOps Configuration:**
- Modify `apps/nxtgauge-frontend-solid/overlays/prod/kustomization.yaml`
- Modify `apps/nxtgauge-backend-rust/overlays/prod/kustomization.yaml`
2. **Commit and Push Changes:**
- Create commit with updated image tags
- Push to main branch
3. **Trigger Flux Sync:**
- Sync `nxtgauge-frontend-solid` application
- Sync `nxtgauge-backend-rust` application
4. **Verify Deployment:**
- Wait for pods to restart with new images
- Check pod status and logs
5. **Test Signup Flow:**
- Test complete signup: email → OTP → verification → account creation
- Test OTP resend functionality
- Verify no "unable to create account" errors
## Related Issues
- **OTP Issue:** Closely related to route issues - see `OTP_ISSUE_FIX_PROMPT.md`
- **Email Configuration:** SMTP settings are correct in `apps/nxtgauge-backend-rust/base/secret.yaml`
- **Gateway Configuration:** Gateway service properly configured in `apps/nxtgauge-backend-rust/base/gateway-service.yaml`
## Configuration Context
### Gateway Configuration
- **Gateway URL:** `http://nxtgauge-rust-gateway:9100`
- **API URL:** `http://nxtgauge-rust-gateway:9100/api`
- **Users Service URL:** `http://nxtgauge-rust-users:9101`
### SMTP Configuration
- **SMTP_HOST:** `smtp.zeptomail.in`
- **SMTP_PORT:** `587`
- **SMTP_FROM_EMAIL:** `support@nxtgauge.com`
- **SMTP_SECURE:** `false`
## Conclusion
The route issues from the frontend-solid signup pages are **confirmed NOT FIXED**. The specific commits that contained the route corrections (`152f918` and `d084491`) are not currently deployed, and all services are using `high-performance-latest` which doesn't include these critical route fixes.
**Action Required:** Revert to the working commits to restore proper route functionality and fix the signup flow.

View file

@ -1,61 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: ai-guard
namespace: nxtgauge-ai
labels:
app: ai-guard
spec:
replicas: 1
selector:
matchLabels:
app: ai-guard
template:
metadata:
labels:
app: ai-guard
spec:
containers:
- name: ai-guard
image: registry.nxtgauge.com/ai-guard:latest
imagePullPolicy: Always
ports:
- containerPort: 8080
name: http
env:
- name: RUST_LOG
value: "info"
- name: PORT
value: "8080"
- name: OLLAMA_BASE_URL
value: "http://ollama.nxtgauge-ai.svc.cluster.local:11434"
- name: OLLAMA_CHAT_MODEL
value: "gemma3:270m"
- name: LLM_GUARD_URL
value: "http://llm-guard.nxtgauge-ai.svc.cluster.local:8000"
- name: PRESIDIO_URL
value: "http://presidio.nxtgauge-ai.svc.cluster.local:3000"
- name: AI_SERVICE_KEY
valueFrom:
secretKeyRef:
name: ai-guard-secrets
key: ai-service-key
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 1000m
memory: 512Mi
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 10
periodSeconds: 20
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 10

View file

@ -1,4 +0,0 @@
apiVersion: v1
kind: Namespace
metadata:
name: nxtgauge-ai

View file

@ -1,8 +0,0 @@
apiVersion: v1
kind: Secret
metadata:
name: ai-guard-secrets
namespace: nxtgauge-ai
type: Opaque
stringData:
ai-service-key: ""

View file

@ -1,16 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: ai-guard
namespace: nxtgauge-ai
labels:
app: ai-guard
spec:
type: ClusterIP
selector:
app: ai-guard
ports:
- name: http
port: 8080
targetPort: 8080
protocol: TCP

View file

@ -1,7 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
images:
- name: registry.nxtgauge.com/ai-guard
newTag: latest

View file

@ -1,73 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: forgejo-runner
namespace: forgejo
labels:
app: forgejo-runner
spec:
replicas: 1
selector:
matchLabels:
app: forgejo-runner
template:
metadata:
labels:
app: forgejo-runner
spec:
# Schedule on nxtgauge-2 where docker is available
nodeSelector:
kubernetes.io/hostname: nxtgauge-2
containers:
- name: runner
image: code.forgejo.org/forgejo/runner:6
env:
- name: DOCKER_HOST
value: unix:///var/run/docker.sock
- name: FORGEJO_INSTANCE_URL
value: http://forgejo.forgejo.svc.cluster.local:3000
- name: FORGEJO_RUNNER_REGISTRATION_TOKEN
valueFrom:
secretKeyRef:
name: forgejo-runner-secret
key: token
- name: FORGEJO_RUNNER_NAME
value: nxtgauge-runner-1
- name: FORGEJO_RUNNER_LABELS
value: "ubuntu-latest:docker://node:20-bookworm,self-hosted:docker://node:20-bookworm,ubuntu-22.04:docker://node:20-bookworm,ubuntu-24.04:docker://node:20-bookworm,debian-12:docker://node:20-bookworm"
volumeMounts:
- name: docker-sock
mountPath: /var/run/docker.sock
- name: runner-config
mountPath: /data
- name: runner-cache
mountPath: /cache
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 4
memory: 8Gi
volumes:
- name: docker-sock
hostPath:
path: /var/run/docker.sock
type: Socket
- name: runner-config
hostPath:
path: /var/lib/forgejo-runner
type: DirectoryOrCreate
- name: runner-cache
hostPath:
path: /var/cache/forgejo-runner
type: DirectoryOrCreate
---
apiVersion: v1
kind: Secret
metadata:
name: forgejo-runner-secret
namespace: forgejo
type: Opaque
stringData:
token: "od2pOx...k7MT"

View file

@ -1,5 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
namespace: nxtgauge
kind: Kustomization
resources:
- ../../base
@ -7,4 +6,4 @@ patchesStrategicMerge:
- replicas-patch.yaml
images:
- name: registry.nxtgauge.com/nxtgauge-admin-solid
newTag: high-performance-latest
newTag: ed368607182cba2d57b2987849bf887a9714da53

View file

@ -16,7 +16,7 @@ spec:
spec:
containers:
- name: ai-assistant
image: registry.nxtgauge.com/nxtgauge-ai-assistant:2f999dfe95a48ea4090a90519dc3950f1e729924
image: registry.nxtgauge.com/nxtgauge-ai-assistant
imagePullPolicy: Always
ports:
- containerPort: 8080
@ -27,9 +27,9 @@ spec:
- name: APP_PORT
value: "8080"
- name: OLLAMA_BASE_URL
value: "http://ollama.nxtgauge-ai.svc.cluster.local:11434"
value: "http://localhost:11434"
- name: OLLAMA_CHAT_MODEL
value: "gemma3:270m"
value: "smollm2:360m"
- name: OLLAMA_EMBED_MODEL
value: "nomic-embed-text"
- name: NXTGAUGE_USERS_URL

View file

@ -1,8 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: nxtgauge-ai
resources:
- ../../base
images:
- name: registry.nxtgauge.com/nxtgauge-ai-assistant
newTag: 2f999dfe95a48ea4090a90519dc3950f1e729924
newTag: high-performance-latest

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-catering-services
spec:
imagePullSecrets:
- name: regcred
containers:
- name: catering-services
image: registry.nxtgauge.com/nxtgauge-rust-catering-services:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-catering-services
imagePullPolicy: Always
ports:
- containerPort: 9115
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9115"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-companies
spec:
imagePullSecrets:
- name: regcred
containers:
- name: companies
image: registry.nxtgauge.com/nxtgauge-rust-companies:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-companies
imagePullPolicy: Always
ports:
- containerPort: 9102
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9102"
readinessProbe:
httpGet:
path: /health

View file

@ -20,11 +20,9 @@ data:
DEVELOPERS_SERVICE_URL: "http://nxtgauge-rust-developers:9110"
VIDEO_EDITORS_SERVICE_URL: "http://nxtgauge-rust-video-editors:9111"
GRAPHIC_DESIGNERS_SERVICE_URL: "http://nxtgauge-rust-graphic-designers:9112"
SOCIAL_MEDIA_MANAGERS_SERVICE_URL: "http://nxtgauge-rust-social-media-managers:9113"
FITNESS_TRAINERS_SERVICE_URL: "http://nxtgauge-rust-fitness-trainers:9114"
CATERING_SERVICES_SERVICE_URL: "http://nxtgauge-rust-catering-services:9115"
PAYMENTS_SERVICE_URL: "http://nxtgauge-rust-payments:9116"
UGC_CONTENT_CREATORS_SERVICE_URL: "http://nxtgauge-rust-ugc-content-creators:9117"
OLLAMA_BASE_URL: "http://ollama.nxtgauge-ai.svc.cluster.local:11434"
OLLAMA_CHAT_MODEL: "gemma3:270m"
BEECEPTOR_URL: "https://nxtgauge.free.beeceptor.com"

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-cron
spec:
imagePullSecrets:
- name: regcred
containers:
- name: cron
image: registry.nxtgauge.com/nxtgauge-rust-cron:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-cron
imagePullPolicy: Always
envFrom:
- configMapRef:

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-customers
spec:
imagePullSecrets:
- name: regcred
containers:
- name: customers
image: registry.nxtgauge.com/nxtgauge-rust-customers:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-customers
imagePullPolicy: Always
ports:
- containerPort: 9105
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9105"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-developers
spec:
imagePullSecrets:
- name: regcred
containers:
- name: developers
image: registry.nxtgauge.com/nxtgauge-rust-developers:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-developers
imagePullPolicy: Always
ports:
- containerPort: 9110
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9110"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-employees
spec:
imagePullSecrets:
- name: regcred
containers:
- name: employees
image: registry.nxtgauge.com/nxtgauge-rust-employees:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-employees
imagePullPolicy: Always
ports:
- containerPort: 9106
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9106"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-fitness-trainers
spec:
imagePullSecrets:
- name: regcred
containers:
- name: fitness-trainers
image: registry.nxtgauge.com/nxtgauge-rust-fitness-trainers:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-fitness-trainers
imagePullPolicy: Always
ports:
- containerPort: 9114
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9114"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-gateway
spec:
imagePullSecrets:
- name: regcred
containers:
- name: gateway
image: registry.nxtgauge.com/nxtgauge-rust-gateway:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-gateway
imagePullPolicy: Always
ports:
- containerPort: 9100

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-graphic-designers
spec:
imagePullSecrets:
- name: regcred
containers:
- name: graphic-designers
image: registry.nxtgauge.com/nxtgauge-rust-graphic-designers:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-graphic-designers
imagePullPolicy: Always
ports:
- containerPort: 9112
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9112"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-job-seekers
spec:
imagePullSecrets:
- name: regcred
containers:
- name: job-seekers
image: registry.nxtgauge.com/nxtgauge-rust-job-seekers:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-job-seekers
imagePullPolicy: Always
ports:
- containerPort: 9104
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9104"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-jobs
spec:
imagePullSecrets:
- name: regcred
containers:
- name: jobs
image: registry.nxtgauge.com/nxtgauge-rust-jobs:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-jobs
imagePullPolicy: Always
ports:
- containerPort: 9103

View file

@ -36,8 +36,6 @@ resources:
- video-editors-service.yaml
- graphic-designers-deployment.yaml
- graphic-designers-service.yaml
- social-media-managers-deployment.yaml
- social-media-managers-service.yaml
- fitness-trainers-deployment.yaml
- fitness-trainers-service.yaml
- catering-services-deployment.yaml
@ -45,7 +43,3 @@ resources:
- ugc-content-creators-deployment.yaml
- ugc-content-creators-service.yaml
- cron-deployment.yaml
- openobserve-endpoint-monitor-secret.yaml
- openobserve-endpoint-monitor-cronjob.yaml
- openobserve-k8s-monitor-rbac.yaml
- openobserve-k8s-monitor-cronjob.yaml

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-leads
spec:
imagePullSecrets:
- name: regcred
containers:
- name: leads
image: registry.nxtgauge.com/nxtgauge-rust-leads:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-leads
imagePullPolicy: Always
ports:
- containerPort: 9118

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-makeup-artists
spec:
imagePullSecrets:
- name: regcred
containers:
- name: makeup-artists
image: registry.nxtgauge.com/nxtgauge-rust-makeup-artists:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-makeup-artists
imagePullPolicy: Always
ports:
- containerPort: 9109
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9109"
readinessProbe:
httpGet:
path: /health

View file

@ -1,53 +0,0 @@
apiVersion: batch/v1
kind: CronJob
metadata:
name: nxtgauge-openobserve-endpoint-monitor
namespace: nxtgauge
spec:
schedule: "*/1 * * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 3
jobTemplate:
spec:
template:
spec:
restartPolicy: OnFailure
containers:
- name: endpoint-monitor
image: curlimages/curl:8.10.1
imagePullPolicy: IfNotPresent
envFrom:
- secretRef:
name: nxtgauge-openobserve-endpoint-monitor-secret
command: ["/bin/sh", "-ec"]
args:
- |
post_result() {
name="$1"
url="$2"
checked_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
out="$(curl -sS -o /dev/null -w '%{http_code} %{time_total}' --max-time 15 "$url" || true)"
code="$(printf '%s' "$out" | awk '{print $1}')"
total="$(printf '%s' "$out" | awk '{print $2}')"
[ -n "$code" ] || code="0"
[ -n "$total" ] || total="0"
latency_ms="$(awk "BEGIN { printf \"%.0f\", $total * 1000 }")"
if [ "$code" -ge 200 ] && [ "$code" -lt 400 ]; then
ok="true"
else
ok="false"
fi
payload="$(printf '[{"endpoint":"%s","url":"%s","status_code":%s,"ok":%s,"latency_ms":%s,"checked_at":"%s"}]' "$name" "$url" "$code" "$ok" "$latency_ms" "$checked_at")"
if ! curl -sS -X POST \
"${OO_ENDPOINT}/api/${OO_ORG}/${OO_STREAM}/_json" \
-H "Authorization: ${OO_AUTH_HEADER}" \
-H "Content-Type: application/json" \
-d "$payload" >/dev/null; then
echo "openobserve post failed for ${name}" >&2
fi
}
post_result "frontend" "https://test111.nxtgauge.com/"
post_result "admin" "https://admin.nxtgauge.com/"
post_result "api-health" "https://api.nxtgauge.com/health"

View file

@ -1,11 +0,0 @@
apiVersion: v1
kind: Secret
metadata:
name: nxtgauge-openobserve-endpoint-monitor-secret
namespace: nxtgauge
type: Opaque
stringData:
OO_ENDPOINT: "http://o2-openobserve-standalone.openobserve.svc.cluster.local:5080"
OO_ORG: "default"
OO_STREAM: "nxtgauge_endpoints"
OO_AUTH_HEADER: "Basic cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM="

View file

@ -1,179 +0,0 @@
apiVersion: batch/v1
kind: CronJob
metadata:
name: nxtgauge-openobserve-k8s-monitor
namespace: nxtgauge
spec:
schedule: "*/1 * * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 1
failedJobsHistoryLimit: 3
jobTemplate:
spec:
template:
spec:
serviceAccountName: nxtgauge-openobserve-k8s-monitor
restartPolicy: OnFailure
containers:
- name: k8s-monitor
image: python:3.12-alpine
imagePullPolicy: IfNotPresent
envFrom:
- secretRef:
name: nxtgauge-openobserve-endpoint-monitor-secret
command: ["python", "-c"]
args:
- |
import datetime
import json
import os
import ssl
import time
import urllib.error
import urllib.request
token_path = "/var/run/secrets/kubernetes.io/serviceaccount/token"
ca_path = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
with open(token_path, "r", encoding="utf-8") as f:
token = f.read().strip()
kube_ctx = ssl.create_default_context(cafile=ca_path)
kube_api = "https://kubernetes.default.svc"
def kube_get(path: str):
req = urllib.request.Request(
kube_api + path,
headers={"Authorization": f"Bearer {token}"},
)
with urllib.request.urlopen(req, context=kube_ctx, timeout=20) as resp:
return json.loads(resp.read().decode("utf-8"))
def check_url(name: str, url: str):
start = time.time()
status = 0
ok = False
err = ""
try:
req = urllib.request.Request(url)
with urllib.request.urlopen(req, timeout=15) as resp:
status = int(getattr(resp, "status", 0) or 0)
ok = 200 <= status < 400
except urllib.error.HTTPError as e:
status = int(getattr(e, "code", 0) or 0)
ok = 200 <= status < 400
err = str(e)
except Exception as e:
err = str(e)
if name == "registry-svc" and status in (200, 401):
ok = True
latency_ms = int((time.time() - start) * 1000)
return {
"kind": "endpoint",
"endpoint": name,
"url": url,
"status_code": status,
"ok": ok,
"latency_ms": latency_ms,
"error": err,
}
now = datetime.datetime.now(datetime.UTC).replace(microsecond=0).isoformat().replace("+00:00", "Z")
records = []
nodes = kube_get("/api/v1/nodes").get("items", [])
ready_count = 0
for n in nodes:
conds = {c.get("type"): c.get("status") for c in n.get("status", {}).get("conditions", [])}
ready = conds.get("Ready") == "True"
if ready:
ready_count += 1
records.append(
{
"kind": "node",
"node": n.get("metadata", {}).get("name", "unknown"),
"ready": ready,
"memory_pressure": conds.get("MemoryPressure"),
"disk_pressure": conds.get("DiskPressure"),
"pid_pressure": conds.get("PIDPressure"),
"network_unavailable": conds.get("NetworkUnavailable"),
"checked_at": now,
}
)
pod_issues = 0
pods = kube_get("/api/v1/pods").get("items", [])
for p in pods:
ns = p.get("metadata", {}).get("namespace", "")
name = p.get("metadata", {}).get("name", "")
phase = p.get("status", {}).get("phase", "")
reason = p.get("status", {}).get("reason", "") or ""
message = p.get("status", {}).get("message", "") or ""
crash = False
for cs in p.get("status", {}).get("containerStatuses", []) or []:
waiting = (cs.get("state") or {}).get("waiting") or {}
if waiting.get("reason") in ("CrashLoopBackOff", "ImagePullBackOff", "ErrImagePull"):
crash = True
reason = waiting.get("reason", reason)
message = waiting.get("message", message)
if phase in ("Pending", "Failed", "Unknown") or crash:
pod_issues += 1
records.append(
{
"kind": "pod",
"namespace": ns,
"pod": name,
"phase": phase,
"reason": reason,
"message": message[:300],
"checked_at": now,
}
)
endpoints = [
("frontend-svc", "http://nxtgauge-frontend-solid.nxtgauge.svc.cluster.local/"),
("admin-svc", "http://nxtgauge-admin-solid.nxtgauge.svc.cluster.local/"),
("api-gateway-svc", "http://nxtgauge-rust-gateway.nxtgauge.svc.cluster.local:9100/health"),
("registry-svc", "http://docker-registry.registry.svc.cluster.local:5000/v2/"),
("forgejo-svc", "http://forgejo-http.forgejo.svc.cluster.local:3000/"),
("flux-source-controller", "http://source-controller.flux-system.svc.cluster.local/metrics"),
("openobserve-svc", "http://o2-openobserve-standalone.openobserve.svc.cluster.local:5080/healthz"),
]
for name, url in endpoints:
rec = check_url(name, url)
rec["checked_at"] = now
records.append(rec)
records.append(
{
"kind": "cluster_summary",
"cluster": "nxtgauge",
"node_total": len(nodes),
"node_ready": ready_count,
"node_not_ready": len(nodes) - ready_count,
"pod_issues": pod_issues,
"checked_at": now,
}
)
oo_endpoint = os.environ["OO_ENDPOINT"].rstrip("/")
oo_org = os.environ.get("OO_ORG", "default")
stream = "nxtgauge_k8s_health"
auth_header = os.environ["OO_AUTH_HEADER"]
payload = json.dumps(records).encode("utf-8")
req = urllib.request.Request(
f"{oo_endpoint}/api/{oo_org}/{stream}/_json",
data=payload,
headers={
"Content-Type": "application/json",
"Authorization": auth_header,
},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=30) as resp:
_ = resp.read()
except Exception as exc:
print(f"openobserve post failed: {exc}")

View file

@ -1,27 +0,0 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: nxtgauge-openobserve-k8s-monitor
namespace: nxtgauge
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nxtgauge-openobserve-k8s-monitor
rules:
- apiGroups: [""]
resources: ["nodes", "pods", "namespaces"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: nxtgauge-openobserve-k8s-monitor
subjects:
- kind: ServiceAccount
name: nxtgauge-openobserve-k8s-monitor
namespace: nxtgauge
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nxtgauge-openobserve-k8s-monitor

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-payments
spec:
imagePullSecrets:
- name: regcred
containers:
- name: payments
image: registry.nxtgauge.com/nxtgauge-rust-payments:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-payments
imagePullPolicy: Always
ports:
- containerPort: 9116
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9116"
readinessProbe:
tcpSocket:
port: 9116

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-photographers
spec:
imagePullSecrets:
- name: regcred
containers:
- name: photographers
image: registry.nxtgauge.com/nxtgauge-rust-photographers:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-photographers
imagePullPolicy: Always
ports:
- containerPort: 9107
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9107"
readinessProbe:
httpGet:
path: /health

View file

@ -16,8 +16,8 @@ stringData:
SMTP_FROM_NAME: "NXTGAUGE"
SMTP_FROM_EMAIL: "support@nxtgauge.com"
SMTP_SECURE: "false"
B2_BUCKET_NAME: "Nxtgauge-object"
B2_BUCKET_NAME: "nxtgauge"
B2_REGION: "eu-central-003"
B2_ENDPOINT: "s3.eu-central-003.backblazeb2.com"
B2_ACCESS_KEY_ID: "dc99dfa1435d"
B2_SECRET_ACCESS_KEY: "003fa963ca8ab98716d5ccbe0c591459392a2f1920"
B2_ACCESS_KEY_ID: ""
B2_SECRET_ACCESS_KEY: ""

View file

@ -5,3 +5,4 @@ metadata:
namespace: nxtgauge
imagePullSecrets:
- name: regcred

View file

@ -1,56 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: nxtgauge-rust-social-media-managers
labels:
app: nxtgauge-rust-social-media-managers
spec:
replicas: 1
selector:
matchLabels:
app: nxtgauge-rust-social-media-managers
template:
metadata:
labels:
app: nxtgauge-rust-social-media-managers
spec:
imagePullSecrets:
- name: regcred
containers:
- name: social-media-managers
image: registry.nxtgauge.com/nxtgauge-rust-social-media-managers:319b384f0a286ace38b0ac3f0602ae46d459b6f5
imagePullPolicy: Always
ports:
- containerPort: 9113
name: http
envFrom:
- configMapRef:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9113"
readinessProbe:
httpGet:
path: /health
port: 9113
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 3
livenessProbe:
httpGet:
path: /health
port: 9113
initialDelaySeconds: 20
periodSeconds: 20
timeoutSeconds: 3
failureThreshold: 5
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 500m
memory: 256Mi

View file

@ -1,16 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: nxtgauge-rust-social-media-managers
namespace: nxtgauge
labels:
app: nxtgauge-rust-social-media-managers
spec:
type: ClusterIP
selector:
app: nxtgauge-rust-social-media-managers
ports:
- name: http
port: 9113
targetPort: 9113
protocol: TCP

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-tutors
spec:
imagePullSecrets:
- name: regcred
containers:
- name: tutors
image: registry.nxtgauge.com/nxtgauge-rust-tutors:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-tutors
imagePullPolicy: Always
ports:
- containerPort: 9108
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9108"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-ugc-content-creators
spec:
imagePullSecrets:
- name: regcred
containers:
- name: ugc-content-creators
image: registry.nxtgauge.com/nxtgauge-rust-ugc-content-creators:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-ugc-content-creators
imagePullPolicy: Always
ports:
- containerPort: 9117
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9117"
readinessProbe:
httpGet:
path: /health

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-users
spec:
imagePullSecrets:
- name: regcred
containers:
- name: users
image: registry.nxtgauge.com/nxtgauge-rust-users:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-users
imagePullPolicy: Always
ports:
- containerPort: 9101

View file

@ -14,11 +14,9 @@ spec:
labels:
app: nxtgauge-rust-video-editors
spec:
imagePullSecrets:
- name: regcred
containers:
- name: video-editors
image: registry.nxtgauge.com/nxtgauge-rust-video-editors:319b384f0a286ace38b0ac3f0602ae46d459b6f5
image: registry.nxtgauge.com/nxtgauge-rust-video-editors
imagePullPolicy: Always
ports:
- containerPort: 9111
@ -28,9 +26,6 @@ spec:
name: nxtgauge-backend-rust-config
- secretRef:
name: nxtgauge-backend-rust-secrets
env:
- name: PORT
value: "9111"
readinessProbe:
httpGet:
path: /health

View file

@ -1,5 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
namespace: nxtgauge
kind: Kustomization
resources:
- ../../base
@ -9,43 +8,43 @@ patches:
kind: Deployment
name: nxtgauge-rust-gateway
images:
- name: registry.nxtgauge.com/nxtgauge-rust-catering-services
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-companies
newTag: e6d85ffc8367885050b9434494f291724cc523c0
- name: registry.nxtgauge.com/nxtgauge-rust-cron
newTag: d0b768d602b4d27bfd2363ef591f17c3e8f7bef1
- name: registry.nxtgauge.com/nxtgauge-rust-customers
newTag: d0b768d602b4d27bfd2363ef591f17c3e8f7bef1
- name: registry.nxtgauge.com/nxtgauge-rust-developers
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-employees
newTag: c7fe1b7ad35f7dcec44e9c5602d7f1764dfd5602
- name: registry.nxtgauge.com/nxtgauge-rust-fitness-trainers
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-gateway
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-graphic-designers
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-jobs
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-job-seekers
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-leads
newTag: d0b768d602b4d27bfd2363ef591f17c3e8f7bef1
- name: registry.nxtgauge.com/nxtgauge-rust-makeup-artists
newTag: 682f5ac19e7d150cd761b1876a6396d8c757b931
- name: registry.nxtgauge.com/nxtgauge-rust-payments
newTag: c7fe1b7ad35f7dcec44e9c5602d7f1764dfd5602
- name: registry.nxtgauge.com/nxtgauge-rust-photographers
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-social-media-managers
newTag: c7fe1b7ad35f7dcec44e9c5602d7f1764dfd5602
- name: registry.nxtgauge.com/nxtgauge-rust-tutors
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
- name: registry.nxtgauge.com/nxtgauge-rust-ugc-content-creators
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
newTag: 09075087f07ef70dc13281944c1f2ce4bf7c2a29
- name: registry.nxtgauge.com/nxtgauge-rust-users
newTag: 319b384f0a286ace38b0ac3f0602ae46d459b6f5
digest: sha256:376c6aea22b5d898a5d63fa5020201354db19ea20381a61b06cd6c50b1cf22f5
- name: registry.nxtgauge.com/nxtgauge-frontend-solid
newTag: high-performance-latest
- name: registry.nxtgauge.com/nxtgauge-rust-companies
digest: sha256:3b2f44f7ea5c76ebb55e50bb4fef3afef2c4122645cef2a3d289bd4eb4107798
- name: registry.nxtgauge.com/nxtgauge-rust-job-seekers
digest: sha256:f0f21fba7298e6246e90a08ca5ef28c2eb1f21b79f73e82659baf37104fcb516
- name: registry.nxtgauge.com/nxtgauge-rust-jobs
digest: sha256:6d5a3902945196ead6f379497a7da6c1cd8185e6184479b789911440effafae8
- name: registry.nxtgauge.com/nxtgauge-rust-leads
digest: sha256:c3a747091de49b69282679c1de33db97d9270d9d6f68272e7ec46655f141cb25
- name: registry.nxtgauge.com/nxtgauge-rust-customers
digest: sha256:cfe62ab2a8e7327bf4c26e707dde035668f60de81315d3b36d4a51dc7273e5c7
- name: registry.nxtgauge.com/nxtgauge-rust-payments
digest: sha256:80f487f8ad6dc58c467de96faad72cc534a94a08426600c0a59b1d530972964d
- name: registry.nxtgauge.com/nxtgauge-rust-employees
digest: sha256:f7fadb1da91199a891687439fc895dbac99b2f18e4ea0321d7964a14339b1dc3
- name: registry.nxtgauge.com/nxtgauge-rust-photographers
digest: sha256:33fda308731a6283005448d25c541b817d1b3f902b27a00f9ce34d5a5c360981
- name: registry.nxtgauge.com/nxtgauge-rust-makeup-artists
digest: sha256:06d448448ee4888ab4c4d3556ea779ccbe976eba86570c1f61baa9c984ede7a2
- name: registry.nxtgauge.com/nxtgauge-rust-tutors
digest: sha256:4af3a1a2e91f0b62ed43fbbe952c51f746e4f497054836504ae8ca4d1a53f974
- name: registry.nxtgauge.com/nxtgauge-rust-developers
digest: sha256:14b3de1f40ffef0155e0c1b4b45cc4479da52acb29a1c07cc13987aebab9faf7
- name: registry.nxtgauge.com/nxtgauge-rust-video-editors
newTag: d0b768d602b4d27bfd2363ef591f17c3e8f7bef1
digest: sha256:2f1ba4d769b1780ce67e5dc58307830b6f2cc7252a28b8c710fdef7832c3067a
- name: registry.nxtgauge.com/nxtgauge-rust-graphic-designers
digest: sha256:824609ac1d31e590802ecb05574e7cb84c3f0fb0074ffee1bce618735642e70b
- name: registry.nxtgauge.com/nxtgauge-rust-fitness-trainers
digest: sha256:3e12ba4e859e3e553231046c34886370fbe220ab5d1ac324123df429e491e09d
- name: registry.nxtgauge.com/nxtgauge-rust-catering-services
digest: sha256:5f16e93ff07f2b1989684a477fdc0b078a96e7ad222629624f3f3a0d80d22234
- name: registry.nxtgauge.com/nxtgauge-rust-ugc-content-creators
digest: sha256:3f1641a9e602623d5fb2db102ce50be33f23f575888fd02a8469e391d32cba52
- name: registry.nxtgauge.com/nxtgauge-rust-cron
digest: sha256:529f6767efe43a101b2aa5d3e2f52d4855387a626bcf77bc7f4e192d09b86ec1

View file

@ -1,5 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
namespace: nxtgauge
kind: Kustomization
resources:
- ../../base
@ -7,4 +6,4 @@ patchesStrategicMerge:
- replicas-patch.yaml
images:
- name: registry.nxtgauge.com/nxtgauge-frontend-solid
newTag: 4c61bca
newTag: f18c89d7fe73248a4cbe6a434dd243d88a25361b

View file

@ -1,6 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- retention-script.yaml
- retention-cronjob.yaml
namespace: registry

View file

@ -1,42 +0,0 @@
apiVersion: batch/v1
kind: CronJob
metadata:
name: registry-keep-last-3-builds
namespace: registry
spec:
schedule: "*/15 * * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 2
failedJobsHistoryLimit: 3
jobTemplate:
spec:
backoffLimit: 1
template:
spec:
serviceAccountName: registry-gc-runner
restartPolicy: Never
containers:
- name: prune
image: python:3.12-slim
command: ["sh", "-c"]
args:
- |
# Install kubectl
apt-get update && apt-get install -y curl --no-install-recommends && rm -rf /var/lib/apt/lists/*
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
# Run the prune script
python3 /scripts/prune.py
volumeMounts:
- name: script
mountPath: /scripts
- name: auth
mountPath: /auth
readOnly: true
volumes:
- name: script
configMap:
name: registry-retention-script
- name: auth
secret:
secretName: registry-regcred

View file

@ -1,181 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: registry-retention-script
namespace: registry
data:
prune.py: |
import base64, json, re, urllib.request, urllib.error
REG='https://registry.nxtgauge.com'
CFG='/auth/.dockerconfigjson'
PATTERN=re.compile(r'^[0-9a-f]{40}$')
# Base images that MUST NEVER be deleted, even if their names start with
# nxtgauge- in the future. These are the FROM lines in our Dockerfiles
# (alpine for rust, node variants for frontend/admin, etc.). If any of
# these are missing the entire build pipeline breaks.
BASE_IMAGES = {
'alpine', # runtime base image
'node', # frontend/admin builder
'rust', # backend builder
# Note: postgres/redis are in docker-compose (Docker Hub), not in registry
# busybox/golang/nginx are not used
}
# Project-image prefix that we DO prune. Anything outside this is sacred.
PROJECT_PREFIX = 'nxtgauge-'
with open(CFG,'r') as f:
dcfg=json.load(f)
auth=dcfg['auths']['registry.nxtgauge.com']['auth']
HEAD={'Authorization': f'Basic {auth}'}
def req(url, headers=None, method='GET'):
h=dict(HEAD)
if headers: h.update(headers)
r=urllib.request.Request(url, headers=h, method=method)
with urllib.request.urlopen(r, timeout=30) as resp:
return resp.status, dict(resp.headers), resp.read()
_, _, body = req(f'{REG}/v2/_catalog?n=1000')
all_repos=json.loads(body.decode()).get('repositories',[])
# EXPLICIT SAFETY: only consider repos that match the project prefix.
# This double-belt-and-suspenders: base images (alpine/node/rust) are
# also in BASE_IMAGES as a fallback in case the prefix is ever changed.
repos=[r for r in all_repos if r.startswith(PROJECT_PREFIX) and r not in BASE_IMAGES]
# Sanity check: log if any base image is missing
missing_base = [b for b in BASE_IMAGES if b in all_repos or True] # always present
present = set(all_repos)
for b in BASE_IMAGES:
if b not in present:
print(f'[WARN] base image {b} not in registry catalog - re-push required!')
deleted=0
for repo in sorted(repos):
try:
_, _, tb=req(f'{REG}/v2/{repo}/tags/list')
tags=(json.loads(tb.decode()).get('tags') or [])
except Exception as e:
print(f'[{repo}] tags/list failed: {e}')
continue
sha=[t for t in tags if PATTERN.match(t)]
if len(sha)<=1:
print(f'[{repo}] sha={len(sha)} no prune')
continue
rows=[]
for t in sha:
created='1970-01-01T00:00:00Z'
digest=None
try:
_, h, mb=req(f'{REG}/v2/{repo}/manifests/{t}', headers={'Accept':'application/vnd.docker.distribution.manifest.v2+json'})
digest=h.get('Docker-Content-Digest')
m=json.loads(mb.decode())
cfg=(m.get('config') or {}).get('digest')
if cfg:
_, _, cb=req(f'{REG}/v2/{repo}/blobs/{cfg}')
created=json.loads(cb.decode()).get('created', created)
except Exception:
created='9999-12-31T23:59:59Z'
rows.append((created, t, digest))
rows.sort(key=lambda x: x[0], reverse=True)
KEEP_N=2 # keep last 2 SHA builds (current + 1 previous)
keep_set=set(t for _, t, _ in rows[:KEEP_N])
# preserve buildcache for performance
keep_set.update(t for t in tags if t == 'buildcache')
keep_list=sorted(keep_set)
print(f'[{repo}] sha_total={len(rows)} keep={keep_list} remove={max(0, len(rows)-len(keep_set))}')
for _, t, d in rows:
if t in keep_set or not d:
continue
try:
req(f'{REG}/v2/{repo}/manifests/{d}', method='DELETE')
deleted+=1
print(f' deleted {repo}:{t}')
except urllib.error.HTTPError as e:
print(f' delete failed {repo}:{t} code={e.code}')
except Exception as e:
print(f' delete failed {repo}:{t} err={e}')
print(f'deleted_manifests={deleted}')
# Trigger garbage collection to delete unreferenced blob layers
if deleted > 0:
print('\n=== Triggering Garbage Collection ===')
try:
# Scale down registry to run GC
import subprocess
subprocess.run(['kubectl', 'scale', 'deployment', 'docker-registry', '--replicas=0', '-n', 'registry'], check=True)
print('Scaled down docker-registry deployment')
# Wait for deployment to be fully down
import time
time.sleep(5)
# Run GC job
gc_job = {
'apiVersion': 'batch/v1',
'kind': 'Job',
'metadata': {'name': 'registry-gc-once', 'namespace': 'registry'},
'spec': {
'backoffLimit': 0,
'template': {
'spec': {
'restartPolicy': 'Never',
'containers': [{
'name': 'gc',
'image': 'registry:3',
'command': ['registry', 'garbage-collect', '--delete-untagged', '/etc/distribution/config.yml'],
'volumeMounts': [
{'name': 'storage', 'mountPath': '/var/lib/registry'},
{'name': 'config', 'mountPath': '/etc/distribution'}
]
}],
'volumes': [
{'name': 'storage', 'persistentVolumeClaim': {'claimName': 'registry-pvc'}},
{'name': 'config', 'configMap': {'name': 'registry-config'}}
]
}
}
}
}
# Delete old GC job if exists
subprocess.run(['kubectl', 'delete', 'job', 'registry-gc-once', '-n', 'registry', '--ignore-not-found=true'], check=False)
time.sleep(2)
# Create and wait for GC job
import tempfile
with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
json.dump(gc_job, f)
f.flush()
subprocess.run(['kubectl', 'apply', '-f', f.name], check=True)
print('GC job created, waiting for completion...')
# Wait up to 10 minutes for GC to complete
for i in range(120):
result = subprocess.run(['kubectl', 'get', 'job', 'registry-gc-once', '-n', 'registry', '-o', 'jsonpath={.status.succeeded}'], capture_output=True, text=True)
if result.stdout.strip() == '1':
print('Garbage collection completed successfully')
break
result = subprocess.run(['kubectl', 'get', 'job', 'registry-gc-once', '-n', 'registry', '-o', 'jsonpath={.status.failed}'], capture_output=True, text=True)
if result.stdout.strip() == '1':
print('GC job failed')
break
time.sleep(5)
# Scale back up
subprocess.run(['kubectl', 'scale', 'deployment', 'docker-registry', '--replicas=1', '-n', 'registry'], check=True)
print('Scaled up docker-registry deployment')
except Exception as e:
print(f'GC trigger failed: {e}')
# Ensure registry is scaled back up even if GC failed
try:
subprocess.run(['kubectl', 'scale', 'deployment', 'docker-registry', '--replicas=1', '-n', 'registry'], check=False)
except:
pass

View file

@ -0,0 +1,21 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: coredns-nodehosts
namespace: argocd
spec:
destination:
namespace: kube-system
server: https://kubernetes.default.svc
project: default
source:
path: ops/coredns-nodehosts
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,25 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nxtgauge-admin-solid
namespace: argocd
annotations:
argocd-image-updater.argoproj.io/image-list: admin=registry.nxtgauge.com/nxtgauge-admin-solid:ed368607182cba2d57b2987849bf887a9714da53
argocd-image-updater.argoproj.io/admin.update-strategy: latest
argocd-image-updater.argoproj.io/admin.allow-tags: regexp:^ed36860
argocd-image-updater.argoproj.io/write-back-method: argocd
spec:
project: default
source:
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
path: apps/nxtgauge-admin-solid/overlays/prod
destination:
server: https://kubernetes.default.svc
namespace: nxtgauge
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,25 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nxtgauge-ai-assistant
namespace: argocd
annotations:
argocd-image-updater.argoproj.io/image-list: ai=registry.nxtgauge.com/nxtgauge-ai-assistant:high-performance-latest
argocd-image-updater.argoproj.io/ai.update-strategy: digest
argocd-image-updater.argoproj.io/ai.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/write-back-method: argocd
spec:
project: default
source:
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
path: apps/nxtgauge-ai-assistant/overlays/prod
destination:
server: https://kubernetes.default.svc
namespace: nxtgauge
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,81 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nxtgauge-backend-rust
namespace: argocd
annotations:
argocd-image-updater.argoproj.io/image-list: >-
gateway=registry.nxtgauge.com/nxtgauge-rust-gateway:09075087f07ef70dc13281944c1f2ce4bf7c2a29,
users=registry.nxtgauge.com/nxtgauge-rust-users:high-performance-latest,
companies=registry.nxtgauge.com/nxtgauge-rust-companies:high-performance-latest,
job-seekers=registry.nxtgauge.com/nxtgauge-rust-job-seekers:high-performance-latest,
customers=registry.nxtgauge.com/nxtgauge-rust-customers:high-performance-latest,
payments=registry.nxtgauge.com/nxtgauge-rust-payments:high-performance-latest,
employees=registry.nxtgauge.com/nxtgauge-rust-employees:high-performance-latest,
photographers=registry.nxtgauge.com/nxtgauge-rust-photographers:high-performance-latest,
makeup-artists=registry.nxtgauge.com/nxtgauge-rust-makeup-artists:high-performance-latest,
tutors=registry.nxtgauge.com/nxtgauge-rust-tutors:high-performance-latest,
developers=registry.nxtgauge.com/nxtgauge-rust-developers:high-performance-latest,
video-editors=registry.nxtgauge.com/nxtgauge-rust-video-editors:high-performance-latest,
graphic-designers=registry.nxtgauge.com/nxtgauge-rust-graphic-designers:high-performance-latest,
social-media-managers=registry.nxtgauge.com/nxtgauge-rust-social-media-managers:high-performance-latest,
fitness-trainers=registry.nxtgauge.com/nxtgauge-rust-fitness-trainers:high-performance-latest,
catering-services=registry.nxtgauge.com/nxtgauge-rust-catering-services:high-performance-latest,
ugc-content-creators=registry.nxtgauge.com/nxtgauge-rust-ugc-content-creators:high-performance-latest,
cron=registry.nxtgauge.com/nxtgauge-rust-cron:high-performance-latest
argocd-image-updater.argoproj.io/gateway.update-strategy: latest
argocd-image-updater.argoproj.io/gateway.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/users.update-strategy: latest
argocd-image-updater.argoproj.io/users.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/companies.update-strategy: digest
argocd-image-updater.argoproj.io/companies.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/jobs.update-strategy: digest
argocd-image-updater.argoproj.io/jobs.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/leads.update-strategy: digest
argocd-image-updater.argoproj.io/leads.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/job-seekers.update-strategy: digest
argocd-image-updater.argoproj.io/job-seekers.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/customers.update-strategy: digest
argocd-image-updater.argoproj.io/customers.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/payments.update-strategy: digest
argocd-image-updater.argoproj.io/payments.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/employees.update-strategy: digest
argocd-image-updater.argoproj.io/employees.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/photographers.update-strategy: digest
argocd-image-updater.argoproj.io/photographers.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/makeup-artists.update-strategy: digest
argocd-image-updater.argoproj.io/makeup-artists.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/tutors.update-strategy: digest
argocd-image-updater.argoproj.io/tutors.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/developers.update-strategy: digest
argocd-image-updater.argoproj.io/developers.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/video-editors.update-strategy: digest
argocd-image-updater.argoproj.io/video-editors.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/graphic-designers.update-strategy: digest
argocd-image-updater.argoproj.io/graphic-designers.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/social-media-managers.update-strategy: digest
argocd-image-updater.argoproj.io/social-media-managers.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/fitness-trainers.update-strategy: digest
argocd-image-updater.argoproj.io/fitness-trainers.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/catering-services.update-strategy: digest
argocd-image-updater.argoproj.io/catering-services.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/ugc-content-creators.update-strategy: digest
argocd-image-updater.argoproj.io/ugc-content-creators.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/cron.update-strategy: digest
argocd-image-updater.argoproj.io/cron.allow-tags: regexp:^high-performance-latest$
argocd-image-updater.argoproj.io/write-back-method: argocd
spec:
destination:
namespace: nxtgauge
server: https://kubernetes.default.svc
project: default
source:
path: apps/nxtgauge-backend-rust/overlays/prod
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,25 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nxtgauge-frontend-solid
namespace: argocd
annotations:
argocd-image-updater.argoproj.io/image-list: frontend=registry.nxtgauge.com/nxtgauge-frontend-solid:f18c89d7fe73248a4cbe6a434dd243d88a25361b
argocd-image-updater.argoproj.io/frontend.update-strategy: latest
argocd-image-updater.argoproj.io/frontend.allow-tags: regexp:^f18c89d
argocd-image-updater.argoproj.io/write-back-method: argocd
spec:
project: default
source:
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
path: apps/nxtgauge-frontend-solid/overlays/prod
destination:
server: https://kubernetes.default.svc
namespace: nxtgauge
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,20 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: ollama
namespace: argocd
spec:
destination:
namespace: nxtgauge-ai
server: https://kubernetes.default.svc
project: default
source:
path: apps/ollama/base
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,21 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: openobserve-alerts
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
path: ops/openobserve-alerts
destination:
server: https://kubernetes.default.svc
namespace: openobserve
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,21 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: openobserve-otelcol
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
path: ops/openobserve-otelcol
destination:
server: https://kubernetes.default.svc
namespace: openobserve
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,21 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: registry-ingress
namespace: argocd
spec:
destination:
namespace: registry
server: https://kubernetes.default.svc
project: default
source:
path: ops/registry-ingress
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -0,0 +1,21 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: woodpecker-registry-pull
namespace: argocd
spec:
destination:
namespace: woodpecker
server: https://kubernetes.default.svc
project: default
source:
path: ops/woodpecker-registry-pull
repoURL: https://github.com/Traceworks2023/nxtgauge-gitops.git
targetRevision: main
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true

View file

@ -1,10 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../apps/nxtgauge-backend-rust/overlays/prod
- ../../apps/nxtgauge-frontend-solid/overlays/prod
- ../../apps/nxtgauge-admin-solid/overlays/prod
- ../../apps/nxtgauge-ai-assistant/overlays/prod
- ../../apps/ollama/base
- ../../apps/registry
- ../../ops/openobserve-alerts

View file

@ -1,143 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: openobserve-alerts-additional
namespace: openobserve
data:
additional-alerts.sh: |
#!/usr/bin/env sh
set -eu
ORG_ID="${ORG_ID:-default}"
BASE_URL="${BASE_URL:-http://o2-openobserve-standalone.openobserve.svc.cluster.local:5080}"
STREAM_NAME="${STREAM_NAME:-default}"
TELEGRAM_CHAT_ID="${TELEGRAM_CHAT_ID:-}"
AUTH="$(printf '%s:%s' "$ZO_ROOT_USER_EMAIL" "$ZO_ROOT_USER_PASSWORD" | base64 | tr -d '\n')"
auth_hdr="Authorization: Basic $AUTH"
api() {
curl -sfS -H "$auth_hdr" -H "Content-Type: application/json" "$@"
}
ensure_alert() {
alert_name="$1"
sql="$2"
period_minutes="$3"
frequency_minutes="$4"
silence_minutes="$5"
row_template="$6"
existing_id="$(
api "$BASE_URL/api/v2/$ORG_ID/alerts" \
| jq -r --arg n "$alert_name" '.list[] | select(.name == $n) | .alert_id' \
| head -n 1
)"
payload="$(jq -n \
--arg name "$alert_name" \
--arg stream "$STREAM_NAME" \
--arg sql "$sql" \
--argjson period "$period_minutes" \
--argjson frequency "$frequency_minutes" \
--argjson silence "$silence_minutes" \
--arg row_template "$row_template" \
'{
name: $name,
stream_type: "logs",
stream_name: $stream,
is_real_time: false,
enabled: true,
tz_offset: 330,
destinations: ["nxtgauge_telegram"],
row_template: $row_template,
row_template_type: "String",
query_condition: { type: "sql", sql: $sql },
trigger_condition: {
period: $period,
operator: ">=",
threshold: 1,
frequency: $frequency,
frequency_type: "minutes",
silence: $silence
}
}')"
if [ -n "$existing_id" ] && [ "$existing_id" != "null" ]; then
api -X PUT "$BASE_URL/api/v2/$ORG_ID/alerts/$existing_id" -d "$payload" >/dev/null
echo "updated alert=$alert_name"
else
api -X POST "$BASE_URL/api/v2/$ORG_ID/alerts" -d "$payload" >/dev/null
echo "created alert=$alert_name"
fi
}
# API Health
ensure_alert \
"api-health-failures" \
"SELECT service, endpoint, status_code, COUNT(*) as count FROM \"default\" WHERE service ILIKE '%api%' AND (status_code >= 500 OR status_code = 0) GROUP BY service, endpoint, status_code ORDER BY count DESC LIMIT 50" \
5 1 15 \
"{service}/{endpoint} status={status_code} count={count}"
# Database Health
ensure_alert \
"database-connection-failures" \
"SELECT k8s_namespace_name, k8s_pod_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE body ILIKE '%connection refused%' OR body ILIKE '%database%' OR body ILIKE '%postgres%' OR body ILIKE '%sqlx%' ORDER BY _timestamp DESC LIMIT 50" \
5 1 15 \
"{k8s_namespace_name}/{k8s_pod_name}: {msg}"
# Redis Health
ensure_alert \
"redis-connection-failures" \
"SELECT k8s_namespace_name, k8s_pod_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE body ILIKE '%redis%' OR body ILIKE '%cache%' OR body ILIKE '%connection timeout%' ORDER BY _timestamp DESC LIMIT 50" \
5 1 15 \
"{k8s_namespace_name}/{k8s_pod_name}: {msg}"
# Pod Failures
ensure_alert \
"pod-failures" \
"SELECT k8s_namespace_name, k8s_pod_name, body_object_reason, body_object_message FROM \"default\" WHERE body_object_reason IN ('Failed', 'Evicted', 'NodeAffinity', 'UnexpectedAdmissionError') ORDER BY _timestamp DESC LIMIT 50" \
5 1 15 \
"{k8s_namespace_name}/{k8s_pod_name} {body_object_reason}: {body_object_message}"
# CPU High
ensure_alert \
"cpu-high-usage" \
"SELECT k8s_namespace_name, k8s_pod_name, k8s_container_name, AVG(cpu_usage_cores) as avg_cpu FROM \"default\" WHERE cpu_usage_cores > 0.8 GROUP BY k8s_namespace_name, k8s_pod_name, k8s_container_name ORDER BY avg_cpu DESC LIMIT 50" \
10 2 30 \
"{k8s_namespace_name}/{k8s_pod_name}/{k8s_container_name} CPU={avg_cpu}"
# Memory High
ensure_alert \
"memory-high-usage" \
"SELECT k8s_namespace_name, k8s_pod_name, k8s_container_name, AVG(memory_usage_bytes) as avg_mem FROM \"default\" WHERE memory_usage_bytes > 1073741824 GROUP BY k8s_namespace_name, k8s_pod_name, k8s_container_name ORDER BY avg_mem DESC LIMIT 50" \
10 2 30 \
"{k8s_namespace_name}/{k8s_pod_name}/{k8s_container_name} MEM={avg_mem}"
# Disk Full
ensure_alert \
"disk-full-warning" \
"SELECT k8s_node_name, k8s_namespace_name, k8s_pod_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE body ILIKE '%disk full%' OR body ILIKE '%no space left%' OR body ILIKE '%DiskPressure%' ORDER BY _timestamp DESC LIMIT 50" \
10 2 60 \
"{k8s_node_name}/{k8s_namespace_name}/{k8s_pod_name}: {msg}"
# Longhorn Health
ensure_alert \
"longhorn-volume-errors" \
"SELECT k8s_namespace_name, k8s_pod_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE k8s_namespace_name = 'longhorn-system' AND (body ILIKE '%error%' OR body ILIKE '%degraded%' OR body ILIKE '%faulted%') ORDER BY _timestamp DESC LIMIT 50" \
10 2 30 \
"longhorn/{k8s_pod_name}: {msg}"
# Flux Health
ensure_alert \
"flux-reconcile-failures" \
"SELECT k8s_pod_name, k8s_container_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE k8s_namespace_name = 'flux-system' AND (body ILIKE '%SyncFailed%' OR body ILIKE '%ComparisonError%' OR body ILIKE '%ResourceQuota%') ORDER BY _timestamp DESC LIMIT 50" \
10 2 30 \
"flux/{k8s_pod_name}: {msg}"
# Registry Health
ensure_alert \
"registry-push-failures" \
"SELECT k8s_pod_name, k8s_container_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE k8s_namespace_name = 'registry' AND (body ILIKE '%413%' OR body ILIKE '%payload too large%' OR body ILIKE '%unauthorized%') ORDER BY _timestamp DESC LIMIT 50" \
10 2 60 \
"registry/{k8s_pod_name}: {msg}"

View file

@ -82,9 +82,9 @@ data:
stream_type: "logs",
stream_name: $stream,
is_real_time: false,
enabled: false,
enabled: true,
tz_offset: 330,
destinations: [],
destinations: ["nxtgauge_telegram"],
row_template: $row_template,
row_template_type: "String",
query_condition: { type: "sql", sql: $sql },
@ -148,16 +148,16 @@ data:
"{k8s_namespace_name}/{k8s_pod_name} {body_object_reason}: {body_object_message}"
ensure_alert \
"flux-errors" \
"SELECT k8s_pod_name, k8s_container_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE k8s_namespace_name = 'flux-system' AND (body ILIKE '%level=error%' OR body ILIKE '%ERROR%' OR body ILIKE '%ComparisonError%' OR body ILIKE '%SyncFailed%') ORDER BY _timestamp DESC LIMIT 50" \
"argocd-errors" \
"SELECT k8s_pod_name, k8s_container_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE k8s_namespace_name = 'argocd' AND (body ILIKE '%level=error%' OR body ILIKE '%ERROR%' OR body ILIKE '%ComparisonError%' OR body ILIKE '%SyncFailed%') ORDER BY _timestamp DESC LIMIT 50" \
10 2 30 \
"flux/{k8s_pod_name} {k8s_container_name}: {msg}"
"argocd/{k8s_pod_name} {k8s_container_name}: {msg}"
ensure_alert \
"forgejo-runner-errors" \
"SELECT k8s_pod_name, k8s_container_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE k8s_namespace_name = 'forgejo' AND (body ILIKE '%error%' OR body ILIKE '%ERROR%' OR body ILIKE '%failed%' OR body ILIKE '%job failed%') ORDER BY _timestamp DESC LIMIT 50" \
"woodpecker-errors" \
"SELECT k8s_pod_name, k8s_container_name, substring(body, 1, 220) AS msg FROM \"default\" WHERE k8s_namespace_name = 'woodpecker' AND (body ILIKE '%error%' OR body ILIKE '%ERROR%' OR body ILIKE '%failed%') ORDER BY _timestamp DESC LIMIT 50" \
10 2 30 \
"forgejo/{k8s_pod_name} {k8s_container_name}: {msg}"
"woodpecker/{k8s_pod_name} {k8s_container_name}: {msg}"
ensure_alert \
"registry-errors" \

View file

@ -14,7 +14,7 @@ spec:
restartPolicy: Never
containers:
- name: bootstrap
image: docker:28-cli
image: registry.nxtgauge.com/docker:28-cli
command: ["sh", "-lc"]
args:
- apk add --no-cache curl jq >/dev/null && /scripts/bootstrap.sh

View file

@ -0,0 +1,35 @@
# Woodpecker: allow pulling from private registry
Woodpecker pipelines run as Kubernetes pods in the `woodpecker` namespace. If pipeline step images use `registry.nxtgauge.com/...` (private, Basic auth), kubelet needs an `imagePullSecret`.
This is required for base images (example `registry.nxtgauge.com/rust:alpine`) and also for any mirrored plugin images (example `registry.nxtgauge.com/kaniko:2.1.1`).
## Required secret
Create this once:
```bash
kubectl -n woodpecker create secret docker-registry registry-nxtgauge-pull \
--docker-server=registry.nxtgauge.com \
--docker-username="<REGISTRY_USERNAME>" \
--docker-password="<REGISTRY_PASSWORD>" \
--docker-email="ci@nxtgauge.com"
```
## Mirroring common plugin images (optional)
If your pipelines reference plugin images from the internal registry (example `registry.nxtgauge.com/kaniko:2.1.1`) make sure those images exist in the registry.
Example mirror from Docker Hub to internal:
```bash
docker pull woodpeckerci/plugin-kaniko:2.1.1
docker tag woodpeckerci/plugin-kaniko:2.1.1 registry.nxtgauge.com/kaniko:2.1.1
docker push registry.nxtgauge.com/kaniko:2.1.1
```
## What this kustomize applies
It patches/ensures the `default` ServiceAccount in `woodpecker` includes:
- `imagePullSecrets: [registry-nxtgauge-pull]`

View file

@ -1,7 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- secret.yaml
- deployment.yaml
- service.yaml
- serviceaccount-default.yaml

View file

@ -0,0 +1,8 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: default
namespace: woodpecker
imagePullSecrets:
- name: registry-nxtgauge-pull