Operations & Deployment

This guide covers running Qadra in production: environment configuration, deployment, the CI/CD pipeline, backups, scaling, health checks, and the protocol for making infrastructure changes. For local development setup, see the Development Guide.

Environment Variables

Configuration is supplied via .env or directly through docker-compose.

Required

DATABASE_URL=postgres://user:pass@localhost/qadra   # PostgreSQL (source of truth)
REDIS_URL=redis://localhost:6379                    # Streams, cache, pub/sub
OPENROUTER_API_KEY=sk-or-v1-...                      # LLM inference

Optional

MONGODB_URL=mongodb://...                  # Audit traces (degrades gracefully if absent)
CEREBRAS_API_KEY=csk-...                   # Nitro mode (fast inference)
PORT=3000                                  # Server port (default: 3000)
HOST=0.0.0.0                               # Bind address
RUST_LOG=info                              # Log level (debug, info, warn, error)
QUICKJS_POOL_SIZE=100                      # JS runtime pool size
REDIS_POOL_SIZE=10                         # Redis connection pool size

Platform Services

POSTGRES_PASSWORD=qadra_dev_password
MONGO_PASSWORD=qadra_traces_dev
GRAFANA_PASSWORD=qadra-grafana
JWT_SECRET=qadra-dev-secret-change-in-production
SMTP2GO_API_KEY=api-...                    # Email delivery (Rust Core)
SMTP2GO_SENDER=noreply@qadra.dev           # From address (default: noreply@qadra.dev)
EMAIL_VERIFICATION_URL=http://localhost:5173/verify-email  # Verification link base URL
MINIO_ROOT_USER=qadra                      # MinIO access key (Gateway)
MINIO_ROOT_PASSWORD=qadra_minio_dev        # MinIO secret key (Gateway)
MINIO_ENDPOINT=http://localhost:9000       # MinIO S3 API endpoint
MINIO_BUCKET=qadra-files                   # Default bucket name
IMAGINARY_URL=http://localhost:9002        # Imaginary image processing service

Production note: JWT_SECRET and all default passwords MUST be replaced before deploying. Credentials currently live in docker-compose.yml for local development only (tech debt TD-004) — use Docker secrets or a vault in production.

Docker Deployment

The full stack runs via docker compose:

# Build the Rust Core image
docker build -t qadra .

# Run the full stack
docker compose up -d

# Restart a service cleanly (use force-recreate on WSL to avoid OCI errors)
docker compose up -d --force-recreate qadra-core

Containers are published to GitHub Container Registry on every push to main:

All services share the qadra-network Docker network; internal discovery uses container names (e.g. postgres:5432, loki:3100). External traffic enters through Kong, which validates Qadra-issued JWTs and injects the X-User-Email, X-Tenant-Id, and Authorization headers.

Running Migrations

Migrations run with sqlx against the target database before the service starts:

DATABASE_URL=postgres://... sqlx migrate run

# Generate offline query data (required for CI builds without a DB)
cargo sqlx prepare

CI/CD Pipeline

GitHub Actions at .github/workflows/ci.yml runs on push to main/develop and on pull requests targeting main. Duplicate runs on the same branch are cancelled automatically (cancel-in-progress: true).

Jobs

Path Gating

A leading changes job inspects the diff and emits per-area flags. Downstream jobs run only when their paths changed:

• web / cypress-* — gated on changes under web/
• rust-* / test-integration / coverage-rust — gated on Rust sources, crates/, migrations/, Cargo.*
• gateway — gated on gateway/
• agent — gated on agent/

Two jobs run only on push to main, never on PRs or feature branches:

• build-rust — the release binary build
• docker — multi-image build, scan, sign, and push to GHCR

Skipped jobs report success, so ci-pass treats a gated-out job as passing. This keeps PR feedback fast — a docs-only or web-only change does not trigger a full Rust compile.

Branch Protection

Set the single required status check to CI Report (the ci-pass aggregate job). It gates on all upstream jobs and renders a unified report to the workflow run summary covering security scans, code quality, test results, build artifact sizes, and (on main) Docker image tags, digests, and signing status.

Supply-Chain Security

A scheduled workflow (.github/workflows/security.yml, Monday 06:00 UTC) runs Gitleaks full-history, cargo audit, and cargo deny.

Health Checks

Both the Rust Core and the Gateway expose liveness and readiness endpoints:

GET /health        # Basic liveness
GET /health/ready  # Full readiness — Rust Core: DB + services; Gateway: NATS connectivity

Use /health for orchestrator liveness probes and /health/ready for readiness gating before routing traffic to a new instance.

Scaling

The Rust Core (qadra-core) is stateless and horizontally scalable — all authoritative state lives in PostgreSQL, with Redis as the hot layer. Scale out by running more replicas behind Kong.

Constraints that make this safe:

• All database access is scoped by tenant_id (row-level isolation).
• Writes go to PostgreSQL first, then publish events to Redis Streams.
• Event consumers are idempotent (at-least-once delivery).
• QuickJS instances are stateless and isolated.

Per-container memory limits are configured in docker-compose.yml:

Performance Targets

Backups

Current gap (tech debt TD-005): There is no automated backup strategy yet. The platform is in an early development phase. Do not treat the running database as durable.

Planned resolution: a pg_dump cron job shipping snapshots to S3, to be implemented before production launch. MongoDB audit traces use TTL-based retention (default 90 days) and are not currently backed up — they are diagnostic, not authoritative.

Infrastructure Change Protocol

When modifying files in deploy/, docker-compose.yml, kong/, or CI/CD workflows, you MUST update the corresponding rules documentation. Stale docs cause hallucinated assumptions.

Required Documentation Updates

• Adding/modifying a Docker service — update the service and resource-limit tables in .claude/rules/infrastructure.md.
• New observability component — update the observability config section, document any new Grafana dashboard, add new alert rules to the alerts table.
• Changing environment variables — update the Environment Variables section in .claude/rules/infrastructure.md.
• Modifying alerting — update the Alert Rules table; document threshold changes and rationale in the commit message.
• Kong/Gateway changes — document route changes and JWT/header injection in .claude/rules/infrastructure.md.

Commit Convention

Tag infrastructure changes with [infra]:

[infra] Add cAdvisor for container metrics

- Added cadvisor service to docker-compose
- Created container-monitoring dashboard
- Added prometheus scrape target

Validation Checklist

Before committing infrastructure changes:

• docker compose config passes (syntax validation)
• Services start cleanly with docker compose up -d
• Grafana loads without provisioning errors
• Prometheus targets show as "up" at http://localhost:9090/targets
• Rules documentation updated in .claude/rules/infrastructure.md