Development Guide

This guide covers local development setup, common tasks, and patterns for contributing to Qadra.

Prerequisites

Quick version check:

rustc --version    # rustc 1.75.0 or higher
docker --version   # Docker 24.x or higher
node --version     # v18.x or higher
yarn --version     # 1.x or higher
python --version   # 3.11 or higher

Local Setup

Clone and Configure

git clone https://github.com/HGP-Technologies/qadra-ai.git
cd qadra-ai

# Copy environment template
cp .env.example .env

# Edit configuration (set API keys, passwords)
vim .env

Start Infrastructure

# Start all infrastructure services
docker compose up -d postgres mongodb nats minio imaginary

# Wait for PostgreSQL to be ready
docker compose exec postgres pg_isready -U qadra

Run Migrations

DATABASE_URL=postgres://qadra:qadra_dev_password@localhost:5432/qadra \
  sqlx migrate run

Start Development Servers

Three application processes run independently during development:

# Terminal 1: Rust Core (NATS handlers, truth layer)
cargo run

# Terminal 2: Node.js Gateway (HTTP API)
cd gateway && yarn install && yarn dev

# Terminal 3: React SPA (frontend)
cd web && yarn install && yarn dev

The React SPA runs at http://localhost:5173, the Gateway at http://localhost:4000, and Rust Core connects to NATS internally.

Project Structure

qadra-ai/
├── Cargo.toml                 # Workspace manifest (10 crates)
├── crates/                    # Library crates
│   ├── qadra-traits/          # ALL interfaces and shared types (tiny, stable)
│   ├── qadra-container/       # IoC container, mock implementations
│   ├── qadra-db-postgres/     # Repository: PostgreSQL
│   ├── qadra-llm-openrouter/  # LLM: OpenRouter (OpenAI-compatible)
│   ├── qadra-cache-redis/     # Cache: Redis
│   ├── qadra-queue-redis/     # Queue: Redis Streams
│   ├── qadra-vector-pgvector/ # Vector: pgvector
│   ├── qadra-semantic-cache-redis/ # Semantic cache for verification results
│   ├── qadra-epistemic-core/  # Epistemic operations (KG queries, attribution)
│   └── qadra-nats/            # NATS client wrapper
├── src/                       # Main Rust application
│   ├── main.rs                # Entry point (8 NATS handler spawns)
│   ├── lib.rs                 # Library exports
│   ├── config.rs              # Configuration loading
│   ├── nats_service.rs        # NATS handler dispatch
│   ├── nats_observer.rs       # Catch-all NATS audit observer
│   ├── flow_executor.rs       # Flow graph execution engine
│   ├── flow_generator.rs      # Flow graph generation
│   ├── email_service.rs       # SMTP2GO email delivery
│   ├── email_renderer.rs      # Handlebars template rendering
│   ├── email_logging.rs       # LoggingEmailService decorator
│   ├── error.rs               # Error types
│   ├── ingestion/             # Document processing pipeline
│   │   ├── mod.rs
│   │   ├── analyzer.rs        # Content routing (SPO/Vector/Both/Discard)
│   │   ├── chunker.rs         # Text chunking
│   │   ├── embedder.rs        # Embedding generation
│   │   ├── extractor.rs       # LLM-based triple extraction
│   │   └── service.rs         # Orchestrates the ingestion pipeline
│   ├── tracing/               # Audit trace to MongoDB
│   └── telemetry/             # OpenTelemetry integration
├── gateway/                   # Node.js HTTP gateway
│   ├── src/
│   │   ├── index.ts           # Express server entry point
│   │   ├── nats.ts            # NATS client and request helper
│   │   ├── config.ts          # Configuration
│   │   ├── storage.ts         # MinIO S3 client
│   │   ├── images.ts          # Imaginary image processing client
│   │   ├── auth/              # JWT middleware
│   │   ├── routes/            # HTTP route handlers (15 route files)
│   │   │   ├── auth.ts        # Login, register, refresh, verify
│   │   │   ├── users.ts       # Profile, avatar
│   │   │   ├── teams.ts       # Members, invites, roles
│   │   │   ├── tenants.ts     # Tenant settings, branding
│   │   │   ├── agents.ts      # Agent CRUD
│   │   │   ├── flows.ts       # Flow CRUD and execution
│   │   │   ├── conversations.ts # Chat conversations and messages
│   │   │   ├── knowledge.ts   # Document ingestion and proposals
│   │   │   ├── artifactTemplates.ts # Artifact template CRUD
│   │   │   ├── files.ts       # File upload/download
│   │   │   ├── email.ts       # Email send
│   │   │   ├── notifications.ts # Email templates, notification logs
│   │   │   ├── audit.ts       # NATS audit trail
│   │   │   ├── admin.ts       # Super admin operations
│   │   │   └── health.ts      # Health checks
│   │   └── types/             # TypeScript type definitions
│   ├── package.json
│   └── tsconfig.json
├── web/                       # React SPA (Vite + Tailwind + zustand)
│   ├── src/
│   │   ├── App.tsx            # Router and layout
│   │   ├── main.tsx           # Entry point
│   │   ├── api/               # API client functions
│   │   ├── stores/            # zustand state stores
│   │   ├── pages/             # Page components
│   │   │   ├── Dashboard.tsx
│   │   │   ├── Conversations.tsx
│   │   │   ├── ConversationChat.tsx
│   │   │   ├── Agents.tsx
│   │   │   ├── AgentEditor.tsx
│   │   │   ├── Flows.tsx
│   │   │   ├── FlowBuilder.tsx
│   │   │   ├── ArtifactTemplates.tsx
│   │   │   ├── ArtifactTemplateEditor.tsx
│   │   │   ├── Settings.tsx
│   │   │   ├── Admin.tsx
│   │   │   ├── Login.tsx
│   │   │   ├── Register.tsx
│   │   │   └── ...
│   │   └── components/        # Reusable components
│   │       └── flow/nodes/    # ReactFlow node type components
│   ├── package.json
│   ├── vite.config.ts
│   └── tailwind.config.ts
├── agent/                     # Python orchestration agent
│   ├── src/qadra_agent/
│   │   └── orchestrator.py    # WorkloadOrchestrator (LLM-driven pipelines)
│   ├── tests/
│   ├── pyproject.toml
│   └── Dockerfile
├── migrations/                # SQL migrations (001-030)
├── tests/                     # Rust integration tests
│   ├── common/                # TestClient, TestDb, fixtures
│   ├── workload_lifecycle.rs
│   ├── task_lifecycle.rs
│   ├── stage_forwarding.rs
│   ├── multi_stage_forwarding.rs
│   └── artifact_pipeline.rs
├── deploy/otel/               # Observability configuration
├── kong/                      # Kong API gateway config
├── docs/                      # Documentation
└── docker-compose.yml         # Full development stack

Crate Workspace

The Rust code is organized into 10 workspace crates plus the root binary:

Design principle: Everything is swappable. Core business logic depends only on traits, not implementations. Switch PostgreSQL for SQLite, swap OpenRouter for Ollama, replace Redis with in-memory---all via configuration.

Common Tasks

Running Tests

# Unit tests (all workspace crates, ~120 tests)
cargo test --workspace --lib

# Integration tests (requires running PostgreSQL with pgvector)
DATABASE_URL="postgres://qadra:qadra_dev_password@localhost:5432/qadra" \
  cargo test --test workload_lifecycle -- --test-threads=1

# All integration test files
DATABASE_URL="postgres://qadra:qadra_dev_password@localhost:5432/qadra" \
  cargo test -- --test-threads=1

# Specific test by name
cargo test test_name

# With output
cargo test -- --nocapture

# Web tests (if test files exist)
cd web && yarn test

# Gateway type checking
cd gateway && yarn build

Linting

# Rust formatting
cargo fmt
cargo fmt --check

# Rust linting
cargo clippy -- -D warnings

# Web linting (Biome)
cd web && npx @biomejs/biome check src/

# Web type checking
cd web && npx tsc --noEmit

Building

# Debug build
cargo build

# Release build
cargo build --release

# Check without building (faster)
cargo check

# Gateway build
cd gateway && yarn build

# Web build
cd web && yarn build

Database Operations

# Connect to database
psql postgres://qadra:qadra_dev_password@localhost:5432/qadra

# Run migrations
DATABASE_URL=postgres://... sqlx migrate run

# Create new migration
sqlx migrate add migration_name

# Revert last migration
sqlx migrate revert

# Prepare offline query data (for CI)
cargo sqlx prepare

Adding New Features

Standard Feature Flow

The typical flow for adding a new feature:

1. Migration -- Create SQL migration for any new tables or columns
2. Traits types -- Add types and trait methods in qadra-traits
3. PostgreSQL impl -- Implement trait methods in qadra-db-postgres
4. Container mock stubs -- Add default return values in qadra-container
5. NATS handler -- Add handler case in src/nats_service.rs
6. Gateway route -- Add HTTP route in gateway/src/routes/
7. Frontend -- Add page/store in web/src/
8. Rule docs -- Update .claude/rules/ files

Adding a New NATS Handler

1. Define the operation in src/nats_service.rs:

#![allow(unused)]
fn main() {
"your_operation" => {
    // Parse request, call repository, return response
}
}

2. Add gateway route in gateway/src/routes/yourFeature.ts:

router.get("/your-endpoint", requireAuth, async (req, res) => {
    const result = await natsRequest(subject, payload);
    res.json(result);
});

3. Register route in gateway/src/index.ts

Adding a New Database Table

1. Create migration:

sqlx migrate add my_table

2. Write SQL in migrations/XXX_my_table.sql:

CREATE TABLE my_table (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
    -- columns
    created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX idx_my_table_tenant ON my_table(tenant_id);

3. Add trait in crates/qadra-traits/src/lib.rs
4. Implement in crates/qadra-db-postgres/src/lib.rs
5. Add mock stub in crates/qadra-container/src/lib.rs

Code Style

Naming Conventions

Error Handling

Use the project's error types:

#![allow(unused)]
fn main() {
use crate::{Error, Result};

fn my_function() -> Result<()> {
    something_that_might_fail()?;
    Ok(())
}
}

Logging

Use tracing macros with structured fields:

#![allow(unused)]
fn main() {
use tracing::{info, warn, error, debug, instrument};

#[instrument(skip(self))]
async fn my_function(&self, id: Uuid) -> Result<()> {
    info!(id = %id, "Processing item");
    Ok(())
}
}

Testing Patterns

Unit Tests

Unit tests live at the bottom of source files (Rust convention):

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_my_function() {
        let result = my_function(input);
        assert_eq!(result, expected);
    }

    #[tokio::test]
    async fn test_async_function() {
        let result = async_function().await;
        assert!(result.is_ok());
    }
}
}

Integration Tests

Integration tests are in tests/ and require a running PostgreSQL instance:

#![allow(unused)]
fn main() {
#[tokio::test]
async fn test_workload_lifecycle() {
    let container = setup_test_container().await;
    let workload = container.workload_repository()
        .create_workload(params)
        .await
        .unwrap();
    assert_eq!(workload.status, "pending");
}
}

Test Database

The TestDb helper resolves the test database URL:

1. TEST_DATABASE_URL -- Explicit test database URL (if set)
2. DATABASE_URL -- Derives test URL by replacing database name with qadra_test
3. Default -- postgres://qadra:qadra_dev_password@localhost:5433/qadra_test

The helper auto-creates the qadra_test database if it does not exist.

Mocking

Use mock implementations from qadra-container:

#![allow(unused)]
fn main() {
use qadra_container::ContainerBuilder;

let container = ContainerBuilder::new()
    .with_mock_repository()
    .with_mock_llm()
    .build();
}

Mock implementations return empty/default values.

Git Workflow

Commit Configuration

All commits must be signed as:

• Name: Mark Scott
• Email: mark.scott@hgp-technologies.com

Use Co-Authored-By: Mark Scott <mark.scott@hgp-technologies.com> for co-authored commits.

Branch Strategy

• main -- Production-ready code
• develop -- Integration branch
• feat/* -- Feature branches
• fix/* -- Bug fix branches

CI Pipeline

GitHub Actions at .github/workflows/ci.yml runs on push to main/develop and PRs targeting main. Key jobs:

Common Pitfalls

Forgetting tenant_id

Every repository method takes tenant_id. Every query includes it. No exceptions.

#![allow(unused)]
fn main() {
// WRONG
sqlx::query!("SELECT * FROM workloads WHERE id = $1", id)

// RIGHT
sqlx::query!("SELECT * FROM workloads WHERE tenant_id = $1 AND id = $2",
             tenant_id, id)
}

Blocking the async runtime

Never call blocking functions directly. Use tokio::spawn_blocking for unavoidable blocking work.

#![allow(unused)]
fn main() {
// WRONG
let result = std::fs::read_to_string("file.txt")?;

// RIGHT
let result = tokio::fs::read_to_string("file.txt").await?;
}

DbWorkload/DbTask/DbArtifact do not derive Clone

Access fields directly instead of .clone().into().

Container repository access patterns

#![allow(unused)]
fn main() {
// Repository (agents, graphs, SPO)
container.repository()

// Auth-specific operations
container.auth()

// Workload operations
container.workload_repository()

// Agent flow operations
container.agent_flow_repository()
}

Debugging

Enabling Debug Logs

# Rust Core logs
RUST_LOG=qadra=debug cargo run

# Specific module
RUST_LOG=qadra::flow_executor=debug cargo run

# SQL queries
RUST_LOG=sqlx=debug cargo run

# Gateway logs
cd gateway && DEBUG=* yarn dev

Inspecting Database State

psql postgres://qadra:qadra_dev_password@localhost:5432/qadra

\dt                           # List tables
SELECT * FROM agent_flows;
SELECT * FROM flow_executions WHERE status = 'running';
SELECT * FROM conversations ORDER BY created_at DESC LIMIT 5;
SELECT * FROM documents WHERE ingestion_status = 'pending_approval';

Inspecting NATS

# Monitor all NATS subjects
nats sub "qadra.>"

# Monitor specific subject
nats sub "qadra.*.flow.do.*"

# NATS server info
curl http://localhost:8222/varz

Inspecting MongoDB Audit

docker exec qadra-mongodb mongosh \
  "mongodb://qadra:qadra_traces_dev@localhost:27017/qadra_traces?authSource=admin" \
  --quiet --eval "db.nats_audit.find().sort({timestamp: -1}).limit(5).toArray()"

Performance Tips

Compile Times

# Use cargo check instead of build when possible
cargo check

# Incremental builds (default)
export CARGO_INCREMENTAL=1

# Full workspace build ~3 min, incremental ~30s

Runtime Performance

#![allow(unused)]
fn main() {
// Pre-allocate vectors when size is known
let mut results = Vec::with_capacity(expected_count);

// Use references instead of clones
fn process(data: &Data) -> Result<()>  // Preferred

// Batch database operations
repo.insert_many(&items).await?;       // 1 query, not N
}