Domain Model

Core Concepts

ConceptDefinitionExample
TenantOrganization with isolated data"Acme Capital Partners"
AgentNamed specialist with a persona and system prompt"Research Analyst"
PipelineConfigurable sequence of stages with draft/live versioning"Deal Pipeline"
WorkloadUnit of work that moves through pipeline stages"Acme Corp Acquisition"
TaskAgent assignment to a workload at a stage"Research Analyst -> Research stage"
ArtifactOutput produced by an agent, optionally from a template"Market Analysis Report.pdf"
FlowVisual graph-based workflow with branching and human gates"Intake Triage Flow"
ConversationDirect agent chat interface with switchable agents"Research chat with Analyst"
DocumentSource material with human-gated ingestion approval"Acme Corp 10-K Filing"

Entity Relationships

Understanding the entity hierarchy is crucial for working with Qadra. Every entity belongs to a tenant, and relationships enforce data integrity.

Tenant
├── Users (via user_tenants junction)
│   ├── is_super_admin (platform-level flag)
│   ├── email_verified
│   └── avatar_url
├── API Keys
├── Team Invites
├── Files (S3/MinIO metadata)
├── Email Templates (override global defaults)
├── Notification Logs (multi-channel audit)
├── Agents[]
├── Pipelines[]
│   ├── stages (live JSONB)
│   └── draft_stages (in-progress edits)
├── Workloads[]
│   ├── pipeline_id -> Pipeline
│   ├── pipeline_snapshot (immutable copy at creation)
│   ├── stage_results (accumulated JSONB)
│   ├── Tasks[]
│   │   └── agent_id -> Agent
│   ├── Artifacts[]
│   │   ├── task_id -> Task
│   │   └── template_id -> ArtifactTemplate
│   └── Usage Records[]
├── Artifact Templates[]
│   └── renderer_id -> Plugin
├── Agent Flows[]
│   └── Flow Executions[]
├── Conversations[]
│   └── Messages[]
│       ├── agent_id -> Agent
│       └── flow_execution_id -> FlowExecution
├── Documents[] (human-gated ingestion)
│   ├── proposed_by -> User
│   ├── approved_by -> User
│   ├── DocumentTriples[] -> SPO Triples
│   └── DocumentChunks[] -> Embeddings
├── SPO Index
│   ├── Nodes[]
│   ├── Triples[]
│   └── Entity Aliases[]
├── Plugins[]
├── Renderers[]
├── BALLS Graphs[]
└── Memory Store[]

Key Relationships Explained

Tenant -> Everything: The tenant is the root of all data. Every table has a tenant_id foreign key. You cannot query across tenants, period.

Users -> Tenants (many-to-many): Users belong to tenants through the user_tenants junction table. Each membership carries a per-tenant role (owner, admin, user) and an is_default flag indicating which tenant loads on login. A user's email is globally unique.

Super Admin: The is_super_admin boolean on the users table is a platform-level flag, not a per-tenant role. Super admins can create tenants, drop into any tenant (bypasses membership), reset passwords, and manage other super admins. The first super admin is promoted via direct SQL.

Pipeline -> Stages: Pipelines define the workflow structure. Stages are stored as JSONB within the pipeline. Pipelines support draft_stages for editing without affecting running workloads, and a pipeline_snapshot is captured on each workload at creation time.

Workload -> Pipeline: A workload references a pipeline and snapshots its stages at creation. This ensures running workloads are unaffected by subsequent pipeline edits.

Workload -> Stage Results: Completed stages record structured results in workloads.stage_results JSONB, keyed by stage name. Downstream stages consume this as forwarded context containing task summaries, merged outputs, and artifact references.

Task -> Agent: Each task is assigned to exactly one agent. The agent performs the work. Multiple tasks in a stage might use the same or different agents.

Task -> Artifacts: Artifacts are outputs. One task might produce multiple artifacts. Artifacts can optionally reference an artifact_template for structured output formatting.

Flow -> Executions: Agent flows are visual graph definitions stored as ReactFlow JSONB. Each execution is a state machine (pending -> running -> paused -> completed/failed/cancelled) with a blackboard state dictionary, per-node states, and an execution queue.

Conversation -> Messages: Conversations are direct agent chat sessions. Messages reference the agent that produced them (switchable per message) and optionally link to a flow execution.

Document -> Triples + Chunks: Documents go through a human-gated approval flow before ingestion. Once approved, the ingestion pipeline extracts SPO triples and/or vector chunks, linking them back to the source document for provenance.

Agents

Agents are the workers in Qadra. Unlike chatbots that respond to prompts, agents are specialists with defined personas and system prompts that perform tasks within workflows and conversations.

Agent Model

Agents have been refactored from the atomics model (JavaScript programs in QuickJS) to a pure persona model. Agents are LLM personas with system prompts, not code artifacts.

Persona fields -- what defines the agent:

  • name: Internal identifier ("research-analyst")
  • display_name: Human-readable name ("Research Analyst")
  • description: What this agent does in plain language
  • avatar_url: Profile image
  • system_prompt: The LLM system prompt that defines behavior and expertise

Orchestration fields -- how the agent participates in workflows:

  • control_type: What happens after the agent completes (retain, relinquish_to_parent, relinquish_to_start)
  • output_visibility: Whether output is shown to users (user_facing) or kept internal (internal)
  • max_calls_per_parent_agent: Safety rail limiting recursive invocations (default: 3)

Agent Schema

CREATE TABLE agents (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    name TEXT NOT NULL,
    version INTEGER DEFAULT 1,

    -- Persona
    display_name TEXT,
    avatar_url TEXT,
    description TEXT,
    system_prompt TEXT NOT NULL DEFAULT '',

    -- Orchestration
    control_type control_type NOT NULL DEFAULT 'relinquish_to_parent',
    output_visibility output_visibility NOT NULL DEFAULT 'user_facing',
    max_calls_per_parent_agent INTEGER NOT NULL DEFAULT 3,

    -- Metadata
    is_active BOOLEAN DEFAULT true,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW(),

    UNIQUE(tenant_id, name, version)
);

Control Types

Control types define what happens to execution flow after an agent completes its work:

TypeBehaviorUse Case
retainAgent keeps control, can be called againConversational agents, iterative research
relinquish_to_parentReturns control to the calling agent or orchestratorMost agents -- do your job, hand back
relinquish_to_startReturns control to the pipeline entry pointTerminal agents that complete a branch

Output Visibility

VisibilityBehaviorUse Case
user_facingOutput displayed to the end userFinal deliverables, reports, answers
internalOutput kept within orchestration, hidden from userIntermediate reasoning, data gathering

Example Agent

{
  "name": "research-analyst",
  "display_name": "Research Analyst",
  "description": "Gathers market data and competitive analysis for deal evaluation",
  "avatar_url": "/avatars/analyst.png",
  "system_prompt": "You are a senior research analyst at an investment bank. Your role is to gather and synthesize market data, competitive intelligence, and industry trends. Always cite sources. Structure your output with clear sections: Market Overview, Competitive Landscape, Key Trends, and Risk Factors.",
  "control_type": "relinquish_to_parent",
  "output_visibility": "user_facing",
  "max_calls_per_parent_agent": 3
}

Why system prompts instead of JavaScript code? The original atomics model required agents to be JavaScript programs executed in QuickJS sandboxes. In practice, agent behavior is better expressed as natural language instructions to an LLM. System prompts are easier to write, test, and iterate on. The orchestration layer handles execution coordination; agents focus on expertise.

Versioning still works the same way. You might have research-analyst v1 and v2 with different system prompts but the same persona. Users see "Research Analyst" throughout -- the upgrade is invisible.

Canonical SPO Pipeline Personas

After the SPO-architecture decision (Story 98), four seeded agent presets form the canonical investment pipeline. Each writes to or reads from the SPO knowledge graph, so claims stay grounded and traceable as a deal moves through the stages.

PersonaRoleBehavior
ScarletSPO claim writerExtracts claims and writes them as SPO triples with provenance (asserter, source, assertion_type, confidence). Output is {"triples":[…],"summary":…} JSON.
AyanaInvestment filterReads Scarlet's SPO triples, applies investment filters, and routes deals PASS/CONDITIONAL/FAIL with a pre-screen score.
ElizaDue-diligence writerWrites due-diligence findings back to the SPO store, grounded in Scarlet's triples.
ReaganSenior Investment ReviewerSynthesizes upstream outputs into a final review (Recommendation, Conviction, Key Positives, Key Risks, Reviewer Notes).

These are the canonical personas following the SPO-architecture decision -- Scarlet establishes grounded facts, Ayana and Eliza consume and extend them, and Reagan produces the human-facing recommendation.

Agent CRUD

Agents are managed through the gateway HTTP API, which routes to Rust Core via NATS.

NATS subjects: qadra.{tenant_id}.agent.do.{operation} where operation is create, get, list, update, or delete.

MethodPathNATS SubjectDescription
POST/agentsqadra.{tid}.agent.do.createCreate a new agent
GET/agentsqadra.{tid}.agent.do.listList agents (paginated)
GET/agents/:agentIdqadra.{tid}.agent.do.getGet agent by ID
PUT/agents/:agentIdqadra.{tid}.agent.do.updateUpdate agent fields
DELETE/agents/:agentIdqadra.{tid}.agent.do.deleteDelete agent

Create request fields (all except name are optional with defaults):

FieldTypeDefaultValidation
namestringrequired1-200 chars
display_namestringnullmax 200 chars
avatar_urlstringnullvalid URL, max 2048 chars
descriptionstringnullmax 2000 chars
system_promptstring""no limit
control_typeenumrelinquish_to_parentretain, relinquish_to_parent, relinquish_to_start
output_visibilityenumuser_facinguser_facing, internal
max_calls_per_parent_agentinteger31-100

Update semantics: The gateway uses a sparse update pattern. Only fields explicitly included in the request body are sent to Rust Core. Missing fields are not updated. Fields set to null clear the value. This means you can update just the system prompt without touching other fields.

Agent Editor (UI)

The agent editor (web/src/pages/AgentEditor.tsx) provides a full editing interface organized into three panels:

  1. Persona panel: Display name, description. These define the agent's identity as seen by users.
  2. System Prompt panel: A monospace textarea for the LLM system prompt. This is the core of the agent -- the instructions that define behavior when invoked by a flow or pipeline. Supports Ctrl/Cmd+S to save.
  3. Behavior panel: Control type (dropdown), output visibility (dropdown), and max calls per parent (number input). These govern how the agent participates in orchestration.

The top toolbar shows the agent name (click to rename inline) and a save button with status indicator (saved/error). The editor loads the agent by ID from the URL param and populates all fields from the store.

Pipelines

Pipeline Schema

CREATE TABLE pipelines (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    name TEXT NOT NULL,
    description TEXT,
    stages JSONB NOT NULL DEFAULT '[]',       -- Live stage definitions
    draft_stages JSONB,                       -- In-progress edits (NULL = no draft)
    max_handoffs_per_turn INTEGER NOT NULL DEFAULT 25,  -- Loop prevention
    is_active BOOLEAN DEFAULT true,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW(),

    UNIQUE(tenant_id, name)
);

Draft/Live Versioning

Pipelines support editing without affecting running workloads:

  • stages: The live configuration. Running workloads use this (via snapshot).
  • draft_stages: An in-progress edit. NULL means no active draft.
  • Publishing: Overwrites stages with draft_stages, then clears the draft.
  • Discarding: Sets draft_stages to NULL, leaving stages untouched.

When a workload is created, the pipeline's current stages are copied into workloads.pipeline_snapshot. This makes the workload immune to subsequent pipeline edits.

Safety Rails

  • max_handoffs_per_turn (pipeline-level, default 25): Maximum agent-to-agent handoffs per orchestration turn. Prevents infinite loops.
  • max_calls_per_parent_agent (agent-level, default 3): Maximum times a parent agent can invoke a specific child agent per turn. Prevents recursive abuse.

Stage Definition

{
  "stages": [
    {
      "name": "Research",
      "order": 1,
      "description": "Gather market data and competitive analysis"
    },
    {
      "name": "Due Diligence",
      "order": 2,
      "description": "Validate claims and assess risks"
    },
    {
      "name": "Investment Memo",
      "order": 3,
      "description": "Synthesize findings into recommendation"
    },
    {
      "name": "Review",
      "order": 4,
      "description": "Final quality check"
    }
  ]
}

Workloads

Workloads are the central concept in Qadra -- they represent units of work that progress through pipeline stages. In investment banking terms, a workload is a deal. In IT terms, it might be a ticket or project.

Unlike chat threads that accumulate messages, workloads accumulate structured context as agents complete tasks. This context flows forward, enabling downstream agents to build on previous work.

Workload Schema

CREATE TABLE workloads (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    pipeline_id UUID REFERENCES pipelines(id),
    title TEXT NOT NULL,
    description TEXT,
    status workload_status DEFAULT 'pending',   -- pending, active, completed, failed, cancelled
    current_stage TEXT,
    current_stage_order INTEGER,
    metadata JSONB DEFAULT '{}',
    context JSONB DEFAULT '{}',                 -- Accumulated context from completed tasks
    pipeline_snapshot JSONB,                    -- Immutable copy of pipeline stages at creation
    stage_results JSONB NOT NULL DEFAULT '{}',  -- Per-stage structured results
    started_at TIMESTAMPTZ,
    completed_at TIMESTAMPTZ,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW()
);

Workload Lifecycle

Workloads follow a strict state machine. Understanding these states is essential for building UIs and debugging issues.

          +-------------+     +-------------+     +-------------+
  [*] --> |   pending   | --> |   active    | --> |  completed  | --> [*]
          +-------------+     +------+------+     +-------------+
                                     |
                                     v
                              +-------------+
                              |   failed    | --> [*]
                              +-------------+

State Descriptions:

StateMeaningWhat's Happening
pendingCreated but not startedWaiting for user to initiate processing
activeProcessing in progressTasks are running, stages advancing
completedAll stages finished successfullyAll artifacts produced, work is done
failedUnrecoverable error occurredA task failed and couldn't be retried
cancelledManually cancelledUser stopped processing

State Transitions:

FromToTriggerNotes
pendingactiveExecution startsCreates tasks for first stage, snapshots pipeline
activeactiveStage advancementAll stage tasks complete -> next stage
activecompletedAll stages completeFinal stage tasks done
activefailedTask failure (no retry)Retry exhausted or unretryable error
activecancelledUser cancellationManual stop

Stage Results and Context Forwarding

Completed pipeline stages produce structured results that flow to downstream stages. This replaces the simple context accumulation model with a richer structure.

Stage Result structure:

{
  "Research": {
    "stage_name": "Research",
    "stage_order": 1,
    "tasks": [
      {
        "task_id": "uuid",
        "agent_id": "uuid",
        "stage": "Research",
        "output": { "market_size": "$50B", "competitors": ["CompA", "CompB"] },
        "artifacts": [
          { "id": "uuid", "name": "Market Analysis", "artifact_type": "document", "content_type": "text/markdown", "size_bytes": 4200 }
        ],
        "completed_at": "2026-01-24T10:00:00Z"
      }
    ],
    "merged_output": { "market_size": "$50B", "competitors": ["CompA", "CompB"] },
    "completed_at": "2026-01-24T10:00:00Z"
  }
}

Key types:

TypeContentsPurpose
ArtifactRef{id, name, artifact_type, content_type, size_bytes}Lightweight pointer, no content blob
TaskSummary{task_id, agent_id, stage, output, artifacts[], completed_at}Task + its artifact references
StageResult{stage_name, stage_order, tasks[], merged_output, completed_at}Complete stage record

Forwarded context is built by build_forwarded_context() and contains {prior_stages, merged_outputs, artifact_refs} for downstream stage consumption.

Why references only? Artifact references store metadata, not content. Full content stays in the artifacts table. This keeps stage_results lightweight while providing enough information for downstream agents to know what's available.

Tasks

Task Schema

CREATE TABLE tasks (
    id UUID PRIMARY KEY,
    workload_id UUID NOT NULL REFERENCES workloads(id),
    agent_id UUID NOT NULL REFERENCES agents(id),
    stage TEXT NOT NULL,
    stage_order INTEGER NOT NULL,
    status task_status DEFAULT 'pending',     -- pending, queued, running, completed, failed, cancelled
    progress INTEGER DEFAULT 0,               -- 0-100
    input JSONB DEFAULT '{}',
    output JSONB,
    error TEXT,
    queued_at TIMESTAMPTZ,
    started_at TIMESTAMPTZ,
    completed_at TIMESTAMPTZ,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW()
);

Task Execution Flow

The Python Agent orchestrator (WorkloadOrchestrator) drives task execution. Data operations go to Rust Core via NATS.

1. Client sends qadra.{tenant}.workload.execute (NATS)
2. Python Agent creates workload -> Rust Core (workload.do.create)
3. For each pipeline stage (sequential):
   a. get_forwarded_context -> Rust Core (prior stage outputs, artifact refs)
   b. LLM selects best agent for stage (based on agent description/persona)
   c. create_task -> Rust Core (agent assigned to stage)
   d. LLM executes with agent system prompt + forwarded context
   e. complete_task -> Rust Core (output stored)
   f. create_artifact -> Rust Core (if output > 200 chars)
   g. record_stage_result -> Rust Core (persist to stage_results JSONB)
   h. advance_stage -> Rust Core (move to next stage)
4. Return WorkloadResult (success, stages_completed, artifacts)

Progress updates flow through Redis pub/sub to WebSocket connections, giving users real-time visibility into long-running tasks.

Artifacts

Artifact Schema

CREATE TABLE artifacts (
    id UUID PRIMARY KEY,
    workload_id UUID NOT NULL REFERENCES workloads(id),
    task_id UUID REFERENCES tasks(id),
    agent_id UUID NOT NULL REFERENCES agents(id),
    template_id UUID REFERENCES artifact_templates(id),  -- Optional template
    name TEXT NOT NULL,
    artifact_type TEXT NOT NULL,           -- 'document', 'analysis', 'memo', 'report', etc.
    content_type TEXT,                     -- MIME type
    content TEXT,                          -- Inline for small artifacts
    file_path TEXT,                        -- Storage path for large artifacts
    size_bytes BIGINT,
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT NOW()
);

Artifact Types

TypeDescriptionExample
documentWritten contentResearch report
analysisStructured analysisRisk assessment
memoInvestment memoIC presentation
reportFormatted reportDue diligence summary
dataStructured dataFinancial model

Artifact Templates

Artifact templates define the expected structure and output format for artifacts. They give agents a blueprint: what sections to produce, what schema to follow, and optionally which renderer plugin transforms the output into a final format (PDF, HTML, etc.).

Template Schema

CREATE TABLE artifact_templates (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
    name TEXT NOT NULL,
    display_name TEXT,
    description TEXT,
    version INTEGER NOT NULL DEFAULT 1,

    -- Template structure
    artifact_type TEXT NOT NULL DEFAULT 'document',   -- 'document', 'analysis', 'memo', 'report', etc.
    expected_sections JSONB NOT NULL DEFAULT '[]',    -- Section definitions (see below)
    output_schema JSONB,                              -- JSON Schema for structured output validation
    output_format TEXT NOT NULL DEFAULT 'markdown',   -- 'markdown', 'html', 'pdf', 'json'

    -- Renderer binding (optional)
    renderer_id UUID REFERENCES plugins(id) ON DELETE SET NULL,

    -- Metadata
    is_active BOOLEAN NOT NULL DEFAULT true,
    is_global BOOLEAN NOT NULL DEFAULT false,

    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),

    UNIQUE(tenant_id, name, version)
);

-- Artifacts can optionally reference a template
ALTER TABLE artifacts ADD COLUMN template_id UUID REFERENCES artifact_templates(id) ON DELETE SET NULL;

Indexes: Tenant-scoped indexes on artifact_type and active templates (partial index on is_active = true).

Creating a Template

A template requires a name and defaults to artifact type document with markdown output. All other fields are optional.

FieldTypeDefaultDescription
namestringrequiredInternal identifier (1-200 chars, unique per tenant+version)
display_namestringnullHuman-readable name (max 200 chars)
descriptionstringnullWhat this template is for (max 2000 chars)
artifact_typestring"document"Category: document, analysis, memo, report, etc.
expected_sectionsarray[]Section definitions (see below)
output_schemaobjectnullJSON Schema for validating structured output
output_formatenum"markdown"markdown, json, html, pdf
renderer_idUUIDnullOptional reference to a renderer plugin
is_activebooleantrueWhether this template is available for use
is_globalbooleanfalseWhether this template is available to all tenants

Renderer binding: When renderer_id is set, it references a plugin (from the plugins table) that transforms the agent's raw output into the target output_format. For example, a renderer plugin might convert markdown content into a styled PDF. If no renderer is bound, the raw output is stored as-is. Deleting the referenced plugin sets renderer_id to NULL (ON DELETE SET NULL).

Expected Sections

The expected_sections JSONB array defines the structure an agent should follow when producing an artifact from this template. Each section is an object:

[
  {
    "name": "Executive Summary",
    "description": "High-level overview of findings and recommendation",
    "required": true
  },
  {
    "name": "Market Analysis",
    "description": "Market size, growth trends, competitive landscape",
    "required": true
  },
  {
    "name": "Risk Factors",
    "description": "Key risks and mitigation strategies",
    "required": true
  },
  {
    "name": "Appendix",
    "description": "Supporting data tables and references",
    "required": false
  }
]
FieldTypeDefaultDescription
namestringrequiredSection heading
descriptionstringoptionalGuidance for what this section should contain
requiredbooleantrueWhether the agent must produce this section

Sections are passed to the agent as part of the prompt context. The agent's output is expected to follow this structure, though enforcement depends on the orchestration layer.

Using Templates

When an artifact references a template_id, the template's expected_sections and output_schema guide the agent's output:

  1. During task execution: The orchestrator includes the template's section definitions in the agent's prompt context, instructing it to produce output matching the template structure.
  2. After generation: If output_schema is set, the output can be validated against the JSON Schema.
  3. Rendering: If renderer_id is set, the renderer plugin transforms the raw output into the target output_format (e.g., markdown to PDF).
  4. Storage: The resulting artifact is stored in the artifacts table with template_id set, linking it back to the template for provenance.

In the investment banking example, the "Investment Memo" artifact at Stage 3 uses a template that ensures every memo has an Executive Summary, Market Analysis, Risk Factors, and Recommendation section -- consistent formatting across all deals.

Artifact Template API

Templates are managed through the gateway HTTP API, routed to Rust Core via NATS.

NATS subjects: qadra.{tenant_id}.artifact_template.do.{operation} where operation is create, get, list, update, or delete.

MethodPathNATS SubjectDescription
POST/artifact-templatesqadra.{tid}.artifact_template.do.createCreate a new template
GET/artifact-templatesqadra.{tid}.artifact_template.do.listList templates (paginated, filterable by artifact_type)
GET/artifact-templates/:idqadra.{tid}.artifact_template.do.getGet template by ID
PUT/artifact-templates/:idqadra.{tid}.artifact_template.do.updateUpdate template fields
DELETE/artifact-templates/:idqadra.{tid}.artifact_template.do.deleteDelete template

List query params: limit (1-100, default 20), offset (default 0), artifact_type (optional filter).

Artifact Template Editor (UI)

The template editor (web/src/pages/ArtifactTemplateEditor.tsx) provides four panels:

  1. Template Info: Display name, description, artifact type (dropdown), and output format (dropdown).
  2. Expected Sections: A dynamic list where you add, remove, and reorder sections. Each section has a name field, optional description, and a required checkbox.
  3. Output Schema: A monospace JSON textarea for defining a JSON Schema. Validated on save -- invalid JSON prevents saving.

The editor supports Ctrl/Cmd+S to save and shows inline save status (saved/error). The name is editable inline from the top toolbar.

Usage Records

Every LLM call, embedding generation, tool invocation, plugin execution, and renderer execution is tracked per-operation:

CREATE TABLE usage_records (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    workload_id UUID REFERENCES workloads(id),
    task_id UUID REFERENCES tasks(id),
    agent_id UUID REFERENCES agents(id),
    model TEXT,
    tokens_in INTEGER NOT NULL DEFAULT 0,
    tokens_out INTEGER NOT NULL DEFAULT 0,
    operation_type usage_operation_type NOT NULL DEFAULT 'llm_call',
    metadata JSONB NOT NULL DEFAULT '{}',
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Operation types: llm_call, embedding, tool_call, plugin_exec, renderer_exec.

Usage records enable aggregate queries (SUM tokens per tenant/workload), time-range filtering via indexed created_at, and batch inserts via UNNEST. Stored in a separate table rather than JSONB inside tasks because aggregate queries across workloads would require scanning all tasks and extracting nested arrays.

Agent Flows

Agent flows provide visual, graph-based workflows with branching, conditions, loops, and human-in-the-loop gates.

Flow Definition Schema

CREATE TABLE agent_flows (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    name TEXT NOT NULL,
    description TEXT,
    flow_data JSONB NOT NULL DEFAULT '{"nodes":[],"edges":[],"viewport":{"x":0,"y":0,"zoom":1}}',
    node_count INTEGER NOT NULL DEFAULT 0,
    is_active BOOLEAN NOT NULL DEFAULT true,
    created_by UUID REFERENCES users(id),
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),

    UNIQUE(tenant_id, name)
);

flow_data stores the ReactFlow graph definition as JSONB: {nodes, edges, viewport}. The executor always loads the full graph, never queries individual nodes. JSONB stores it verbatim with no impedance mismatch.

Flow Execution Schema

CREATE TABLE flow_executions (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    flow_id UUID NOT NULL REFERENCES agent_flows(id),
    flow_snapshot JSONB NOT NULL,              -- Immutable copy at execution start
    status flow_execution_status NOT NULL DEFAULT 'pending',
    state JSONB NOT NULL DEFAULT '{}',         -- Blackboard dictionary
    node_states JSONB NOT NULL DEFAULT '{}',   -- Per-node execution state
    execution_queue JSONB NOT NULL DEFAULT '[]',
    current_node_id TEXT,
    error TEXT,
    triggered_by UUID REFERENCES users(id),
    input JSONB NOT NULL DEFAULT '{}',
    started_at TIMESTAMPTZ,
    completed_at TIMESTAMPTZ,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Flow Execution Lifecycle

stateDiagram-v2
    [*] --> pending
    pending --> running : execute
    running --> completed : End node reached
    running --> failed : node error / safety limit
    running --> cancelled : user cancels
    running --> paused : HumanInput node
    paused --> running : user submits input
    paused --> cancelled : user cancels
StatusMeaning
pendingCreated but not started
runningActively executing nodes
pausedWaiting for human input at a HumanInput node
completedEnd node reached
failedNode error or safety limit exceeded
cancelledUser cancelled

Node Types

The flow executor supports 14 node types. Each node stores its configuration in the data JSONB field of the ReactFlow node object. All nodes that produce output can specify an output_key in their data; after execution, the output is merged into the flow state dictionary under that key.

Start

Entry point for the flow. Exactly one per flow.

FieldTypeDescription
labelstringDisplay name

Behavior: Pass-through. The execution input (provided when starting the flow) becomes the initial state. All successors are enqueued immediately. No output_key -- state is seeded from execution input directly.

End

Terminal node. At least one per flow. When reached, the execution status becomes completed.

FieldTypeDescription
labelstringDisplay name
output_keysstring[]Optional. If set, only these keys from state are included in the final output. If omitted, the entire state dictionary is the output.

Behavior: Collects final output. If output_keys is specified, filters the state to only those keys. Otherwise returns the full state. No successors are enqueued.

Agent

Invokes an LLM with the agent's system prompt and a rendered prompt template. This is the primary intelligence node.

FieldTypeDescription
labelstringDisplay name
agent_idUUID stringReferences an agent in the agents table. The agent's system_prompt is loaded and used as the LLM system message.
prompt_templatestringInstructions for this step. Supports {{variable}} template substitution from state.
output_keystringState key where the result is stored.

Behavior: Renders prompt_template by replacing {{key}} placeholders with values from the flow state. Loads the agent's system_prompt by agent_id. Injects upstream state data as a second system message so the agent has context from prior nodes. Calls the LLM with temperature: 0.7, max_tokens: 4096. Output is a JSON object {response, model, tokens}. All successors are enqueued.

Condition

Boolean branching node. Evaluates an expression against the flow state and routes to the matching branch.

FieldTypeDescription
labelstringDisplay name
expressionstringExpression to evaluate. Supports: key == value, key != value, or key (truthy check).
modestring"expression" (default) or "llm" (LLM-based evaluation, placeholder).

Expression evaluation rules:

  • key == value: Compares state[key] against value. Strings, numbers, and booleans are compared by string representation. Returns route "true" or "false".
  • key != value: Inverse of ==. Returns "true" or "false".
  • key (bare): Truthy check. true for non-empty strings, any number, true booleans, non-empty arrays/objects. false for empty strings, null, missing keys, false booleans.

Edge behavior: Outgoing edges MUST use sourceHandle to indicate which route they belong to. An edge with sourceHandle: "true" is followed when the expression evaluates to true; sourceHandle: "false" for the false branch. Only the matching branch is enqueued. Nodes on the non-matching branch (and their entire downstream subtree) are marked Skipped, unless they are also reachable from the taken branch.

HumanInput

Pauses the entire flow execution and waits for a user to submit input via the API.

FieldTypeDescription
labelstringDisplay name
promptstringMessage shown to the user describing what input is needed.
timeout_secondsnumberOptional. Not currently enforced.
output_keystringState key where the user's response is stored after resumption.

Behavior: Immediately returns NodeResult::Paused. The node status is set to WaitingForInput and the execution status is set to Paused. State and queue are persisted to the database. When the user submits input via POST /flows/executions/:execId/input, the NATS handler stores the response in the node's output, marks the node as Completed, merges the response into state under output_key, re-enqueues successors, and resumes the execution loop.

Loop

Iterates over a collection in the flow state. Currently a simplified batch processor -- it reads the array, caps it to max_iterations, and outputs the items. It does NOT re-execute successor nodes per item.

FieldTypeDescription
labelstringDisplay name
collection_keystringState key containing the array to iterate over. Default: "items".
max_iterationsnumberMaximum items to process. Default: 100.

Behavior: Reads state[collection_key] as an array. If missing or not an array, outputs {iterations: 0, results: []}. Otherwise caps at max_iterations and outputs {iterations, results}. All successors are enqueued once (not per item).

Http

Makes an outbound HTTP request. Supports GET, POST, PUT, PATCH, DELETE.

FieldTypeDescription
labelstringDisplay name
methodstringHTTP method. Default: "GET".
urlstringRequired. Supports {{variable}} template substitution from state.
headersobjectOptional. Key-value map of request headers.
bodyJSONOptional. Request body for POST/PUT/PATCH (sent as JSON).
output_keystringState key where the response is stored.

Behavior: Templates the URL by replacing {{key}} with state values. Sends the request with a 30-second timeout. Output is {status, body} where body is parsed as JSON if possible, otherwise stored as a string. All successors are enqueued.

ExecuteFlow

Triggers another flow as a sub-flow. Currently a stub -- returns a placeholder message.

FieldTypeDescription
labelstringDisplay name
target_flow_idUUID stringThe flow to execute.
input_mappingstringHow to map current state to sub-flow input.

Behavior: Returns {message: "Nested flow execution not yet implemented"}. All successors are enqueued. Future implementation will create a child FlowExecution, run it to completion, and merge its output back into the parent state.

Transform

Applies a data transformation by mapping state fields to new output fields.

FieldTypeDescription
labelstringDisplay name
mappingsobjectKey-value map where keys are output field names and values are either state field names (string) to copy, or literal JSON values.
output_keystringState key where the result is stored.

Behavior: For each entry in mappings: if the value is a string, it is treated as a state key reference and the corresponding state value is copied. If the value is any other JSON type, it is used as a literal. Output is a JSON object with the mapped fields. All successors are enqueued.

Example: {"mappings": {"company": "company_name", "ready": true}} copies state.company_name into output.company and sets output.ready to true.

Delay

Waits for a specified duration before continuing. Capped at 300 seconds.

FieldTypeDescription
labelstringDisplay name
duration_secondsnumberSeconds to wait. Default: 1. Maximum: 300 (5 minutes).

Behavior: Sleeps for min(duration_seconds, 300) seconds using tokio::time::sleep. Output is {delayed_seconds}. All successors are enqueued after the delay.

KnowledgeLookup

Queries the tenant's SPO knowledge graph for relevant triples.

FieldTypeDescription
labelstringDisplay name
querystringSearch query. Supports {{variable}} template substitution.
sourcestring"spo", "vector", or "both". Default: "both". Currently queries SPO regardless of this value.
output_keystringState key where results are stored. Default: "knowledge".
limitnumberMaximum results. Default: 10.

Behavior: Renders the query template with state variables. Searches SPO nodes matching the query text via search_spo_nodes, then fetches triples connected to those nodes via get_triples_for_nodes. Deduplicates by triple ID. Output is {query, source, limit, results} where results is an array of {subject, predicate, object, confidence, source} objects. Gracefully returns empty results on error. All successors are enqueued.

Research

Performs external research via LLM. Designed for Exa neural search integration (not yet wired -- currently uses LLM knowledge as a stand-in).

FieldTypeDescription
labelstringDisplay name
querystringResearch query. Supports {{variable}} template substitution.
max_resultsnumberMaximum research results. Default: 3.
output_keystringState key where findings are stored. Default: "research_result".

Behavior: Renders the query template. Calls the LLM with a research analyst system prompt and the rendered query plus workflow context (truncated to 4000 chars). Output is {query, findings, model, source: "llm_knowledge"}. All successors are enqueued.

Artifact

Produces a structured artifact, optionally formatted against an artifact template.

FieldTypeDescription
labelstringDisplay name
template_idUUID stringOptional. References an artifact_templates row. Provides section structure, output schema, and format guidance.
artifact_namestringName for the artifact. Supports {{variable}} template substitution. Default: "Untitled Artifact".
content_keystringState key containing upstream content to format. If empty or null, the full state is used as context.
output_keystringState key where the artifact is stored. Default: "artifact".

Behavior: Four modes depending on available inputs:

  1. No upstream content, no template: Calls LLM to generate content from the full state context.
  2. No upstream content, with template: Calls LLM with template section guidance to generate structured content.
  3. Upstream content, with template: Calls LLM to reformat the upstream content into the template structure.
  4. Upstream content, no template: Uses the upstream content as-is (no LLM call).

Output is {artifact_name, template_id, artifact_type, output_format, content}. All successors are enqueued.

Extract

Extracts structured parameters from text using an LLM. Typically placed immediately after the Start node to parse user intent into typed variables that downstream nodes reference via {{variable_name}}.

FieldTypeDescription
labelstringDisplay name
input_keystringState key containing the raw text to extract from. Default: "user_message".
output_variablesstringComma-separated list of variable names to extract (e.g., "company_name, metric, time_period").
instructionsstringOptional. Additional extraction instructions for the LLM.

Behavior: Reads state[input_key] as the raw text. If empty, sets all output variables to null. Otherwise, calls the LLM (temperature: 0.0, max_tokens: 1024) with a prompt asking it to extract the named variables as a JSON object. Strips markdown fences from the response if present. Parses the JSON and maps each requested variable into the output. If the LLM returns non-JSON, gracefully sets all variables to null. The output is a flat object -- each extracted variable becomes a top-level key. Unlike other nodes, Extract merges its output variables directly into state (each variable becomes a state key), making them available to all downstream nodes via {{variable_name}}.

Note: Extract nodes do NOT use output_key. Instead, each extracted variable is merged individually into the flow state. This is because the node returns a flat object (not nested under a key), and the executor's output-merge logic writes each top-level key into state.

AI Flow Generation

Flows can be generated from natural language descriptions using an LLM. The flow generator (src/flow_generator.rs) translates user prompts into valid ReactFlow graph definitions.

How It Works

  1. User submits a prompt via POST /flows/generate with a natural language description of the desired workflow.
  2. Gateway forwards the request to Rust Core via NATS subject qadra.{tid}.flow.do.generate.
  3. Rust Core loads context: active agents and artifact templates for the tenant.
  4. LLM generates the graph: A system prompt describes all 14 node types with their data field schemas, edge format, layout rules, and structural constraints. The user message includes the available agents (with IDs, names, descriptions) and artifact templates, followed by the user's request.
  5. Validation: The generated JSON is validated for structural correctness.
  6. Position assignment: If nodes lack positions, a top-to-bottom grid layout is applied automatically.
  7. Result returned: The validated flow_data JSONB is returned to the gateway, ready to be saved as a flow.

Node Catalog

The node catalog (NODE_CATALOG in flow_generator.rs) is the single source of truth for node types. It defines the type_name, description, data_example, and notes for each node type. The LLM system prompt, validation logic, and frontend palette all derive from this catalog. Adding a new node type means adding one NodeSpec entry.

LLM Prompt Structure

The system prompt instructs the LLM to return raw JSON (no markdown fences) with this structure:

{
  "nodes": [
    {
      "id": "{type}_{unique_number}",
      "type": "{node_type}",
      "position": { "x": number, "y": number },
      "data": { "label": "Display Name", ...type_specific_fields }
    }
  ],
  "edges": [
    {
      "id": "e_{source_id}-{target_id}",
      "source": "{source_node_id}",
      "target": "{target_node_id}",
      "animated": true,
      "sourceHandle": "true|false"
    }
  ],
  "viewport": { "x": 0, "y": 0, "zoom": 1 }
}

Layout rules: Start node at (400, 50), 200px vertical spacing, 300px horizontal spacing for parallel branches. Condition edges use sourceHandle of "true" or "false".

LLM parameters: temperature: 0.3, max_tokens: 4096.

Validation

validate_flow_data() checks:

  • nodes and edges arrays exist and are non-empty
  • Exactly 1 start node
  • At least 1 end node
  • Every node type is in the catalog (rejects unknown types)
  • Every edge references valid source and target node IDs

If the LLM wraps its response in markdown code fences despite instructions, they are stripped before parsing.

Position Assignment

ensure_positions() checks whether any node is missing a position or has a position without x/y. If so, all nodes receive a simple top-to-bottom grid layout starting at (400, 50) with 200px vertical spacing.

Flow Execution Engine

The event-driven graph executor in src/flow_executor.rs processes flow graphs. It is a Rust in-process executor that shares the PostgreSQL connection pool and NATS handlers with the rest of Rust Core.

Execution Loop

1. Load execution record (must be status=Pending)
2. Parse graph from flow_snapshot (immutable copy)
3. Initialize: all nodes -> Pending, seed queue with Start node
4. Set execution status -> Running
5. LOOP:
   a. Pop node_id from front of queue
   b. Check all predecessors are Completed or Skipped (join gate)
      - If not all done: re-enqueue and continue
   c. Set node status -> Running, record started_at, snapshot state as input
   d. Persist state to database (crash recovery point)
   e. Emit "node_started" progress event via SSE channel
   f. Execute the node (dispatch by type)
   g. Handle result:
      - Completed: mark Completed, merge output into state via output_key,
        enqueue successors (respecting condition routing), skip non-taken branches
      - Paused: mark WaitingForInput, persist state, set execution -> Paused, RETURN
      - Failed: mark Failed, set execution -> Failed with error, RETURN
   h. Emit "node_completed" or "node_failed" progress event
6. Queue empty: set execution status -> Completed, persist final state

State Merging

The flow state is a JSONB blackboard dictionary shared by all nodes. State flows through the graph as follows:

  • Execution input seeds the initial state (e.g., {"user_message": "Analyze NVIDIA"})
  • After each node completes, if the node has an output_key in its data, the node's output is stored as state[output_key] = output
  • Extract nodes are special: they return a flat object of extracted variables, and each variable becomes an individual state key (no nesting under output_key)
  • Each node receives the full state as its input. Nodes read from state via template substitution ({{key}}) or direct key access
  • State is never pruned -- it accumulates throughout the execution. All prior node outputs remain available to all downstream nodes

Predecessor Join Gate

Before executing a node, the executor checks that ALL predecessor nodes (based on incoming edges) have status Completed or Skipped. This implements an implicit AND-join for nodes with multiple incoming edges. If predecessors are not yet done, the node is re-enqueued at the back of the queue.

Condition Routing and Branch Skipping

When a Condition node completes, it returns a route value ("true" or "false"). The executor then:

  1. For each outgoing edge, checks if the edge's sourceHandle matches the route
  2. Matching edges: target nodes are enqueued
  3. Non-matching edges: the skip_subtree() function marks the target node and its entire downstream subtree as Skipped (via depth-first traversal), unless a node is also reachable from the taken branch

This ensures that only the relevant branch executes, and nodes on dead branches are marked Skipped rather than left Pending.

HumanInput Pause and Resume

Pause:

  1. HumanInput node immediately returns NodeResult::Paused
  2. Node status set to WaitingForInput
  3. Full execution state (state dict, node_states, queue) persisted to database
  4. Execution status set to Paused
  5. Executor returns -- the execution is now dormant

Resume (triggered by POST /flows/executions/:execId/input):

  1. NATS handler stores user input in the waiting node's output field
  2. FlowExecutor::resume() is called
  3. Loads persisted state from database
  4. Finds the WaitingForInput node, marks it Completed
  5. Merges the node's output into state under its output_key
  6. Re-enqueues the node's successors
  7. Sets execution status to Running
  8. Re-enters the same execution loop as run()

Nested ExecuteFlow

The ExecuteFlow node type is currently a stub. It returns {message: "Nested flow execution not yet implemented"} and enqueues successors normally. The intended future behavior: load the target flow by target_flow_id, create a child FlowExecution, run it synchronously, and merge its final state back into the parent flow's state.

Safety Rails

RailLimitBehavior on Violation
MAX_TOTAL_STEPS500 node executions per runExecution fails with error
Loop max iterations100 itemsArray truncated silently
Delay max duration300 seconds (5 min)Duration capped silently
HTTP timeout30 seconds per requestNode fails with error

MAX_TOTAL_STEPS is checked on every iteration of the execution loop (both initial run() and resume()). It counts total node executions, not unique nodes -- a loop that re-enqueues nodes can exhaust this limit.

State Persistence and Crash Recovery

The executor persists the full execution state to PostgreSQL after every node execution via update_execution_state(). This writes the current state, node_states, execution_queue, and current_node_id to the flow_executions row. If the Rust process crashes mid-execution, the persisted state contains enough information to determine which nodes completed and where the queue was, though automatic crash recovery is not currently implemented.

Progress Events

The executor optionally emits real-time progress events via an mpsc::UnboundedSender<FlowProgressEvent> channel (attached via with_progress()). Events are fire-and-forget and never block execution. Three event types:

EventDataWhen
node_started{node_id, node_type, label}Before node execution
node_completed{node_id, output_preview (200 char max), duration_ms}After successful execution
node_failed{node_id, error}After failed execution

These events are streamed to the frontend via SSE for live flow execution visualization.

Conversations

Conversations provide a direct agent chat interface, separate from pipeline-based workloads. While workloads move structured data through pipeline stages, conversations are freeform -- a user talks to agents, switches between them, and can even trigger flow executions inline.

Conversation Model

A conversation belongs to a single user within a tenant. It holds an ordered sequence of messages, each of which records the agent that responded (if any), the LLM model used, performance metrics, and optional links to flow executions.

CREATE TABLE conversations (
    id          UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id   UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
    user_id     UUID NOT NULL REFERENCES users(id) ON DELETE SET NULL,
    title       TEXT,                            -- Auto-generated or user-set
    is_archived BOOLEAN NOT NULL DEFAULT false,
    deleted_at  TIMESTAMPTZ,                     -- Soft delete (filtered by default)
    created_at  TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at  TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE conversation_messages (
    id                  UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    conversation_id     UUID NOT NULL REFERENCES conversations(id) ON DELETE CASCADE,
    agent_id            UUID REFERENCES agents(id) ON DELETE SET NULL,
    role                TEXT NOT NULL CHECK (role IN ('user', 'assistant', 'system')),
    content             TEXT NOT NULL,
    model               TEXT,                    -- LLM model identifier
    token_count         INTEGER,                 -- Total tokens consumed
    latency_ms          INTEGER,                 -- End-to-end response time
    finish_reason       TEXT,                    -- 'stop', 'length', 'tool_calls', etc.
    rating              SMALLINT CHECK (rating IN (-1, 1)),  -- Thumbs up/down
    flow_execution_id   UUID REFERENCES flow_executions(id) ON DELETE SET NULL,
    metadata            JSONB NOT NULL DEFAULT '{}',
    created_at          TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Indexes:

  • idx_conversations_tenant -- tenant-scoped listing, excludes soft-deleted (WHERE deleted_at IS NULL)
  • idx_conversations_tenant_user -- per-user listing within a tenant, excludes soft-deleted
  • idx_conversation_messages_conv_created -- ordered message retrieval by conversation
  • idx_conversation_messages_agent -- find all messages by a specific agent
  • idx_conversation_messages_rating -- query rated messages for feedback analysis

Relationship to other entities:

  • Each conversation_messages.agent_id points to the agent that produced the assistant response. User messages have agent_id = NULL.
  • conversation_messages.flow_execution_id links to a flow_executions row when the message was produced by running a flow.
  • conversation_messages.metadata is a JSONB bag that carries artifacts (inline content blocks for rendering) and suggested_actions (follow-up prompts the UI can present).

Agent Switching

Unlike most chat interfaces where the "assistant" is a single model, Qadra conversations let users switch agents on every message. The agent_id lives on conversation_messages, not on conversations.

How it works:

  1. The user selects an agent in the UI (agent picker dropdown).
  2. When sending a message, the selected agent_id is included in the request body.
  3. The backend invokes the chosen agent's system prompt + conversation history to generate a response.
  4. Both the user message and assistant message are persisted. The assistant message records the agent_id that produced it.
  5. The UI auto-selects the last responding agent for the next message, but the user can switch at any time.

This means a single conversation can contain messages from multiple agents. For example, a user might ask the Research Analyst about market data, then switch to the Due Diligence Specialist to probe risk factors -- all within the same conversation thread.

Flow Tool-Use

Agents can trigger flow executions from within a conversation. When an agent determines that a user's request maps to an existing flow, the backend executes the flow and streams progress events back to the user in real time.

How flow integration works:

  1. The send_message handler on the Rust Core evaluates whether the agent should invoke a flow (based on the agent's tool configuration and the conversation context).
  2. If a flow is triggered, the executor runs the flow graph and publishes progress events to a NATS progress subject.
  3. The gateway forwards these NATS events as SSE events to the client (see SSE Streaming below).
  4. When the flow completes, the agent receives the flow's output state and synthesizes a final response.
  5. The assistant message is persisted with flow_execution_id set, linking the conversational response to the full flow execution record.

The frontend renders flow progress inline in the chat: a live visualization of node execution with status indicators, output previews, and duration metrics.

SSE Streaming

The POST /conversations/:id/messages/stream endpoint uses Server-Sent Events to deliver real-time progress during message processing. This is a POST endpoint (not GET) because it sends the user's message in the request body and streams the response.

Connection setup:

  • The gateway sets SSE headers (Content-Type: text/event-stream, Cache-Control: no-cache, Connection: keep-alive, X-Accel-Buffering: no).
  • A unique NATS progress subject (_PROGRESS.{uuid}) is created for this request.
  • The gateway subscribes to the progress subject, then sends the NATS request to Rust Core with the progress_subject field set.
  • Rust Core publishes FlowProgressEvent messages ({event, data, seq}) to the progress subject during processing.
  • The gateway forwards each event to the SSE stream.

Event types and their data payloads:

EventDataWhen
user_saved{user_message}User message persisted to DB (replaces optimistic message)
thinking{status}Agent is processing (e.g. "Thinking...")
flow_started{flow_name, execution_id}A flow execution has begun
node_started{node_id, node_type, label}A flow node started executing
node_completed{node_id, output_preview, duration_ms}A flow node finished successfully
node_failed{node_id, error}A flow node failed
flow_completed{}Flow execution completed
flow_failed{error}Flow execution failed
assistant_message{user_message, assistant_message, conversation}Final response with both persisted messages and updated conversation
error{message}An error occurred
done{}Stream is complete, client should close connection

Wire format (standard SSE):

event: thinking
data: {"status":"Thinking..."}

event: flow_started
data: {"flow_name":"Intake Triage","execution_id":"abc-123"}

event: node_started
data: {"node_id":"node-1","node_type":"Agent","label":"Classify Request"}

event: node_completed
data: {"node_id":"node-1","output_preview":"Category: Research","duration_ms":1200}

event: assistant_message
data: {"user_message":{...},"assistant_message":{...},"conversation":{...}}

event: done
data: {}

Timeouts and fallbacks:

  • The NATS request has a 240-second timeout (flow executor 180s + follow-up LLM ~30s + buffer).
  • The gateway has a 250-second overall safety timeout on the SSE stream.
  • If the progress events do not deliver an assistant_message before the NATS reply arrives, the gateway uses the NATS reply as a fallback and writes it as an assistant_message SSE event.
  • If the SSE stream fails entirely, the frontend falls back to the non-streaming POST /conversations/:id/messages endpoint.

Message Rating

Users can rate individual assistant messages with thumbs up (+1) or thumbs down (-1). Ratings are stored directly on the conversation_messages row.

  • Endpoint: POST /conversations/:id/messages/:messageId/rate
  • Body: { "rating": 1 } or { "rating": -1 }
  • NATS subject: qadra.{tenant_id}.chat.do.rate_message
  • Constraint: rating column is SMALLINT CHECK (rating IN (-1, 1)). Only -1 and 1 are valid.
  • Idempotent: Rating the same message again overwrites the previous rating.
  • Index: idx_conversation_messages_rating enables efficient queries on rated messages for feedback analysis and model evaluation.

Conversation Lifecycle

Create -- POST /conversations

Creates an empty conversation. Title is optional (can be auto-generated later from the first message).

Send message -- POST /conversations/:id/messages (non-streaming) or POST /conversations/:id/messages/stream (SSE)

Sends a user message and receives the agent's response. The request body includes agent_id (required), content (required, max 100,000 characters), and optionally flow_execution_id. Both endpoints persist the user message and assistant message, returning them along with the updated conversation record.

List conversations -- GET /conversations?limit=20&offset=0

Returns ConversationSummary objects with message_count, last_message_preview, and last_agent_id for sidebar display. Excludes soft-deleted conversations.

Get conversation -- GET /conversations/:id

Returns the full conversation record.

List messages -- GET /conversations/:id/messages?limit=100&offset=0

Returns messages ordered by created_at. Used on page load and as a recovery mechanism when SSE streaming fails.

Update title -- PATCH /conversations/:id

Updates the conversation title.

Delete -- DELETE /conversations/:id

Soft-deletes the conversation by setting deleted_at. The conversation and its messages are preserved for audit but excluded from all listing queries.

NATS Subjects

All conversation operations go through tenant-scoped NATS subjects:

OperationNATS Subject
Createqadra.{tid}.chat.do.create
Getqadra.{tid}.chat.do.get
Listqadra.{tid}.chat.do.list
Update titleqadra.{tid}.chat.do.update_title
Deleteqadra.{tid}.chat.do.delete
Send messageqadra.{tid}.chat.do.send_message
List messagesqadra.{tid}.chat.do.list_messages
Rate messageqadra.{tid}.chat.do.rate_message

Key Design Decisions

  • agent_id on messages, not conversations: Users can switch agents mid-conversation. One message might come from the Research Analyst, the next from the Due Diligence Specialist. The conversation itself is agent-agnostic.
  • flow_execution_id on messages: Messages can be produced by flow executions, linking conversational output to the full flow execution record with its node states and blackboard data.
  • Soft delete with deleted_at: Conversations are never hard-deleted. Setting deleted_at excludes them from listing queries while preserving the audit trail.
  • metadata JSONB bag: Carries artifacts (inline content blocks like charts or agent-produced documents) and suggested_actions (follow-up prompts). Extensible without schema migration.
  • SSE over WebSocket for streaming: SSE is simpler to implement (one-way server-to-client), works through proxies/CDNs, and naturally maps to the request-response pattern of sending a message and streaming the response. WebSocket would be overkill for this use case.

Document Ingestion

Documents go through a human-gated approval flow before knowledge ingestion. Agents can propose documents, but humans must approve before ingestion runs.

Document Schema

CREATE TABLE documents (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    title TEXT,
    content TEXT NOT NULL,
    content_type TEXT DEFAULT 'text/plain',
    source_url TEXT,
    source_type TEXT,                          -- 'upload', 'web', 'api', 'research'

    -- Ingestion lifecycle
    ingestion_status TEXT DEFAULT 'pending_approval',
    ingestion_result JSONB,

    -- Proposal gate
    proposed_by UUID REFERENCES users(id),     -- Who submitted for review
    approved_by UUID REFERENCES users(id),     -- Human who opened the gate
    approved_at TIMESTAMPTZ,
    rejected_by UUID REFERENCES users(id),     -- Who rejected (if rejected)
    rejected_at TIMESTAMPTZ,
    rejection_reason TEXT,

    created_at TIMESTAMPTZ DEFAULT NOW(),
    processed_at TIMESTAMPTZ
);

Document Lifecycle

pending_approval --> approved --> processing --> completed
                                            --> failed
pending_approval --> rejected
StatusMeaning
pending_approvalProposed but awaiting human review
approvedHuman approved, ready for ingestion
processingIngestion pipeline running (routing, extraction, embedding)
completedTriples and/or chunks created
failedIngestion error
rejectedHuman rejected the proposal

Why human-gated? Agents can autonomously discover and propose documents for ingestion, but a human must approve before content enters the knowledge graph. This prevents polluting the KG with low-quality or irrelevant content.

Ingestion Routing

Once approved, the ingestion pipeline routes content:

RouteActionWhen
SPOExtract triples onlyShort structured content (facts, definitions)
VectorChunk and embed onlyLong narrative (articles, reports)
BothExtract triples AND chunkMixed content
DiscardSkip (boilerplate, junk)Low-value content

Results link back to the document:

  • document_triples: Which SPO triples came from this document
  • document_chunks: Which vector chunks came from this document

Users and Authentication

User Schema

CREATE TABLE users (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),  -- Legacy, kept for backward compat
    email TEXT NOT NULL,
    email_verified BOOLEAN NOT NULL DEFAULT false,
    password_hash TEXT NOT NULL,
    name TEXT NOT NULL,
    role TEXT NOT NULL DEFAULT 'user',
    avatar_url TEXT,
    is_super_admin BOOLEAN NOT NULL DEFAULT false,
    settings JSONB DEFAULT '{}',
    last_login_at TIMESTAMPTZ,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW()
);

Multi-Tenant Memberships

CREATE TABLE user_tenants (
    user_id UUID NOT NULL REFERENCES users(id),
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    role TEXT NOT NULL DEFAULT 'user',       -- 'owner', 'admin', 'user'
    is_default BOOLEAN NOT NULL DEFAULT false,
    joined_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    PRIMARY KEY (user_id, tenant_id)
);
  • Globally unique email: One account per email, belonging to multiple tenants
  • Per-tenant roles: owner (immutable), admin, user
  • is_default: Which tenant loads on login
  • Registration: Atomic transaction creates tenant + user + user_tenants in one operation

Email Verification

CREATE TABLE email_verification_tokens (
    id UUID PRIMARY KEY,
    user_id UUID NOT NULL REFERENCES users(id),
    token TEXT NOT NULL UNIQUE,            -- 32 random bytes, hex-encoded (64 chars)
    expires_at TIMESTAMPTZ NOT NULL,       -- 24h expiry
    used_at TIMESTAMPTZ,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
  • Token validated + not used + not expired -> mark used_at, set email_verified = true
  • Post-registration: fire-and-forget async task (doesn't block registration response)
  • Resend: invalidates existing tokens, creates new, sends new email

Team Invites

CREATE TABLE team_invites (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    email TEXT NOT NULL,
    role TEXT NOT NULL DEFAULT 'user',
    invited_by UUID NOT NULL REFERENCES users(id),
    token TEXT NOT NULL UNIQUE,            -- 32 random bytes, hex-encoded, 7-day expiry
    expires_at TIMESTAMPTZ NOT NULL,
    accepted_at TIMESTAMPTZ,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Smart invite logic: If a user with the email already exists globally, they are added to the tenant directly. If not, an invite is created and an email is sent. Only owner/admin can invite, change roles, or remove members.

Email Templates

CREATE TABLE email_templates (
    id UUID PRIMARY KEY,
    tenant_id UUID REFERENCES tenants(id),  -- NULL = global default
    slug TEXT NOT NULL,                      -- 'verification', 'team_invite', etc.
    name TEXT NOT NULL,
    subject TEXT NOT NULL,                   -- Handlebars: 'Verify your email, {{user_name}}'
    html_body TEXT NOT NULL,                 -- Handlebars HTML template
    text_body TEXT,                          -- Handlebars plain text fallback
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    UNIQUE(tenant_id, slug)
);
  • Template resolution: Tenant-specific wins over global (ORDER BY tenant_id IS NULL ASC LIMIT 1)
  • Handlebars rendering: {{var}} syntax for subject, html_body, text_body
  • Tenant overrides: Tenants can customize any global template by slug

Super Admin

The is_super_admin boolean on the users table is a platform-level privilege:

CapabilityDescription
Create tenantsCreate tenant + assign owner
Drop into any tenantSynthetic JWT with target tenant context (bypasses membership)
Reset passwordsForce-reset any user's password
List all tenants/usersPlatform-wide visibility
Promote/demoteGrant or revoke super admin (cannot self-promote/demote)

The first super admin is promoted via direct SQL. JWT carries sa: true claim when present.

Files

File metadata is stored in PostgreSQL. Actual blobs live in S3-compatible object storage (MinIO by default, tenant-configurable).

CREATE TABLE files (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    uploaded_by UUID NOT NULL REFERENCES users(id),
    storage_provider TEXT NOT NULL DEFAULT 'internal',  -- 'internal' (MinIO) or 's3'
    bucket TEXT NOT NULL DEFAULT 'qadra-files',
    object_key TEXT NOT NULL,
    filename TEXT NOT NULL,
    content_type TEXT NOT NULL,
    size_bytes BIGINT NOT NULL,
    purpose TEXT NOT NULL,                     -- 'avatar', 'artifact', 'logo', 'attachment'
    entity_type TEXT,                          -- Polymorphic: 'user', 'tenant', 'workload'
    entity_id UUID,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
  • Gateway-proxied downloads: URLs are always /api/files/{id}/download, never direct S3 links
  • Image processing: Avatar uploads auto-resized via imaginary (HTTP service) to 256x256 WebP
  • Purpose-based validation: avatar/logo have 2MB limit + image-only types; others 10MB default
  • Object key format: tenants/{tenant_id}/{purpose}/{entity_id}{ext}

Notification Logs

Every notification sent by the platform is automatically logged. Multi-channel from day one.

CREATE TABLE notification_logs (
    id UUID PRIMARY KEY,
    tenant_id UUID REFERENCES tenants(id),
    channel TEXT NOT NULL DEFAULT 'email',      -- 'email', future: 'sms', 'push'
    recipient TEXT NOT NULL,
    template_slug TEXT,
    subject TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'sent',        -- 'sent', 'failed'
    error TEXT,
    provider_id TEXT,                           -- SMTP2GO email_id, future: Twilio SID
    triggered_by TEXT,                          -- 'registration', 'verification', 'team_invite', etc.
    user_id UUID REFERENCES users(id),
    metadata JSONB NOT NULL DEFAULT '{}',
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Logging is transparent via the LoggingEmailService decorator pattern. All callers of EmailService::send() get automatic audit logging without code changes. Fire-and-forget via tokio::spawn to avoid blocking delivery.

Multitenancy Model

Each tenant is fully isolated with its own resources:

Example Tenant: Acme Capital

ResourceItemsDetails
Users51 owner, 1 admin, 3 users (via user_tenants)
API Keys2qk_live_abc... (admin), qk_live_xyz... (read-only)
Agents4research-analyst, due-diligence-specialist, memo-writer, reviewer
Pipelines1deal-pipeline: Research -> DD -> Memo -> Review
Workloads2Acme Corp Acquisition (4 tasks, 3 artifacts), Beta Inc Analysis (2 tasks, 1 artifact)
Flows1Intake triage flow (8 nodes, 3 executions)
Conversations12Direct agent chats by team members
Knowledge Graph--1,234 SPO nodes, 5,678 triples, 89 documents (43 approved, 6 pending)
Files155 avatars, 2 logos, 8 attachments
Email Templates3Custom verification, invite, and welcome templates

Row-level security enforced at the database layer. The user_tenants junction table determines which tenants a user can access.

Example: Investment Banking Pipeline

Putting it all together with a concrete example:

Pipeline: "Deal Pipeline" with 4 stages

Agents:

  • Research Analyst: Gathers market data, competitive analysis (system_prompt focused on sourcing and synthesis)
  • Due Diligence Specialist: Validates claims, checks risks (system_prompt focused on verification and risk assessment)
  • Memo Writer: Synthesizes findings into investment memo (system_prompt focused on clear financial writing)
  • Reviewer: Final quality check (system_prompt focused on completeness and accuracy)

Workload: "Acme Corp Acquisition Analysis"

Stage 1: Research
  -> Research Analyst executes with forwarded context: {}
  -> Output: { market_size: "$50B", competitors: [...], key_trends: [...] }
  -> Artifact: "Acme Corp Market Analysis" (markdown)
  -> stage_results["Research"] recorded

Stage 2: Due Diligence
  -> Due Diligence Specialist executes with forwarded context from Research
  -> Output: { risk_score: 0.3, red_flags: [], verified_claims: 12 }
  -> Artifact: "Acme Corp Risk Assessment" (markdown)
  -> stage_results["Due Diligence"] recorded

Stage 3: Investment Memo
  -> Memo Writer executes with forwarded context from Research + DD
  -> Output: { recommendation: "proceed", confidence: 0.85 }
  -> Artifact: "Acme Corp Investment Memo" (pdf, via artifact template + renderer)
  -> stage_results["Investment Memo"] recorded

Stage 4: Review
  -> Reviewer executes with forwarded context from all prior stages
  -> Output: { approved: true, comments: "Strong analysis, proceed to IC" }
  -> Artifact: "Review Sign-off" (markdown)
  -> Workload status -> completed

Each stage builds on the structured output of previous stages. The artifact template for the investment memo ensures consistent formatting across all deals.