Domain Model
Core Concepts
| Concept | Definition | Example |
|---|---|---|
| Tenant | Organization with isolated data | "Acme Capital Partners" |
| Agent | Named specialist with a persona and system prompt | "Research Analyst" |
| Pipeline | Configurable sequence of stages with draft/live versioning | "Deal Pipeline" |
| Workload | Unit of work that moves through pipeline stages | "Acme Corp Acquisition" |
| Task | Agent assignment to a workload at a stage | "Research Analyst -> Research stage" |
| Artifact | Output produced by an agent, optionally from a template | "Market Analysis Report.pdf" |
| Flow | Visual graph-based workflow with branching and human gates | "Intake Triage Flow" |
| Conversation | Direct agent chat interface with switchable agents | "Research chat with Analyst" |
| Document | Source material with human-gated ingestion approval | "Acme Corp 10-K Filing" |
Entity Relationships
Understanding the entity hierarchy is crucial for working with Qadra. Every entity belongs to a tenant, and relationships enforce data integrity.
Tenant
├── Users (via user_tenants junction)
│ ├── is_super_admin (platform-level flag)
│ ├── email_verified
│ └── avatar_url
├── API Keys
├── Team Invites
├── Files (S3/MinIO metadata)
├── Email Templates (override global defaults)
├── Notification Logs (multi-channel audit)
├── Agents[]
├── Pipelines[]
│ ├── stages (live JSONB)
│ └── draft_stages (in-progress edits)
├── Workloads[]
│ ├── pipeline_id -> Pipeline
│ ├── pipeline_snapshot (immutable copy at creation)
│ ├── stage_results (accumulated JSONB)
│ ├── Tasks[]
│ │ └── agent_id -> Agent
│ ├── Artifacts[]
│ │ ├── task_id -> Task
│ │ └── template_id -> ArtifactTemplate
│ └── Usage Records[]
├── Artifact Templates[]
│ └── renderer_id -> Plugin
├── Agent Flows[]
│ └── Flow Executions[]
├── Conversations[]
│ └── Messages[]
│ ├── agent_id -> Agent
│ └── flow_execution_id -> FlowExecution
├── Documents[] (human-gated ingestion)
│ ├── proposed_by -> User
│ ├── approved_by -> User
│ ├── DocumentTriples[] -> SPO Triples
│ └── DocumentChunks[] -> Embeddings
├── SPO Index
│ ├── Nodes[]
│ ├── Triples[]
│ └── Entity Aliases[]
├── Plugins[]
├── Renderers[]
├── BALLS Graphs[]
└── Memory Store[]
Key Relationships Explained
Tenant -> Everything: The tenant is the root of all data. Every table has a tenant_id foreign key. You cannot query across tenants, period.
Users -> Tenants (many-to-many): Users belong to tenants through the user_tenants junction table. Each membership carries a per-tenant role (owner, admin, user) and an is_default flag indicating which tenant loads on login. A user's email is globally unique.
Super Admin: The is_super_admin boolean on the users table is a platform-level flag, not a per-tenant role. Super admins can create tenants, drop into any tenant (bypasses membership), reset passwords, and manage other super admins. The first super admin is promoted via direct SQL.
Pipeline -> Stages: Pipelines define the workflow structure. Stages are stored as JSONB within the pipeline. Pipelines support draft_stages for editing without affecting running workloads, and a pipeline_snapshot is captured on each workload at creation time.
Workload -> Pipeline: A workload references a pipeline and snapshots its stages at creation. This ensures running workloads are unaffected by subsequent pipeline edits.
Workload -> Stage Results: Completed stages record structured results in workloads.stage_results JSONB, keyed by stage name. Downstream stages consume this as forwarded context containing task summaries, merged outputs, and artifact references.
Task -> Agent: Each task is assigned to exactly one agent. The agent performs the work. Multiple tasks in a stage might use the same or different agents.
Task -> Artifacts: Artifacts are outputs. One task might produce multiple artifacts. Artifacts can optionally reference an artifact_template for structured output formatting.
Flow -> Executions: Agent flows are visual graph definitions stored as ReactFlow JSONB. Each execution is a state machine (pending -> running -> paused -> completed/failed/cancelled) with a blackboard state dictionary, per-node states, and an execution queue.
Conversation -> Messages: Conversations are direct agent chat sessions. Messages reference the agent that produced them (switchable per message) and optionally link to a flow execution.
Document -> Triples + Chunks: Documents go through a human-gated approval flow before ingestion. Once approved, the ingestion pipeline extracts SPO triples and/or vector chunks, linking them back to the source document for provenance.
Agents
Agents are the workers in Qadra. Unlike chatbots that respond to prompts, agents are specialists with defined personas and system prompts that perform tasks within workflows and conversations.
Agent Model
Agents have been refactored from the atomics model (JavaScript programs in QuickJS) to a pure persona model. Agents are LLM personas with system prompts, not code artifacts.
Persona fields -- what defines the agent:
name: Internal identifier ("research-analyst")display_name: Human-readable name ("Research Analyst")description: What this agent does in plain languageavatar_url: Profile imagesystem_prompt: The LLM system prompt that defines behavior and expertise
Orchestration fields -- how the agent participates in workflows:
control_type: What happens after the agent completes (retain,relinquish_to_parent,relinquish_to_start)output_visibility: Whether output is shown to users (user_facing) or kept internal (internal)max_calls_per_parent_agent: Safety rail limiting recursive invocations (default: 3)
Agent Schema
CREATE TABLE agents (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
name TEXT NOT NULL,
version INTEGER DEFAULT 1,
-- Persona
display_name TEXT,
avatar_url TEXT,
description TEXT,
system_prompt TEXT NOT NULL DEFAULT '',
-- Orchestration
control_type control_type NOT NULL DEFAULT 'relinquish_to_parent',
output_visibility output_visibility NOT NULL DEFAULT 'user_facing',
max_calls_per_parent_agent INTEGER NOT NULL DEFAULT 3,
-- Metadata
is_active BOOLEAN DEFAULT true,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(tenant_id, name, version)
);
Control Types
Control types define what happens to execution flow after an agent completes its work:
| Type | Behavior | Use Case |
|---|---|---|
retain | Agent keeps control, can be called again | Conversational agents, iterative research |
relinquish_to_parent | Returns control to the calling agent or orchestrator | Most agents -- do your job, hand back |
relinquish_to_start | Returns control to the pipeline entry point | Terminal agents that complete a branch |
Output Visibility
| Visibility | Behavior | Use Case |
|---|---|---|
user_facing | Output displayed to the end user | Final deliverables, reports, answers |
internal | Output kept within orchestration, hidden from user | Intermediate reasoning, data gathering |
Example Agent
{
"name": "research-analyst",
"display_name": "Research Analyst",
"description": "Gathers market data and competitive analysis for deal evaluation",
"avatar_url": "/avatars/analyst.png",
"system_prompt": "You are a senior research analyst at an investment bank. Your role is to gather and synthesize market data, competitive intelligence, and industry trends. Always cite sources. Structure your output with clear sections: Market Overview, Competitive Landscape, Key Trends, and Risk Factors.",
"control_type": "relinquish_to_parent",
"output_visibility": "user_facing",
"max_calls_per_parent_agent": 3
}
Why system prompts instead of JavaScript code? The original atomics model required agents to be JavaScript programs executed in QuickJS sandboxes. In practice, agent behavior is better expressed as natural language instructions to an LLM. System prompts are easier to write, test, and iterate on. The orchestration layer handles execution coordination; agents focus on expertise.
Versioning still works the same way. You might have research-analyst v1 and v2 with different system prompts but the same persona. Users see "Research Analyst" throughout -- the upgrade is invisible.
Canonical SPO Pipeline Personas
After the SPO-architecture decision (Story 98), four seeded agent presets form the canonical investment pipeline. Each writes to or reads from the SPO knowledge graph, so claims stay grounded and traceable as a deal moves through the stages.
| Persona | Role | Behavior |
|---|---|---|
| Scarlet | SPO claim writer | Extracts claims and writes them as SPO triples with provenance (asserter, source, assertion_type, confidence). Output is {"triples":[…],"summary":…} JSON. |
| Ayana | Investment filter | Reads Scarlet's SPO triples, applies investment filters, and routes deals PASS/CONDITIONAL/FAIL with a pre-screen score. |
| Eliza | Due-diligence writer | Writes due-diligence findings back to the SPO store, grounded in Scarlet's triples. |
| Reagan | Senior Investment Reviewer | Synthesizes upstream outputs into a final review (Recommendation, Conviction, Key Positives, Key Risks, Reviewer Notes). |
These are the canonical personas following the SPO-architecture decision -- Scarlet establishes grounded facts, Ayana and Eliza consume and extend them, and Reagan produces the human-facing recommendation.
Agent CRUD
Agents are managed through the gateway HTTP API, which routes to Rust Core via NATS.
NATS subjects: qadra.{tenant_id}.agent.do.{operation} where operation is create, get, list, update, or delete.
| Method | Path | NATS Subject | Description |
|---|---|---|---|
POST | /agents | qadra.{tid}.agent.do.create | Create a new agent |
GET | /agents | qadra.{tid}.agent.do.list | List agents (paginated) |
GET | /agents/:agentId | qadra.{tid}.agent.do.get | Get agent by ID |
PUT | /agents/:agentId | qadra.{tid}.agent.do.update | Update agent fields |
DELETE | /agents/:agentId | qadra.{tid}.agent.do.delete | Delete agent |
Create request fields (all except name are optional with defaults):
| Field | Type | Default | Validation |
|---|---|---|---|
name | string | required | 1-200 chars |
display_name | string | null | max 200 chars |
avatar_url | string | null | valid URL, max 2048 chars |
description | string | null | max 2000 chars |
system_prompt | string | "" | no limit |
control_type | enum | relinquish_to_parent | retain, relinquish_to_parent, relinquish_to_start |
output_visibility | enum | user_facing | user_facing, internal |
max_calls_per_parent_agent | integer | 3 | 1-100 |
Update semantics: The gateway uses a sparse update pattern. Only fields explicitly included in the request body are sent to Rust Core. Missing fields are not updated. Fields set to null clear the value. This means you can update just the system prompt without touching other fields.
Agent Editor (UI)
The agent editor (web/src/pages/AgentEditor.tsx) provides a full editing interface organized into three panels:
- Persona panel: Display name, description. These define the agent's identity as seen by users.
- System Prompt panel: A monospace textarea for the LLM system prompt. This is the core of the agent -- the instructions that define behavior when invoked by a flow or pipeline. Supports Ctrl/Cmd+S to save.
- Behavior panel: Control type (dropdown), output visibility (dropdown), and max calls per parent (number input). These govern how the agent participates in orchestration.
The top toolbar shows the agent name (click to rename inline) and a save button with status indicator (saved/error). The editor loads the agent by ID from the URL param and populates all fields from the store.
Pipelines
Pipeline Schema
CREATE TABLE pipelines (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
name TEXT NOT NULL,
description TEXT,
stages JSONB NOT NULL DEFAULT '[]', -- Live stage definitions
draft_stages JSONB, -- In-progress edits (NULL = no draft)
max_handoffs_per_turn INTEGER NOT NULL DEFAULT 25, -- Loop prevention
is_active BOOLEAN DEFAULT true,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(tenant_id, name)
);
Draft/Live Versioning
Pipelines support editing without affecting running workloads:
stages: The live configuration. Running workloads use this (via snapshot).draft_stages: An in-progress edit. NULL means no active draft.- Publishing: Overwrites
stageswithdraft_stages, then clears the draft. - Discarding: Sets
draft_stagesto NULL, leavingstagesuntouched.
When a workload is created, the pipeline's current stages are copied into workloads.pipeline_snapshot. This makes the workload immune to subsequent pipeline edits.
Safety Rails
max_handoffs_per_turn(pipeline-level, default 25): Maximum agent-to-agent handoffs per orchestration turn. Prevents infinite loops.max_calls_per_parent_agent(agent-level, default 3): Maximum times a parent agent can invoke a specific child agent per turn. Prevents recursive abuse.
Stage Definition
{
"stages": [
{
"name": "Research",
"order": 1,
"description": "Gather market data and competitive analysis"
},
{
"name": "Due Diligence",
"order": 2,
"description": "Validate claims and assess risks"
},
{
"name": "Investment Memo",
"order": 3,
"description": "Synthesize findings into recommendation"
},
{
"name": "Review",
"order": 4,
"description": "Final quality check"
}
]
}
Workloads
Workloads are the central concept in Qadra -- they represent units of work that progress through pipeline stages. In investment banking terms, a workload is a deal. In IT terms, it might be a ticket or project.
Unlike chat threads that accumulate messages, workloads accumulate structured context as agents complete tasks. This context flows forward, enabling downstream agents to build on previous work.
Workload Schema
CREATE TABLE workloads (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
pipeline_id UUID REFERENCES pipelines(id),
title TEXT NOT NULL,
description TEXT,
status workload_status DEFAULT 'pending', -- pending, active, completed, failed, cancelled
current_stage TEXT,
current_stage_order INTEGER,
metadata JSONB DEFAULT '{}',
context JSONB DEFAULT '{}', -- Accumulated context from completed tasks
pipeline_snapshot JSONB, -- Immutable copy of pipeline stages at creation
stage_results JSONB NOT NULL DEFAULT '{}', -- Per-stage structured results
started_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
Workload Lifecycle
Workloads follow a strict state machine. Understanding these states is essential for building UIs and debugging issues.
+-------------+ +-------------+ +-------------+
[*] --> | pending | --> | active | --> | completed | --> [*]
+-------------+ +------+------+ +-------------+
|
v
+-------------+
| failed | --> [*]
+-------------+
State Descriptions:
| State | Meaning | What's Happening |
|---|---|---|
pending | Created but not started | Waiting for user to initiate processing |
active | Processing in progress | Tasks are running, stages advancing |
completed | All stages finished successfully | All artifacts produced, work is done |
failed | Unrecoverable error occurred | A task failed and couldn't be retried |
cancelled | Manually cancelled | User stopped processing |
State Transitions:
| From | To | Trigger | Notes |
|---|---|---|---|
| pending | active | Execution starts | Creates tasks for first stage, snapshots pipeline |
| active | active | Stage advancement | All stage tasks complete -> next stage |
| active | completed | All stages complete | Final stage tasks done |
| active | failed | Task failure (no retry) | Retry exhausted or unretryable error |
| active | cancelled | User cancellation | Manual stop |
Stage Results and Context Forwarding
Completed pipeline stages produce structured results that flow to downstream stages. This replaces the simple context accumulation model with a richer structure.
Stage Result structure:
{
"Research": {
"stage_name": "Research",
"stage_order": 1,
"tasks": [
{
"task_id": "uuid",
"agent_id": "uuid",
"stage": "Research",
"output": { "market_size": "$50B", "competitors": ["CompA", "CompB"] },
"artifacts": [
{ "id": "uuid", "name": "Market Analysis", "artifact_type": "document", "content_type": "text/markdown", "size_bytes": 4200 }
],
"completed_at": "2026-01-24T10:00:00Z"
}
],
"merged_output": { "market_size": "$50B", "competitors": ["CompA", "CompB"] },
"completed_at": "2026-01-24T10:00:00Z"
}
}
Key types:
| Type | Contents | Purpose |
|---|---|---|
| ArtifactRef | {id, name, artifact_type, content_type, size_bytes} | Lightweight pointer, no content blob |
| TaskSummary | {task_id, agent_id, stage, output, artifacts[], completed_at} | Task + its artifact references |
| StageResult | {stage_name, stage_order, tasks[], merged_output, completed_at} | Complete stage record |
Forwarded context is built by build_forwarded_context() and contains {prior_stages, merged_outputs, artifact_refs} for downstream stage consumption.
Why references only? Artifact references store metadata, not content. Full content stays in the artifacts table. This keeps stage_results lightweight while providing enough information for downstream agents to know what's available.
Tasks
Task Schema
CREATE TABLE tasks (
id UUID PRIMARY KEY,
workload_id UUID NOT NULL REFERENCES workloads(id),
agent_id UUID NOT NULL REFERENCES agents(id),
stage TEXT NOT NULL,
stage_order INTEGER NOT NULL,
status task_status DEFAULT 'pending', -- pending, queued, running, completed, failed, cancelled
progress INTEGER DEFAULT 0, -- 0-100
input JSONB DEFAULT '{}',
output JSONB,
error TEXT,
queued_at TIMESTAMPTZ,
started_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
Task Execution Flow
The Python Agent orchestrator (WorkloadOrchestrator) drives task execution. Data operations go to Rust Core via NATS.
1. Client sends qadra.{tenant}.workload.execute (NATS)
2. Python Agent creates workload -> Rust Core (workload.do.create)
3. For each pipeline stage (sequential):
a. get_forwarded_context -> Rust Core (prior stage outputs, artifact refs)
b. LLM selects best agent for stage (based on agent description/persona)
c. create_task -> Rust Core (agent assigned to stage)
d. LLM executes with agent system prompt + forwarded context
e. complete_task -> Rust Core (output stored)
f. create_artifact -> Rust Core (if output > 200 chars)
g. record_stage_result -> Rust Core (persist to stage_results JSONB)
h. advance_stage -> Rust Core (move to next stage)
4. Return WorkloadResult (success, stages_completed, artifacts)
Progress updates flow through Redis pub/sub to WebSocket connections, giving users real-time visibility into long-running tasks.
Artifacts
Artifact Schema
CREATE TABLE artifacts (
id UUID PRIMARY KEY,
workload_id UUID NOT NULL REFERENCES workloads(id),
task_id UUID REFERENCES tasks(id),
agent_id UUID NOT NULL REFERENCES agents(id),
template_id UUID REFERENCES artifact_templates(id), -- Optional template
name TEXT NOT NULL,
artifact_type TEXT NOT NULL, -- 'document', 'analysis', 'memo', 'report', etc.
content_type TEXT, -- MIME type
content TEXT, -- Inline for small artifacts
file_path TEXT, -- Storage path for large artifacts
size_bytes BIGINT,
metadata JSONB DEFAULT '{}',
created_at TIMESTAMPTZ DEFAULT NOW()
);
Artifact Types
| Type | Description | Example |
|---|---|---|
document | Written content | Research report |
analysis | Structured analysis | Risk assessment |
memo | Investment memo | IC presentation |
report | Formatted report | Due diligence summary |
data | Structured data | Financial model |
Artifact Templates
Artifact templates define the expected structure and output format for artifacts. They give agents a blueprint: what sections to produce, what schema to follow, and optionally which renderer plugin transforms the output into a final format (PDF, HTML, etc.).
Template Schema
CREATE TABLE artifact_templates (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
name TEXT NOT NULL,
display_name TEXT,
description TEXT,
version INTEGER NOT NULL DEFAULT 1,
-- Template structure
artifact_type TEXT NOT NULL DEFAULT 'document', -- 'document', 'analysis', 'memo', 'report', etc.
expected_sections JSONB NOT NULL DEFAULT '[]', -- Section definitions (see below)
output_schema JSONB, -- JSON Schema for structured output validation
output_format TEXT NOT NULL DEFAULT 'markdown', -- 'markdown', 'html', 'pdf', 'json'
-- Renderer binding (optional)
renderer_id UUID REFERENCES plugins(id) ON DELETE SET NULL,
-- Metadata
is_active BOOLEAN NOT NULL DEFAULT true,
is_global BOOLEAN NOT NULL DEFAULT false,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(tenant_id, name, version)
);
-- Artifacts can optionally reference a template
ALTER TABLE artifacts ADD COLUMN template_id UUID REFERENCES artifact_templates(id) ON DELETE SET NULL;
Indexes: Tenant-scoped indexes on artifact_type and active templates (partial index on is_active = true).
Creating a Template
A template requires a name and defaults to artifact type document with markdown output. All other fields are optional.
| Field | Type | Default | Description |
|---|---|---|---|
name | string | required | Internal identifier (1-200 chars, unique per tenant+version) |
display_name | string | null | Human-readable name (max 200 chars) |
description | string | null | What this template is for (max 2000 chars) |
artifact_type | string | "document" | Category: document, analysis, memo, report, etc. |
expected_sections | array | [] | Section definitions (see below) |
output_schema | object | null | JSON Schema for validating structured output |
output_format | enum | "markdown" | markdown, json, html, pdf |
renderer_id | UUID | null | Optional reference to a renderer plugin |
is_active | boolean | true | Whether this template is available for use |
is_global | boolean | false | Whether this template is available to all tenants |
Renderer binding: When renderer_id is set, it references a plugin (from the plugins table) that transforms the agent's raw output into the target output_format. For example, a renderer plugin might convert markdown content into a styled PDF. If no renderer is bound, the raw output is stored as-is. Deleting the referenced plugin sets renderer_id to NULL (ON DELETE SET NULL).
Expected Sections
The expected_sections JSONB array defines the structure an agent should follow when producing an artifact from this template. Each section is an object:
[
{
"name": "Executive Summary",
"description": "High-level overview of findings and recommendation",
"required": true
},
{
"name": "Market Analysis",
"description": "Market size, growth trends, competitive landscape",
"required": true
},
{
"name": "Risk Factors",
"description": "Key risks and mitigation strategies",
"required": true
},
{
"name": "Appendix",
"description": "Supporting data tables and references",
"required": false
}
]
| Field | Type | Default | Description |
|---|---|---|---|
name | string | required | Section heading |
description | string | optional | Guidance for what this section should contain |
required | boolean | true | Whether the agent must produce this section |
Sections are passed to the agent as part of the prompt context. The agent's output is expected to follow this structure, though enforcement depends on the orchestration layer.
Using Templates
When an artifact references a template_id, the template's expected_sections and output_schema guide the agent's output:
- During task execution: The orchestrator includes the template's section definitions in the agent's prompt context, instructing it to produce output matching the template structure.
- After generation: If
output_schemais set, the output can be validated against the JSON Schema. - Rendering: If
renderer_idis set, the renderer plugin transforms the raw output into the targetoutput_format(e.g., markdown to PDF). - Storage: The resulting artifact is stored in the
artifactstable withtemplate_idset, linking it back to the template for provenance.
In the investment banking example, the "Investment Memo" artifact at Stage 3 uses a template that ensures every memo has an Executive Summary, Market Analysis, Risk Factors, and Recommendation section -- consistent formatting across all deals.
Artifact Template API
Templates are managed through the gateway HTTP API, routed to Rust Core via NATS.
NATS subjects: qadra.{tenant_id}.artifact_template.do.{operation} where operation is create, get, list, update, or delete.
| Method | Path | NATS Subject | Description |
|---|---|---|---|
POST | /artifact-templates | qadra.{tid}.artifact_template.do.create | Create a new template |
GET | /artifact-templates | qadra.{tid}.artifact_template.do.list | List templates (paginated, filterable by artifact_type) |
GET | /artifact-templates/:id | qadra.{tid}.artifact_template.do.get | Get template by ID |
PUT | /artifact-templates/:id | qadra.{tid}.artifact_template.do.update | Update template fields |
DELETE | /artifact-templates/:id | qadra.{tid}.artifact_template.do.delete | Delete template |
List query params: limit (1-100, default 20), offset (default 0), artifact_type (optional filter).
Artifact Template Editor (UI)
The template editor (web/src/pages/ArtifactTemplateEditor.tsx) provides four panels:
- Template Info: Display name, description, artifact type (dropdown), and output format (dropdown).
- Expected Sections: A dynamic list where you add, remove, and reorder sections. Each section has a name field, optional description, and a required checkbox.
- Output Schema: A monospace JSON textarea for defining a JSON Schema. Validated on save -- invalid JSON prevents saving.
The editor supports Ctrl/Cmd+S to save and shows inline save status (saved/error). The name is editable inline from the top toolbar.
Usage Records
Every LLM call, embedding generation, tool invocation, plugin execution, and renderer execution is tracked per-operation:
CREATE TABLE usage_records (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
workload_id UUID REFERENCES workloads(id),
task_id UUID REFERENCES tasks(id),
agent_id UUID REFERENCES agents(id),
model TEXT,
tokens_in INTEGER NOT NULL DEFAULT 0,
tokens_out INTEGER NOT NULL DEFAULT 0,
operation_type usage_operation_type NOT NULL DEFAULT 'llm_call',
metadata JSONB NOT NULL DEFAULT '{}',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
Operation types: llm_call, embedding, tool_call, plugin_exec, renderer_exec.
Usage records enable aggregate queries (SUM tokens per tenant/workload), time-range filtering via indexed created_at, and batch inserts via UNNEST. Stored in a separate table rather than JSONB inside tasks because aggregate queries across workloads would require scanning all tasks and extracting nested arrays.
Agent Flows
Agent flows provide visual, graph-based workflows with branching, conditions, loops, and human-in-the-loop gates.
Flow Definition Schema
CREATE TABLE agent_flows (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
name TEXT NOT NULL,
description TEXT,
flow_data JSONB NOT NULL DEFAULT '{"nodes":[],"edges":[],"viewport":{"x":0,"y":0,"zoom":1}}',
node_count INTEGER NOT NULL DEFAULT 0,
is_active BOOLEAN NOT NULL DEFAULT true,
created_by UUID REFERENCES users(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(tenant_id, name)
);
flow_data stores the ReactFlow graph definition as JSONB: {nodes, edges, viewport}. The executor always loads the full graph, never queries individual nodes. JSONB stores it verbatim with no impedance mismatch.
Flow Execution Schema
CREATE TABLE flow_executions (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
flow_id UUID NOT NULL REFERENCES agent_flows(id),
flow_snapshot JSONB NOT NULL, -- Immutable copy at execution start
status flow_execution_status NOT NULL DEFAULT 'pending',
state JSONB NOT NULL DEFAULT '{}', -- Blackboard dictionary
node_states JSONB NOT NULL DEFAULT '{}', -- Per-node execution state
execution_queue JSONB NOT NULL DEFAULT '[]',
current_node_id TEXT,
error TEXT,
triggered_by UUID REFERENCES users(id),
input JSONB NOT NULL DEFAULT '{}',
started_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
Flow Execution Lifecycle
stateDiagram-v2
[*] --> pending
pending --> running : execute
running --> completed : End node reached
running --> failed : node error / safety limit
running --> cancelled : user cancels
running --> paused : HumanInput node
paused --> running : user submits input
paused --> cancelled : user cancels
| Status | Meaning |
|---|---|
pending | Created but not started |
running | Actively executing nodes |
paused | Waiting for human input at a HumanInput node |
completed | End node reached |
failed | Node error or safety limit exceeded |
cancelled | User cancelled |
Node Types
The flow executor supports 14 node types. Each node stores its configuration in the data JSONB field of the ReactFlow node object. All nodes that produce output can specify an output_key in their data; after execution, the output is merged into the flow state dictionary under that key.
Start
Entry point for the flow. Exactly one per flow.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
Behavior: Pass-through. The execution input (provided when starting the flow) becomes the initial state. All successors are enqueued immediately. No output_key -- state is seeded from execution input directly.
End
Terminal node. At least one per flow. When reached, the execution status becomes completed.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
output_keys | string[] | Optional. If set, only these keys from state are included in the final output. If omitted, the entire state dictionary is the output. |
Behavior: Collects final output. If output_keys is specified, filters the state to only those keys. Otherwise returns the full state. No successors are enqueued.
Agent
Invokes an LLM with the agent's system prompt and a rendered prompt template. This is the primary intelligence node.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
agent_id | UUID string | References an agent in the agents table. The agent's system_prompt is loaded and used as the LLM system message. |
prompt_template | string | Instructions for this step. Supports {{variable}} template substitution from state. |
output_key | string | State key where the result is stored. |
Behavior: Renders prompt_template by replacing {{key}} placeholders with values from the flow state. Loads the agent's system_prompt by agent_id. Injects upstream state data as a second system message so the agent has context from prior nodes. Calls the LLM with temperature: 0.7, max_tokens: 4096. Output is a JSON object {response, model, tokens}. All successors are enqueued.
Condition
Boolean branching node. Evaluates an expression against the flow state and routes to the matching branch.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
expression | string | Expression to evaluate. Supports: key == value, key != value, or key (truthy check). |
mode | string | "expression" (default) or "llm" (LLM-based evaluation, placeholder). |
Expression evaluation rules:
key == value: Comparesstate[key]againstvalue. Strings, numbers, and booleans are compared by string representation. Returns route"true"or"false".key != value: Inverse of==. Returns"true"or"false".key(bare): Truthy check.truefor non-empty strings, any number,truebooleans, non-empty arrays/objects.falsefor empty strings,null, missing keys,falsebooleans.
Edge behavior: Outgoing edges MUST use sourceHandle to indicate which route they belong to. An edge with sourceHandle: "true" is followed when the expression evaluates to true; sourceHandle: "false" for the false branch. Only the matching branch is enqueued. Nodes on the non-matching branch (and their entire downstream subtree) are marked Skipped, unless they are also reachable from the taken branch.
HumanInput
Pauses the entire flow execution and waits for a user to submit input via the API.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
prompt | string | Message shown to the user describing what input is needed. |
timeout_seconds | number | Optional. Not currently enforced. |
output_key | string | State key where the user's response is stored after resumption. |
Behavior: Immediately returns NodeResult::Paused. The node status is set to WaitingForInput and the execution status is set to Paused. State and queue are persisted to the database. When the user submits input via POST /flows/executions/:execId/input, the NATS handler stores the response in the node's output, marks the node as Completed, merges the response into state under output_key, re-enqueues successors, and resumes the execution loop.
Loop
Iterates over a collection in the flow state. Currently a simplified batch processor -- it reads the array, caps it to max_iterations, and outputs the items. It does NOT re-execute successor nodes per item.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
collection_key | string | State key containing the array to iterate over. Default: "items". |
max_iterations | number | Maximum items to process. Default: 100. |
Behavior: Reads state[collection_key] as an array. If missing or not an array, outputs {iterations: 0, results: []}. Otherwise caps at max_iterations and outputs {iterations, results}. All successors are enqueued once (not per item).
Http
Makes an outbound HTTP request. Supports GET, POST, PUT, PATCH, DELETE.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
method | string | HTTP method. Default: "GET". |
url | string | Required. Supports {{variable}} template substitution from state. |
headers | object | Optional. Key-value map of request headers. |
body | JSON | Optional. Request body for POST/PUT/PATCH (sent as JSON). |
output_key | string | State key where the response is stored. |
Behavior: Templates the URL by replacing {{key}} with state values. Sends the request with a 30-second timeout. Output is {status, body} where body is parsed as JSON if possible, otherwise stored as a string. All successors are enqueued.
ExecuteFlow
Triggers another flow as a sub-flow. Currently a stub -- returns a placeholder message.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
target_flow_id | UUID string | The flow to execute. |
input_mapping | string | How to map current state to sub-flow input. |
Behavior: Returns {message: "Nested flow execution not yet implemented"}. All successors are enqueued. Future implementation will create a child FlowExecution, run it to completion, and merge its output back into the parent state.
Transform
Applies a data transformation by mapping state fields to new output fields.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
mappings | object | Key-value map where keys are output field names and values are either state field names (string) to copy, or literal JSON values. |
output_key | string | State key where the result is stored. |
Behavior: For each entry in mappings: if the value is a string, it is treated as a state key reference and the corresponding state value is copied. If the value is any other JSON type, it is used as a literal. Output is a JSON object with the mapped fields. All successors are enqueued.
Example: {"mappings": {"company": "company_name", "ready": true}} copies state.company_name into output.company and sets output.ready to true.
Delay
Waits for a specified duration before continuing. Capped at 300 seconds.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
duration_seconds | number | Seconds to wait. Default: 1. Maximum: 300 (5 minutes). |
Behavior: Sleeps for min(duration_seconds, 300) seconds using tokio::time::sleep. Output is {delayed_seconds}. All successors are enqueued after the delay.
KnowledgeLookup
Queries the tenant's SPO knowledge graph for relevant triples.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
query | string | Search query. Supports {{variable}} template substitution. |
source | string | "spo", "vector", or "both". Default: "both". Currently queries SPO regardless of this value. |
output_key | string | State key where results are stored. Default: "knowledge". |
limit | number | Maximum results. Default: 10. |
Behavior: Renders the query template with state variables. Searches SPO nodes matching the query text via search_spo_nodes, then fetches triples connected to those nodes via get_triples_for_nodes. Deduplicates by triple ID. Output is {query, source, limit, results} where results is an array of {subject, predicate, object, confidence, source} objects. Gracefully returns empty results on error. All successors are enqueued.
Research
Performs external research via LLM. Designed for Exa neural search integration (not yet wired -- currently uses LLM knowledge as a stand-in).
| Field | Type | Description |
|---|---|---|
label | string | Display name |
query | string | Research query. Supports {{variable}} template substitution. |
max_results | number | Maximum research results. Default: 3. |
output_key | string | State key where findings are stored. Default: "research_result". |
Behavior: Renders the query template. Calls the LLM with a research analyst system prompt and the rendered query plus workflow context (truncated to 4000 chars). Output is {query, findings, model, source: "llm_knowledge"}. All successors are enqueued.
Artifact
Produces a structured artifact, optionally formatted against an artifact template.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
template_id | UUID string | Optional. References an artifact_templates row. Provides section structure, output schema, and format guidance. |
artifact_name | string | Name for the artifact. Supports {{variable}} template substitution. Default: "Untitled Artifact". |
content_key | string | State key containing upstream content to format. If empty or null, the full state is used as context. |
output_key | string | State key where the artifact is stored. Default: "artifact". |
Behavior: Four modes depending on available inputs:
- No upstream content, no template: Calls LLM to generate content from the full state context.
- No upstream content, with template: Calls LLM with template section guidance to generate structured content.
- Upstream content, with template: Calls LLM to reformat the upstream content into the template structure.
- Upstream content, no template: Uses the upstream content as-is (no LLM call).
Output is {artifact_name, template_id, artifact_type, output_format, content}. All successors are enqueued.
Extract
Extracts structured parameters from text using an LLM. Typically placed immediately after the Start node to parse user intent into typed variables that downstream nodes reference via {{variable_name}}.
| Field | Type | Description |
|---|---|---|
label | string | Display name |
input_key | string | State key containing the raw text to extract from. Default: "user_message". |
output_variables | string | Comma-separated list of variable names to extract (e.g., "company_name, metric, time_period"). |
instructions | string | Optional. Additional extraction instructions for the LLM. |
Behavior: Reads state[input_key] as the raw text. If empty, sets all output variables to null. Otherwise, calls the LLM (temperature: 0.0, max_tokens: 1024) with a prompt asking it to extract the named variables as a JSON object. Strips markdown fences from the response if present. Parses the JSON and maps each requested variable into the output. If the LLM returns non-JSON, gracefully sets all variables to null. The output is a flat object -- each extracted variable becomes a top-level key. Unlike other nodes, Extract merges its output variables directly into state (each variable becomes a state key), making them available to all downstream nodes via {{variable_name}}.
Note: Extract nodes do NOT use output_key. Instead, each extracted variable is merged individually into the flow state. This is because the node returns a flat object (not nested under a key), and the executor's output-merge logic writes each top-level key into state.
AI Flow Generation
Flows can be generated from natural language descriptions using an LLM. The flow generator (src/flow_generator.rs) translates user prompts into valid ReactFlow graph definitions.
How It Works
- User submits a prompt via
POST /flows/generatewith a natural language description of the desired workflow. - Gateway forwards the request to Rust Core via NATS subject
qadra.{tid}.flow.do.generate. - Rust Core loads context: active agents and artifact templates for the tenant.
- LLM generates the graph: A system prompt describes all 14 node types with their data field schemas, edge format, layout rules, and structural constraints. The user message includes the available agents (with IDs, names, descriptions) and artifact templates, followed by the user's request.
- Validation: The generated JSON is validated for structural correctness.
- Position assignment: If nodes lack positions, a top-to-bottom grid layout is applied automatically.
- Result returned: The validated
flow_dataJSONB is returned to the gateway, ready to be saved as a flow.
Node Catalog
The node catalog (NODE_CATALOG in flow_generator.rs) is the single source of truth for node types. It defines the type_name, description, data_example, and notes for each node type. The LLM system prompt, validation logic, and frontend palette all derive from this catalog. Adding a new node type means adding one NodeSpec entry.
LLM Prompt Structure
The system prompt instructs the LLM to return raw JSON (no markdown fences) with this structure:
{
"nodes": [
{
"id": "{type}_{unique_number}",
"type": "{node_type}",
"position": { "x": number, "y": number },
"data": { "label": "Display Name", ...type_specific_fields }
}
],
"edges": [
{
"id": "e_{source_id}-{target_id}",
"source": "{source_node_id}",
"target": "{target_node_id}",
"animated": true,
"sourceHandle": "true|false"
}
],
"viewport": { "x": 0, "y": 0, "zoom": 1 }
}
Layout rules: Start node at (400, 50), 200px vertical spacing, 300px horizontal spacing for parallel branches. Condition edges use sourceHandle of "true" or "false".
LLM parameters: temperature: 0.3, max_tokens: 4096.
Validation
validate_flow_data() checks:
nodesandedgesarrays exist and are non-empty- Exactly 1
startnode - At least 1
endnode - Every node type is in the catalog (rejects unknown types)
- Every edge references valid source and target node IDs
If the LLM wraps its response in markdown code fences despite instructions, they are stripped before parsing.
Position Assignment
ensure_positions() checks whether any node is missing a position or has a position without x/y. If so, all nodes receive a simple top-to-bottom grid layout starting at (400, 50) with 200px vertical spacing.
Flow Execution Engine
The event-driven graph executor in src/flow_executor.rs processes flow graphs. It is a Rust in-process executor that shares the PostgreSQL connection pool and NATS handlers with the rest of Rust Core.
Execution Loop
1. Load execution record (must be status=Pending)
2. Parse graph from flow_snapshot (immutable copy)
3. Initialize: all nodes -> Pending, seed queue with Start node
4. Set execution status -> Running
5. LOOP:
a. Pop node_id from front of queue
b. Check all predecessors are Completed or Skipped (join gate)
- If not all done: re-enqueue and continue
c. Set node status -> Running, record started_at, snapshot state as input
d. Persist state to database (crash recovery point)
e. Emit "node_started" progress event via SSE channel
f. Execute the node (dispatch by type)
g. Handle result:
- Completed: mark Completed, merge output into state via output_key,
enqueue successors (respecting condition routing), skip non-taken branches
- Paused: mark WaitingForInput, persist state, set execution -> Paused, RETURN
- Failed: mark Failed, set execution -> Failed with error, RETURN
h. Emit "node_completed" or "node_failed" progress event
6. Queue empty: set execution status -> Completed, persist final state
State Merging
The flow state is a JSONB blackboard dictionary shared by all nodes. State flows through the graph as follows:
- Execution input seeds the initial state (e.g.,
{"user_message": "Analyze NVIDIA"}) - After each node completes, if the node has an
output_keyin its data, the node's output is stored asstate[output_key] = output - Extract nodes are special: they return a flat object of extracted variables, and each variable becomes an individual state key (no nesting under output_key)
- Each node receives the full state as its input. Nodes read from state via template substitution (
{{key}}) or direct key access - State is never pruned -- it accumulates throughout the execution. All prior node outputs remain available to all downstream nodes
Predecessor Join Gate
Before executing a node, the executor checks that ALL predecessor nodes (based on incoming edges) have status Completed or Skipped. This implements an implicit AND-join for nodes with multiple incoming edges. If predecessors are not yet done, the node is re-enqueued at the back of the queue.
Condition Routing and Branch Skipping
When a Condition node completes, it returns a route value ("true" or "false"). The executor then:
- For each outgoing edge, checks if the edge's
sourceHandlematches the route - Matching edges: target nodes are enqueued
- Non-matching edges: the
skip_subtree()function marks the target node and its entire downstream subtree asSkipped(via depth-first traversal), unless a node is also reachable from the taken branch
This ensures that only the relevant branch executes, and nodes on dead branches are marked Skipped rather than left Pending.
HumanInput Pause and Resume
Pause:
- HumanInput node immediately returns
NodeResult::Paused - Node status set to
WaitingForInput - Full execution state (state dict, node_states, queue) persisted to database
- Execution status set to
Paused - Executor returns -- the execution is now dormant
Resume (triggered by POST /flows/executions/:execId/input):
- NATS handler stores user input in the waiting node's
outputfield FlowExecutor::resume()is called- Loads persisted state from database
- Finds the
WaitingForInputnode, marks itCompleted - Merges the node's output into state under its
output_key - Re-enqueues the node's successors
- Sets execution status to
Running - Re-enters the same execution loop as
run()
Nested ExecuteFlow
The ExecuteFlow node type is currently a stub. It returns {message: "Nested flow execution not yet implemented"} and enqueues successors normally. The intended future behavior: load the target flow by target_flow_id, create a child FlowExecution, run it synchronously, and merge its final state back into the parent flow's state.
Safety Rails
| Rail | Limit | Behavior on Violation |
|---|---|---|
MAX_TOTAL_STEPS | 500 node executions per run | Execution fails with error |
| Loop max iterations | 100 items | Array truncated silently |
| Delay max duration | 300 seconds (5 min) | Duration capped silently |
| HTTP timeout | 30 seconds per request | Node fails with error |
MAX_TOTAL_STEPS is checked on every iteration of the execution loop (both initial run() and resume()). It counts total node executions, not unique nodes -- a loop that re-enqueues nodes can exhaust this limit.
State Persistence and Crash Recovery
The executor persists the full execution state to PostgreSQL after every node execution via update_execution_state(). This writes the current state, node_states, execution_queue, and current_node_id to the flow_executions row. If the Rust process crashes mid-execution, the persisted state contains enough information to determine which nodes completed and where the queue was, though automatic crash recovery is not currently implemented.
Progress Events
The executor optionally emits real-time progress events via an mpsc::UnboundedSender<FlowProgressEvent> channel (attached via with_progress()). Events are fire-and-forget and never block execution. Three event types:
| Event | Data | When |
|---|---|---|
node_started | {node_id, node_type, label} | Before node execution |
node_completed | {node_id, output_preview (200 char max), duration_ms} | After successful execution |
node_failed | {node_id, error} | After failed execution |
These events are streamed to the frontend via SSE for live flow execution visualization.
Conversations
Conversations provide a direct agent chat interface, separate from pipeline-based workloads. While workloads move structured data through pipeline stages, conversations are freeform -- a user talks to agents, switches between them, and can even trigger flow executions inline.
Conversation Model
A conversation belongs to a single user within a tenant. It holds an ordered sequence of messages, each of which records the agent that responded (if any), the LLM model used, performance metrics, and optional links to flow executions.
CREATE TABLE conversations (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE SET NULL,
title TEXT, -- Auto-generated or user-set
is_archived BOOLEAN NOT NULL DEFAULT false,
deleted_at TIMESTAMPTZ, -- Soft delete (filtered by default)
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE conversation_messages (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
conversation_id UUID NOT NULL REFERENCES conversations(id) ON DELETE CASCADE,
agent_id UUID REFERENCES agents(id) ON DELETE SET NULL,
role TEXT NOT NULL CHECK (role IN ('user', 'assistant', 'system')),
content TEXT NOT NULL,
model TEXT, -- LLM model identifier
token_count INTEGER, -- Total tokens consumed
latency_ms INTEGER, -- End-to-end response time
finish_reason TEXT, -- 'stop', 'length', 'tool_calls', etc.
rating SMALLINT CHECK (rating IN (-1, 1)), -- Thumbs up/down
flow_execution_id UUID REFERENCES flow_executions(id) ON DELETE SET NULL,
metadata JSONB NOT NULL DEFAULT '{}',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
Indexes:
idx_conversations_tenant-- tenant-scoped listing, excludes soft-deleted (WHERE deleted_at IS NULL)idx_conversations_tenant_user-- per-user listing within a tenant, excludes soft-deletedidx_conversation_messages_conv_created-- ordered message retrieval by conversationidx_conversation_messages_agent-- find all messages by a specific agentidx_conversation_messages_rating-- query rated messages for feedback analysis
Relationship to other entities:
- Each
conversation_messages.agent_idpoints to the agent that produced the assistant response. User messages haveagent_id = NULL. conversation_messages.flow_execution_idlinks to aflow_executionsrow when the message was produced by running a flow.conversation_messages.metadatais a JSONB bag that carriesartifacts(inline content blocks for rendering) andsuggested_actions(follow-up prompts the UI can present).
Agent Switching
Unlike most chat interfaces where the "assistant" is a single model, Qadra conversations let users switch agents on every message. The agent_id lives on conversation_messages, not on conversations.
How it works:
- The user selects an agent in the UI (agent picker dropdown).
- When sending a message, the selected
agent_idis included in the request body. - The backend invokes the chosen agent's system prompt + conversation history to generate a response.
- Both the user message and assistant message are persisted. The assistant message records the
agent_idthat produced it. - The UI auto-selects the last responding agent for the next message, but the user can switch at any time.
This means a single conversation can contain messages from multiple agents. For example, a user might ask the Research Analyst about market data, then switch to the Due Diligence Specialist to probe risk factors -- all within the same conversation thread.
Flow Tool-Use
Agents can trigger flow executions from within a conversation. When an agent determines that a user's request maps to an existing flow, the backend executes the flow and streams progress events back to the user in real time.
How flow integration works:
- The
send_messagehandler on the Rust Core evaluates whether the agent should invoke a flow (based on the agent's tool configuration and the conversation context). - If a flow is triggered, the executor runs the flow graph and publishes progress events to a NATS progress subject.
- The gateway forwards these NATS events as SSE events to the client (see SSE Streaming below).
- When the flow completes, the agent receives the flow's output state and synthesizes a final response.
- The assistant message is persisted with
flow_execution_idset, linking the conversational response to the full flow execution record.
The frontend renders flow progress inline in the chat: a live visualization of node execution with status indicators, output previews, and duration metrics.
SSE Streaming
The POST /conversations/:id/messages/stream endpoint uses Server-Sent Events to deliver real-time progress during message processing. This is a POST endpoint (not GET) because it sends the user's message in the request body and streams the response.
Connection setup:
- The gateway sets SSE headers (
Content-Type: text/event-stream,Cache-Control: no-cache,Connection: keep-alive,X-Accel-Buffering: no). - A unique NATS progress subject (
_PROGRESS.{uuid}) is created for this request. - The gateway subscribes to the progress subject, then sends the NATS request to Rust Core with the
progress_subjectfield set. - Rust Core publishes
FlowProgressEventmessages ({event, data, seq}) to the progress subject during processing. - The gateway forwards each event to the SSE stream.
Event types and their data payloads:
| Event | Data | When |
|---|---|---|
user_saved | {user_message} | User message persisted to DB (replaces optimistic message) |
thinking | {status} | Agent is processing (e.g. "Thinking...") |
flow_started | {flow_name, execution_id} | A flow execution has begun |
node_started | {node_id, node_type, label} | A flow node started executing |
node_completed | {node_id, output_preview, duration_ms} | A flow node finished successfully |
node_failed | {node_id, error} | A flow node failed |
flow_completed | {} | Flow execution completed |
flow_failed | {error} | Flow execution failed |
assistant_message | {user_message, assistant_message, conversation} | Final response with both persisted messages and updated conversation |
error | {message} | An error occurred |
done | {} | Stream is complete, client should close connection |
Wire format (standard SSE):
event: thinking
data: {"status":"Thinking..."}
event: flow_started
data: {"flow_name":"Intake Triage","execution_id":"abc-123"}
event: node_started
data: {"node_id":"node-1","node_type":"Agent","label":"Classify Request"}
event: node_completed
data: {"node_id":"node-1","output_preview":"Category: Research","duration_ms":1200}
event: assistant_message
data: {"user_message":{...},"assistant_message":{...},"conversation":{...}}
event: done
data: {}
Timeouts and fallbacks:
- The NATS request has a 240-second timeout (flow executor 180s + follow-up LLM ~30s + buffer).
- The gateway has a 250-second overall safety timeout on the SSE stream.
- If the progress events do not deliver an
assistant_messagebefore the NATS reply arrives, the gateway uses the NATS reply as a fallback and writes it as anassistant_messageSSE event. - If the SSE stream fails entirely, the frontend falls back to the non-streaming
POST /conversations/:id/messagesendpoint.
Message Rating
Users can rate individual assistant messages with thumbs up (+1) or thumbs down (-1). Ratings are stored directly on the conversation_messages row.
- Endpoint:
POST /conversations/:id/messages/:messageId/rate - Body:
{ "rating": 1 }or{ "rating": -1 } - NATS subject:
qadra.{tenant_id}.chat.do.rate_message - Constraint:
ratingcolumn isSMALLINT CHECK (rating IN (-1, 1)). Only-1and1are valid. - Idempotent: Rating the same message again overwrites the previous rating.
- Index:
idx_conversation_messages_ratingenables efficient queries on rated messages for feedback analysis and model evaluation.
Conversation Lifecycle
Create -- POST /conversations
Creates an empty conversation. Title is optional (can be auto-generated later from the first message).
Send message -- POST /conversations/:id/messages (non-streaming) or POST /conversations/:id/messages/stream (SSE)
Sends a user message and receives the agent's response. The request body includes agent_id (required), content (required, max 100,000 characters), and optionally flow_execution_id. Both endpoints persist the user message and assistant message, returning them along with the updated conversation record.
List conversations -- GET /conversations?limit=20&offset=0
Returns ConversationSummary objects with message_count, last_message_preview, and last_agent_id for sidebar display. Excludes soft-deleted conversations.
Get conversation -- GET /conversations/:id
Returns the full conversation record.
List messages -- GET /conversations/:id/messages?limit=100&offset=0
Returns messages ordered by created_at. Used on page load and as a recovery mechanism when SSE streaming fails.
Update title -- PATCH /conversations/:id
Updates the conversation title.
Delete -- DELETE /conversations/:id
Soft-deletes the conversation by setting deleted_at. The conversation and its messages are preserved for audit but excluded from all listing queries.
NATS Subjects
All conversation operations go through tenant-scoped NATS subjects:
| Operation | NATS Subject |
|---|---|
| Create | qadra.{tid}.chat.do.create |
| Get | qadra.{tid}.chat.do.get |
| List | qadra.{tid}.chat.do.list |
| Update title | qadra.{tid}.chat.do.update_title |
| Delete | qadra.{tid}.chat.do.delete |
| Send message | qadra.{tid}.chat.do.send_message |
| List messages | qadra.{tid}.chat.do.list_messages |
| Rate message | qadra.{tid}.chat.do.rate_message |
Key Design Decisions
agent_idon messages, not conversations: Users can switch agents mid-conversation. One message might come from the Research Analyst, the next from the Due Diligence Specialist. The conversation itself is agent-agnostic.flow_execution_idon messages: Messages can be produced by flow executions, linking conversational output to the full flow execution record with its node states and blackboard data.- Soft delete with
deleted_at: Conversations are never hard-deleted. Settingdeleted_atexcludes them from listing queries while preserving the audit trail. metadataJSONB bag: Carriesartifacts(inline content blocks like charts or agent-produced documents) andsuggested_actions(follow-up prompts). Extensible without schema migration.- SSE over WebSocket for streaming: SSE is simpler to implement (one-way server-to-client), works through proxies/CDNs, and naturally maps to the request-response pattern of sending a message and streaming the response. WebSocket would be overkill for this use case.
Document Ingestion
Documents go through a human-gated approval flow before knowledge ingestion. Agents can propose documents, but humans must approve before ingestion runs.
Document Schema
CREATE TABLE documents (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
title TEXT,
content TEXT NOT NULL,
content_type TEXT DEFAULT 'text/plain',
source_url TEXT,
source_type TEXT, -- 'upload', 'web', 'api', 'research'
-- Ingestion lifecycle
ingestion_status TEXT DEFAULT 'pending_approval',
ingestion_result JSONB,
-- Proposal gate
proposed_by UUID REFERENCES users(id), -- Who submitted for review
approved_by UUID REFERENCES users(id), -- Human who opened the gate
approved_at TIMESTAMPTZ,
rejected_by UUID REFERENCES users(id), -- Who rejected (if rejected)
rejected_at TIMESTAMPTZ,
rejection_reason TEXT,
created_at TIMESTAMPTZ DEFAULT NOW(),
processed_at TIMESTAMPTZ
);
Document Lifecycle
pending_approval --> approved --> processing --> completed
--> failed
pending_approval --> rejected
| Status | Meaning |
|---|---|
pending_approval | Proposed but awaiting human review |
approved | Human approved, ready for ingestion |
processing | Ingestion pipeline running (routing, extraction, embedding) |
completed | Triples and/or chunks created |
failed | Ingestion error |
rejected | Human rejected the proposal |
Why human-gated? Agents can autonomously discover and propose documents for ingestion, but a human must approve before content enters the knowledge graph. This prevents polluting the KG with low-quality or irrelevant content.
Ingestion Routing
Once approved, the ingestion pipeline routes content:
| Route | Action | When |
|---|---|---|
| SPO | Extract triples only | Short structured content (facts, definitions) |
| Vector | Chunk and embed only | Long narrative (articles, reports) |
| Both | Extract triples AND chunk | Mixed content |
| Discard | Skip (boilerplate, junk) | Low-value content |
Results link back to the document:
document_triples: Which SPO triples came from this documentdocument_chunks: Which vector chunks came from this document
Users and Authentication
User Schema
CREATE TABLE users (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id), -- Legacy, kept for backward compat
email TEXT NOT NULL,
email_verified BOOLEAN NOT NULL DEFAULT false,
password_hash TEXT NOT NULL,
name TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'user',
avatar_url TEXT,
is_super_admin BOOLEAN NOT NULL DEFAULT false,
settings JSONB DEFAULT '{}',
last_login_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
Multi-Tenant Memberships
CREATE TABLE user_tenants (
user_id UUID NOT NULL REFERENCES users(id),
tenant_id UUID NOT NULL REFERENCES tenants(id),
role TEXT NOT NULL DEFAULT 'user', -- 'owner', 'admin', 'user'
is_default BOOLEAN NOT NULL DEFAULT false,
joined_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (user_id, tenant_id)
);
- Globally unique email: One account per email, belonging to multiple tenants
- Per-tenant roles:
owner(immutable),admin,user is_default: Which tenant loads on login- Registration: Atomic transaction creates tenant + user + user_tenants in one operation
Email Verification
CREATE TABLE email_verification_tokens (
id UUID PRIMARY KEY,
user_id UUID NOT NULL REFERENCES users(id),
token TEXT NOT NULL UNIQUE, -- 32 random bytes, hex-encoded (64 chars)
expires_at TIMESTAMPTZ NOT NULL, -- 24h expiry
used_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
- Token validated + not used + not expired -> mark
used_at, setemail_verified = true - Post-registration: fire-and-forget async task (doesn't block registration response)
- Resend: invalidates existing tokens, creates new, sends new email
Team Invites
CREATE TABLE team_invites (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
email TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'user',
invited_by UUID NOT NULL REFERENCES users(id),
token TEXT NOT NULL UNIQUE, -- 32 random bytes, hex-encoded, 7-day expiry
expires_at TIMESTAMPTZ NOT NULL,
accepted_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
Smart invite logic: If a user with the email already exists globally, they are added to the tenant directly. If not, an invite is created and an email is sent. Only owner/admin can invite, change roles, or remove members.
Email Templates
CREATE TABLE email_templates (
id UUID PRIMARY KEY,
tenant_id UUID REFERENCES tenants(id), -- NULL = global default
slug TEXT NOT NULL, -- 'verification', 'team_invite', etc.
name TEXT NOT NULL,
subject TEXT NOT NULL, -- Handlebars: 'Verify your email, {{user_name}}'
html_body TEXT NOT NULL, -- Handlebars HTML template
text_body TEXT, -- Handlebars plain text fallback
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(tenant_id, slug)
);
- Template resolution: Tenant-specific wins over global (
ORDER BY tenant_id IS NULL ASC LIMIT 1) - Handlebars rendering:
{{var}}syntax for subject, html_body, text_body - Tenant overrides: Tenants can customize any global template by slug
Super Admin
The is_super_admin boolean on the users table is a platform-level privilege:
| Capability | Description |
|---|---|
| Create tenants | Create tenant + assign owner |
| Drop into any tenant | Synthetic JWT with target tenant context (bypasses membership) |
| Reset passwords | Force-reset any user's password |
| List all tenants/users | Platform-wide visibility |
| Promote/demote | Grant or revoke super admin (cannot self-promote/demote) |
The first super admin is promoted via direct SQL. JWT carries sa: true claim when present.
Files
File metadata is stored in PostgreSQL. Actual blobs live in S3-compatible object storage (MinIO by default, tenant-configurable).
CREATE TABLE files (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL REFERENCES tenants(id),
uploaded_by UUID NOT NULL REFERENCES users(id),
storage_provider TEXT NOT NULL DEFAULT 'internal', -- 'internal' (MinIO) or 's3'
bucket TEXT NOT NULL DEFAULT 'qadra-files',
object_key TEXT NOT NULL,
filename TEXT NOT NULL,
content_type TEXT NOT NULL,
size_bytes BIGINT NOT NULL,
purpose TEXT NOT NULL, -- 'avatar', 'artifact', 'logo', 'attachment'
entity_type TEXT, -- Polymorphic: 'user', 'tenant', 'workload'
entity_id UUID,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
- Gateway-proxied downloads: URLs are always
/api/files/{id}/download, never direct S3 links - Image processing: Avatar uploads auto-resized via imaginary (HTTP service) to 256x256 WebP
- Purpose-based validation:
avatar/logohave 2MB limit + image-only types; others 10MB default - Object key format:
tenants/{tenant_id}/{purpose}/{entity_id}{ext}
Notification Logs
Every notification sent by the platform is automatically logged. Multi-channel from day one.
CREATE TABLE notification_logs (
id UUID PRIMARY KEY,
tenant_id UUID REFERENCES tenants(id),
channel TEXT NOT NULL DEFAULT 'email', -- 'email', future: 'sms', 'push'
recipient TEXT NOT NULL,
template_slug TEXT,
subject TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'sent', -- 'sent', 'failed'
error TEXT,
provider_id TEXT, -- SMTP2GO email_id, future: Twilio SID
triggered_by TEXT, -- 'registration', 'verification', 'team_invite', etc.
user_id UUID REFERENCES users(id),
metadata JSONB NOT NULL DEFAULT '{}',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
Logging is transparent via the LoggingEmailService decorator pattern. All callers of EmailService::send() get automatic audit logging without code changes. Fire-and-forget via tokio::spawn to avoid blocking delivery.
Multitenancy Model
Each tenant is fully isolated with its own resources:
Example Tenant: Acme Capital
| Resource | Items | Details |
|---|---|---|
| Users | 5 | 1 owner, 1 admin, 3 users (via user_tenants) |
| API Keys | 2 | qk_live_abc... (admin), qk_live_xyz... (read-only) |
| Agents | 4 | research-analyst, due-diligence-specialist, memo-writer, reviewer |
| Pipelines | 1 | deal-pipeline: Research -> DD -> Memo -> Review |
| Workloads | 2 | Acme Corp Acquisition (4 tasks, 3 artifacts), Beta Inc Analysis (2 tasks, 1 artifact) |
| Flows | 1 | Intake triage flow (8 nodes, 3 executions) |
| Conversations | 12 | Direct agent chats by team members |
| Knowledge Graph | -- | 1,234 SPO nodes, 5,678 triples, 89 documents (43 approved, 6 pending) |
| Files | 15 | 5 avatars, 2 logos, 8 attachments |
| Email Templates | 3 | Custom verification, invite, and welcome templates |
Row-level security enforced at the database layer. The user_tenants junction table determines which tenants a user can access.
Example: Investment Banking Pipeline
Putting it all together with a concrete example:
Pipeline: "Deal Pipeline" with 4 stages
Agents:
- Research Analyst: Gathers market data, competitive analysis (
system_promptfocused on sourcing and synthesis) - Due Diligence Specialist: Validates claims, checks risks (
system_promptfocused on verification and risk assessment) - Memo Writer: Synthesizes findings into investment memo (
system_promptfocused on clear financial writing) - Reviewer: Final quality check (
system_promptfocused on completeness and accuracy)
Workload: "Acme Corp Acquisition Analysis"
Stage 1: Research
-> Research Analyst executes with forwarded context: {}
-> Output: { market_size: "$50B", competitors: [...], key_trends: [...] }
-> Artifact: "Acme Corp Market Analysis" (markdown)
-> stage_results["Research"] recorded
Stage 2: Due Diligence
-> Due Diligence Specialist executes with forwarded context from Research
-> Output: { risk_score: 0.3, red_flags: [], verified_claims: 12 }
-> Artifact: "Acme Corp Risk Assessment" (markdown)
-> stage_results["Due Diligence"] recorded
Stage 3: Investment Memo
-> Memo Writer executes with forwarded context from Research + DD
-> Output: { recommendation: "proceed", confidence: 0.85 }
-> Artifact: "Acme Corp Investment Memo" (pdf, via artifact template + renderer)
-> stage_results["Investment Memo"] recorded
Stage 4: Review
-> Reviewer executes with forwarded context from all prior stages
-> Output: { approved: true, comments: "Strong analysis, proceed to IC" }
-> Artifact: "Review Sign-off" (markdown)
-> Workload status -> completed
Each stage builds on the structured output of previous stages. The artifact template for the investment memo ensures consistent formatting across all deals.