Knowledge Graph

Overview

Qadra uses a dual-storage approach for knowledge retrieval, combining the precision of graph databases with the flexibility of semantic search.

The core insight: Different types of knowledge need different retrieval methods.

• Facts ("Apple was founded in 1976") --- Exact lookup via SPO triples
• Concepts ("What are the risks of this acquisition?") --- Semantic similarity via vectors

Why not just vectors? Vector search is powerful but imprecise. Ask "When was Apple founded?" and you might get chunks discussing Apple's founding, but not necessarily the exact year. SPO gives you (Apple, founded_in, 1976) directly.

Why not just SPO? Triple stores require structured data. For narrative content, exploratory queries, or fuzzy concepts, vectors find relevant context that wouldn't match any predefined pattern.

SPO Index

What is SPO?

SPO stands for Subject-Predicate-Object --- the fundamental unit of knowledge representation used in semantic web and knowledge graph systems.

Example Triple: (Apple, founded_by, Steve Jobs)

Why SPO over property graphs?

Property graphs (Neo4j, etc.) allow arbitrary properties on nodes and edges. SPO is more constrained --- everything is a triple. This constraint is a feature:

• Uniformity: All facts have the same shape, simplifying queries and storage
• Composability: Complex facts decompose into simple triples
• Interoperability: RDF/SPARQL compatibility if needed later

Example: Representing a complex fact

"Apple was founded by Steve Jobs in Cupertino in 1976"

Becomes multiple triples:

(Apple, founded_by, Steve Jobs)
(Apple, founded_in_location, Cupertino)
(Apple, founded_in_year, 1976)
(Steve Jobs, co_founded, Apple)

Each fact is independently queryable, updateable, and traceable to its source document.

Schema

-- Nodes store entities and relationships
CREATE TABLE spo_nodes (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    value TEXT NOT NULL,
    node_type TEXT NOT NULL,               -- 'subject', 'predicate', 'object'
    embedding vector(1536),                -- For semantic search
    content JSONB,                         -- Rich metadata
    source_document TEXT,                  -- Provenance
    confidence REAL DEFAULT 1.0,
    created_at TIMESTAMPTZ DEFAULT NOW(),

    UNIQUE(tenant_id, value, node_type)
);

-- Triples connect nodes
CREATE TABLE spo_triples (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL REFERENCES tenants(id),
    subject_id UUID NOT NULL REFERENCES spo_nodes(id),
    predicate_id UUID NOT NULL REFERENCES spo_nodes(id),
    object_id UUID NOT NULL REFERENCES spo_nodes(id),
    confidence REAL DEFAULT 1.0,
    source TEXT,                           -- "document:uuid", "user:uuid"
    asserter TEXT,                         -- Who made this claim
    assertion_type TEXT,                   -- 'fact', 'claim', 'opinion'
    created_at TIMESTAMPTZ DEFAULT NOW(),

    UNIQUE(tenant_id, subject_id, predicate_id, object_id)
);

-- All 6 SPO permutation indexes for O(1) lookups
CREATE INDEX idx_spo ON spo_triples(tenant_id, subject_id, predicate_id, object_id);
CREATE INDEX idx_sop ON spo_triples(tenant_id, subject_id, object_id, predicate_id);
CREATE INDEX idx_pso ON spo_triples(tenant_id, predicate_id, subject_id, object_id);
CREATE INDEX idx_pos ON spo_triples(tenant_id, predicate_id, object_id, subject_id);
CREATE INDEX idx_osp ON spo_triples(tenant_id, object_id, subject_id, predicate_id);
CREATE INDEX idx_ops ON spo_triples(tenant_id, object_id, predicate_id, subject_id);

-- HNSW index for embedding similarity on SPO nodes
CREATE INDEX idx_spo_nodes_embedding ON spo_nodes
    USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 64);

Query Patterns

SPO supports six fundamental query patterns, all scoped by tenant_id:

Examples:

-- SP?: What is Apple's founding year?
-- (Apple, founded_in_year, ?) -> ["1976"]
SELECT o.value FROM spo_nodes s
JOIN spo_triples t ON t.subject_id = s.id
JOIN spo_nodes p ON t.predicate_id = p.id
JOIN spo_nodes o ON t.object_id = o.id
WHERE t.tenant_id = $1 AND s.value = 'Apple' AND p.value = 'founded_in_year';

-- ?PO: Who founded Apple?
-- (?, founded_by, Apple) -> ["Steve Jobs"]
SELECT s.value FROM spo_nodes s
JOIN spo_triples t ON t.subject_id = s.id
JOIN spo_nodes p ON t.predicate_id = p.id
JOIN spo_nodes o ON t.object_id = o.id
WHERE t.tenant_id = $1 AND p.value = 'founded_by' AND o.value = 'Apple';

-- S?O: What is the relationship between Apple and Steve Jobs?
-- (Apple, ?, Steve Jobs) -> ["founded_by", "co_founded_by"]
SELECT p.value FROM spo_nodes s
JOIN spo_triples t ON t.subject_id = s.id
JOIN spo_nodes p ON t.predicate_id = p.id
JOIN spo_nodes o ON t.object_id = o.id
WHERE t.tenant_id = $1 AND s.value = 'Apple' AND o.value = 'Steve Jobs';

Predicate Design

Good predicates are meaningful relationships:

Rules:

• Predicates should be verbs or verb phrases
• Use snake_case: is_located_in, not isLocatedIn
• Be specific: founded_by not associated_with
• Avoid meta-predicates that describe the data, not relationships

The extraction pipeline enforces this with a junk-predicate filter that rejects is_about, key_point, research_finding, relates_to, has_aspect, mentioned_in, discussed, and noted.

Vector Store

pgvector with HNSW

All vector storage uses PostgreSQL pgvector with HNSW (Hierarchical Navigable Small World) indexes for fast approximate nearest-neighbor search.

CREATE TABLE embeddings (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL,
    chunk_id UUID NOT NULL,
    document_id UUID NOT NULL,
    embedding vector(1536) NOT NULL,
    content TEXT NOT NULL,
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_embeddings_embedding ON embeddings
    USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 64);

Two things get embedded:

1. Document chunks: Paragraph-boundary text chunks (~500 chars) for narrative retrieval
2. SPO node values: Subject/predicate/object values for semantic search on the knowledge graph itself

Semantic Search

#![allow(unused)]
fn main() {
let results = vector_index.search(
    tenant_id,
    query_embedding,
    k: 10,
    filter: None,
).await?;

for hit in results {
    println!("{}: {:.3}", hit.content, hit.score);
}
}

SPO navigates structure; pgvector retrieves similar content. Standard RAG searches everything; SPO-RAG searches where SPO points.

Document Ingestion Pipeline

Human-Gated Design

Ingestion into the Knowledge Core is explicitly human-gated. Agents can surface matching documents and propose them for ingestion, but a human must confirm before any data is written to the SPO graph or vector store.

Why: Prevents accidental data co-mingling across tenants or knowledge domains. We never want unintended data polluting the Knowledge Core.

How it works:

1. An agent discovers relevant content (via search, connector, or user upload)
2. The agent proposes the document for ingestion --- it appears in a review queue
3. A human reviews and approves (or rejects) the proposal
4. Only after explicit confirmation does the ingestion pipeline run

No automatic or background ingestion of discovered content. The UI presents proposed documents for review, and only after explicit confirmation does data enter the system.

Pipeline Overview

Once a document is approved, the pipeline transforms raw content into queryable knowledge:

Approved Document
    |
    v
1. Record document (status='processing')
    |
    v
2. Content Analyzer (LLM routing: Spo / Both / Discard)
    |
    +--- Discard ---> Mark discarded, stop
    |
    v
3a. Triple Extractor (LLM, epistemic metadata, windowed for large docs)
    |                --> batch-create SPO nodes + triples
    |                --> embed SPO node values into HNSW index
    |
3b. Text Chunker (if routing = Both)
    |                --> pure Rust, paragraph boundaries, ~500 chars
    |                --> batch embed chunks into pgvector
    |
    v
4. Provenance linking (document_triples, document_chunks)
    |
    v
5. Update document status='completed'

Key principle: SPO is always the index. Every non-discard document gets SPO triples extracted. There is no "vector-only" category. The Both routing adds vector chunks on top of SPO extraction for documents that also benefit from narrative retrieval.

Implementation: src/ingestion/ module, orchestrated by IngestionPipeline (implements the IngestionService trait).

Content Analyzer

The analyzer determines how to route content. It uses an LLM classifier with a heuristic fallback.

Routing categories:

There is no "vector-only" routing. If the LLM returns "vector", the analyzer upgrades it to "both" --- SPO extraction always runs on non-junk content.

LLM classification: The analyzer sends a 2000-character sample to a haiku-tier model with a routing prompt. The LLM returns JSON with routing, has_extractable_facts, estimated_chunks, confidence, and reasoning.

Heuristic fallback: When the LLM is unavailable or returns unparseable output, the analyzer falls back to length-based heuristics:

• Content < 500 chars --> Spo (SPO extraction only)
• Content >= 500 chars --> Both (SPO + vector chunks)

Implementation: src/ingestion/analyzer.rs

Triple Extraction

The extractor uses an LLM to pull Subject-Predicate-Object triples from text, including epistemic metadata on every triple.

Extraction prompt rules:

• Use meaningful predicates: founded_by, is_located_in, acquired, revenue_is
• Reject junk predicates: is_about, key_point, research_finding, relates_to
• Each triple must represent a concrete, verifiable relationship
• Include assertion type: fact, claim, or opinion
• Include the source excerpt for provenance

Windowed extraction for large documents: When content exceeds 2000 characters, the pipeline chunks it into extraction windows (~2000 chars with 100-char overlap) and runs extraction on each window independently. Results are deduplicated by (subject, predicate, object), keeping the highest-confidence version.

Post-extraction filtering:

• Triples below 0.5 confidence are discarded
• Triples with junk predicates are discarded
• Remaining triples are batch-created as SPO nodes and triples in PostgreSQL
• Subject node values are embedded into the HNSW index for semantic search on the KG itself

Example extraction:

Input: "Acme Corporation was founded in 1990 by John Smith.
        The company is headquartered in San Francisco."

Output:
[
  {
    "subject": "Acme Corporation",
    "predicate": "founded_in",
    "object": "1990",
    "confidence": 0.95,
    "assertion_type": "fact",
    "excerpt": "Acme Corporation was founded in 1990"
  },
  {
    "subject": "Acme Corporation",
    "predicate": "founded_by",
    "object": "John Smith",
    "confidence": 0.95,
    "assertion_type": "fact",
    "excerpt": "Acme Corporation was founded in 1990 by John Smith"
  },
  {
    "subject": "Acme Corporation",
    "predicate": "headquartered_in",
    "object": "San Francisco",
    "confidence": 0.90,
    "assertion_type": "fact",
    "excerpt": "The company is headquartered in San Francisco"
  }
]

Implementation: src/ingestion/extractor.rs

Text Chunking

Large content routed as Both gets split into overlapping chunks for vector embedding. The chunker is pure Rust --- no LLM calls.

Algorithm:

• Target chunk size: ~500 characters
• Overlap: ~50 characters (re-includes the tail of the previous chunk)
• Split on paragraph boundaries (double newlines)
• Each chunk tracks index, start_offset, and end_offset for provenance

The chunker accumulates paragraphs until the target size is exceeded, then flushes. Overlap is achieved by carrying the last paragraph(s) from the previous chunk into the next, up to the overlap budget.

Implementation: src/ingestion/chunker.rs

Embedding

The embedder handles two kinds of embedding in batch:

1. Document chunks --> embed_and_store_chunks: Batch-embeds chunk content, upserts EmbeddingRecord entries into the vector index with metadata (chunk_index, start_offset, end_offset)
2. SPO node values --> embed_spo_nodes: Batch-embeds subject node values, updates their embedding column in spo_nodes via VectorSearchRepository

Both operations use the EmbeddingProvider trait, which can be backed by OpenAI, local models, or any provider implementing the trait.

Implementation: src/ingestion/embedder.rs

Provenance Linking

All ingested content maintains full provenance --- every triple and every chunk links back to its source document.

-- Source documents
CREATE TABLE documents (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL,
    title TEXT,
    content TEXT NOT NULL,
    content_type TEXT,
    source_url TEXT,
    source_type TEXT,                      -- 'upload', 'web', 'api', 'research'
    ingestion_status TEXT,                 -- 'pending', 'processing', 'completed', 'discarded'
    ingestion_result JSONB,                -- Routing decision, counts, analysis
    created_at TIMESTAMPTZ,
    processed_at TIMESTAMPTZ
);

-- Links documents to extracted triples
CREATE TABLE document_triples (
    document_id UUID REFERENCES documents(id),
    triple_id UUID REFERENCES spo_triples(id),
    confidence REAL,
    excerpt TEXT,                          -- Source text the triple was extracted from
    PRIMARY KEY (document_id, triple_id)
);

-- Links documents to vector chunks
CREATE TABLE document_chunks (
    document_id UUID REFERENCES documents(id),
    chunk_id UUID,
    chunk_index INTEGER,
    start_offset INTEGER,
    end_offset INTEGER,
    PRIMARY KEY (document_id, chunk_id)
);

This enables:

• Tracing any triple back to its source document and excerpt
• Tracing any vector chunk back to its document and character offsets
• Answering "where did this fact come from?" for audit and compliance

Chain-of-Custody Provenance

Provenance linking answers which document a triple came from. Chain-of-custody extends this to a complete, queryable audit trail: every SPO triple resolves to its originating document along with when the source was published and when Qadra retrieved it.

Migration 035 adds custody columns:

-- On spo_triples: where this triple came from and when it was captured
ALTER TABLE spo_triples ADD COLUMN source_uri TEXT;
ALTER TABLE spo_triples ADD COLUMN retrieval_timestamp TIMESTAMPTZ;

-- On documents: when the source was published vs. when we ingested it
ALTER TABLE documents ADD COLUMN publication_date TIMESTAMPTZ;
ALTER TABLE documents ADD COLUMN retrieval_timestamp TIMESTAMPTZ;

The TripleProvenance type resolves the full chain in a single indexed join --- triple to its document, carrying title, source URL, publication date, retrieval timestamp, and the source excerpt:

#![allow(unused)]
fn main() {
pub struct TripleProvenance {
    pub triple_id: Uuid,
    pub source_uri: Option<String>,
    pub retrieval_timestamp: Option<DateTime<Utc>>,
    pub document_title: Option<String>,
    pub document_source_url: Option<String>,
    pub publication_date: Option<DateTime<Utc>>,
    pub document_retrieval_timestamp: Option<DateTime<Utc>>,
    pub excerpt: Option<String>,
}
}

NATS subject: qadra.{tid}.epistemic.trace_triple --- resolves the custody chain for a single triple. The single-join design keeps resolution under a 100 ms budget.

This answers not just "where did this fact come from?" but "how old is the source, and when did we last see it?" --- essential for time-sensitive financial knowledge where a stale figure is worse than no figure.

Claim Trace

Where chain-of-custody traces a single triple, claim trace runs the inverse direction for investment-committee (IC) sign-off: given a free-text claim from a memo (e.g. "Acme's 2024 revenue was $391B"), trace it back to the supporting SPO triples --- with their asserter and source --- and the source documents, including the excerpt and full content.

Flow: two indexed queries plus a reverse document join.

1. SPO node search --- match the claim text against SPO node values (subject/object) to find candidate nodes
2. Triple lookup --- fetch triples involving those nodes, carrying asserter and assertion_type
3. Reverse document join --- walk document_triples back to documents for each supporting triple, returning excerpt + content

The whole trace is budgeted under 500 ms (a 450 ms internal timeout guards the budget; partial results are returned rather than failing the IC review).

The response groups supporting triples (with asserter + source) under the claim and links each to its source documents (with excerpt + content), so a reviewer can verify a memo statement against the underlying knowledge in one round trip.

Epistemic Metadata

Not All Knowledge Is Equal

A fact from a regulatory filing differs from an opinion in a blog post. Qadra tracks who made a claim and its epistemic status on every SPO triple.

Assertion Types

The default assertion type is Claim (conservative). If the extractor is uncertain, it defaults to claim rather than fact.

Schema

Two columns on spo_triples:

ALTER TABLE spo_triples ADD COLUMN asserter TEXT;
ALTER TABLE spo_triples ADD COLUMN assertion_type TEXT
    CHECK (assertion_type IN ('fact', 'claim', 'opinion'));

Extraction

The triple extraction prompt explicitly asks the LLM to classify each triple's assertion type. The ExtractedTriple struct carries this through the pipeline:

#![allow(unused)]
fn main() {
pub struct ExtractedTriple {
    pub subject: String,
    pub predicate: String,
    pub object: String,
    pub confidence: f32,
    pub assertion_type: Option<AssertionType>,  // fact, claim, or opinion
    pub excerpt: Option<String>,
}
}

Query Filtering

Epistemic metadata enables filtering by trust level:

#![allow(unused)]
fn main() {
// Get only verified facts
let facts = repository
    .query_triples(tenant_id)
    .where_assertion_type(AssertionType::Fact)
    .fetch_all()
    .await?;

// Get claims from a specific analyst
let analyst_claims = repository
    .query_triples(tenant_id)
    .where_asserter("Goldman Sachs")
    .where_assertion_type(AssertionType::Claim)
    .fetch_all()
    .await?;
}

Claim Classification

Beyond the three-way assertion type (fact, claim, opinion), the ClaimClassifier in qadra-epistemic-core provides fine-grained classification of assertions into six epistemic categories.

ClaimType enum:

How classification works:

The classifier uses pre-compiled regex indicator patterns (word-boundary matching, inflection-aware) across four categories: projection, opinion, assessment, and fact. For each input text:

1. Score the text against each indicator set (count of matches / total indicators in set)
2. Apply priority ordering: projection > opinion > assessment > fact
3. The first category exceeding a 0.05 score threshold wins
4. Confidence is computed as 0.6 + (score * 0.4), capped at 1.0
5. Multiple fact indicators (score > 0.15) promote StatedFact to VerifiedFact
6. If no indicators match, default to StatedFact at 0.5 confidence

Response-level classification:

classify_response splits a full response into sentences (handling abbreviations like "Dr.", "U.S.", "e.g.") and classifies each sentence independently. This enables per-claim epistemic tracking within a single agent output.

Integration with groundedness validation:

The ClaimType maps to AssertionType for storage via to_assertion_type(). This bridges the fine-grained classification (useful for analysis and debugging) with the storage-level three-way system. During groundedness validation, claim classification helps distinguish which parts of a response need strict fact-checking (verified facts) versus which are inherently subjective (opinions, projections).

Customization:

The classifier supports custom indicator lists via builder methods (with_projection_indicators, with_opinion_indicators) for domain-specific vocabulary. For example, an investment banking pipeline might add financial projection indicators like "DCF implies" or "base case".

Implementation: crates/qadra-epistemic-core/src/claims.rs

Confidence Weighting

Epistemic status feeds into synthesis context weighting:

In synthesis prompts, facts and claims are separated:

VERIFIED FACTS:
(Apple, founded_in_year, 1976) [asserter: SEC filings]

ANALYST CLAIMS (verify independently):
(NVIDIA, will_reach, $5T_market_cap) [asserter: Morgan Stanley]

KG-SMILE Attribution

The Two Gates Model

Every agent response passes through two gates that determine whether the system has the competence to answer and whether the answer stayed grounded in known facts.

Query --> Extract Entities --> Competence Gate --> Synthesis --> Verification Gate --> Response
                                    |                                |
                             "Should I attempt?"            "Did I stay grounded?"

Based on the KG-SMILE paper --- explainable GraphRAG attribution.

Gate 1: Competence (Pre-Synthesis)

Question: "Does my knowledge domain cover this query?"

Before the agent synthesizes a response, the competence gate checks whether the SPO graph has meaningful coverage of the query's entities.

#![allow(unused)]
fn main() {
CompetenceResult {
    passed: bool,
    entities: Vec<String>,      // Extracted from query
    coverage: f32,              // Fraction with SPO triples (0-1)
    connectivity: f32,          // How connected in SPO graph (0-1)
    relevant_triples: Vec<Uuid>,
}
}

Thresholds:

• min_coverage: 0.3 --- At least 30% of query entities must have triples
• min_connectivity: 0.2 --- At least 20% of entity pairs must be connected

Failure modes:

• "Insufficient SPO coverage" --- Query about an unknown domain
• "Insufficient connectivity" --- Entities exist but aren't related in the graph

If the competence gate fails, the system can decline to answer, trigger research (see 5.9), or return a response with a low-confidence warning.

Gate 2: Verification (Post-Synthesis)

Question: "Does my output's semantic structure exist in my knowledge domain?"

After synthesis, the verification gate checks whether the response's implied facts are grounded in the SPO graph. Qadra ships three verifier implementations with different cost/precision trade-offs. Two of them — both LLM-free — run on every chat response in production; the perturbation-based attribution path is implemented but not yet wired into the live pipeline.

Both production gates write their result into the assistant message's metadata (metadata["verification"] and metadata["belnap"]) and always surface ungrounded or contradicted claims rather than silently dropping them. Both are a passing no-op when the query produced no context triples — this prevents false negatives when the knowledge graph has nothing relevant to say.

Implementation: crates/qadra-epistemic-core/src/attribution/verification.rs (rule-based + perturbation) and crates/qadra-epistemic-core/src/belnap.rs (Belnap).

Rule-Based Grounding Gate (production)

#![allow(unused)]
fn main() {
VerificationGateResult {
    passed: bool,
    fidelity: f32,
    faithfulness: f32,                   // fraction of implied triples that are grounded
    attributions: Vec<AttributedEdge>,
    ungrounded_claims: Vec<UngroundedClaim>,
    reason: Option<String>,
}
}

Process:

1. Extract implied triples from the response — split into sentences; detect a predicate keyword (e.g. "reported", "produces", "owns"); subject = noun phrase before the keyword, object = noun phrase after.
2. Classify each triple — compute word overlap with every context triple; SUPPORTED if any overlap exists, otherwise UNGROUNDED.
3. Faithfulness = supported / total implied (1.0 if none were extracted).
4. Gate decision — pass iff faithfulness >= min_faithfulness (default 0.5).
5. Always surface ungrounded_claims.

fidelity and stability are reported as rule-based proxies (fidelity mirrors faithfulness; stability is a determinism flag) — they become true R²/Jaccard only on the perturbation path below.

Belnap Four-Valued Gate (production)

A second, complementary gate from the qadra-belnap crate. It evaluates each claim over Belnap four-valued logic, where the graph can simultaneously support and contradict a claim, and "no evidence" is distinct from "false":

Pipeline (verify_with_belnap):

1. build_belnap_graph(triples) — each SPO triple becomes a polarity-tagged edge. Edge polarity is derived from the predicate string via predicate_polarity (a negation/contrast-marker heuristic) — v1. The research-faithful source is classify_polarity over the asserting excerpt computed at SPO write time; that is a tracked follow-up (see crates/qadra-belnap/MODEL.md).
2. Parse the response into claims (subject entity, predicate polarity) using the UD parser.
3. Ground each claim against the graph and reconcile the claim's polarity against the four-valued result.
4. passed is false iff the graph contradicts a claim. (Ungrounded/contested claims are reported but do not fail the gate.)

The UD parser comes from shared_parser(), a process-wide OnceLock. Under the onnx-ort Cargo feature it loads the real ONNX goeswith parser (KoichiYasuoka/roberta-base-english-ud-goeswith, run via ort) from QADRA_BELNAP_MODEL / QADRA_BELNAP_TOKENIZER / QADRA_BELNAP_LABELS. Without the feature it returns a StubParser whose empty parse makes the gate a passing no-op — so the default build is unchanged, and the Belnap gate only does real work when the feature is on and the model assets are present. BelnapVerification is serialized into metadata["belnap"] only when the response yields at least one claim.

Perturbation Attribution (future, higher-precision)

The verify_response path measures grounding by perturbation — removing subsets of triples, re-synthesizing, and measuring how much the response changes — then trains a weighted surrogate model to produce per-triple attribution scores (fidelity = R², faithfulness = correlation with actual impact, stability = Jaccard robustness). It is the highest-precision verifier but costs one LLM re-synthesis per perturbation, so it is not yet invoked by the live chat pipeline. Its distinguishing optimization — curvature-guided perturbation, which targets structurally important edges instead of 20 blind random removals — is described next.

Curvature-Guided Perturbation

The KG-SMILE paper recommends 20 random perturbations to measure how triple removal affects response quality. Random selection wastes LLM calls on redundant edges that change nothing. Qadra replaces random selection with Forman-Ricci curvature --- a discrete curvature measure computed directly on the knowledge graph --- to target perturbations at structurally important edges.

Implementation: crates/qadra-epistemic-core/src/curvature/

What Is Forman-Ricci Curvature

Forman-Ricci curvature is a discrete analogue of Ricci curvature from differential geometry, adapted for graphs. For an edge $e = (u, v)$, the formula is:

$$\kappa_F(e) = 4 - \deg(u) - \deg(v) + 3 \cdot |\triangle(u, v)|$$

Where:

• $\deg(u)$ is the degree of node $u$ (number of neighbors)
• $\deg(v)$ is the degree of node $v$
• $|\triangle(u, v)|$ is the number of triangles containing edge $e$ (i.e., common neighbors of $u$ and $v$)

Intuition: High-degree nodes pull curvature negative (the edge is a bottleneck funneling many paths). Triangles push curvature positive (the edge has parallel alternatives through shared neighbors).

Complexity: $O(|E| \cdot d_{\max})$ where $d_{\max}$ is the maximum node degree --- linear in graph size for bounded-degree graphs.

Worked examples:

Edge Classification

Curvature values partition edges into three structural roles:

The threshold $-0.1$ is configurable via CurvatureConfig::bridge_threshold.

graph LR
    subgraph "Cluster (kappa = 3)"
        A((A)) --- B((B))
        B --- C((C))
        C --- A
    end

    subgraph "Bridge (kappa = -3)"
        D((D)) --- H((Hub))
        E((E)) --- H
        F((F)) --- H
        G((G)) --- H
    end

    C -. "bridge edge<br/>kappa < -0.1" .-> D

    style H fill:#f96,stroke:#333

Structure analysis: Before choosing a sampling method, the system computes the percentage of bridge edges. If bridges make up at least 15% of all edges (min_bridge_percentage), curvature-guided sampling is recommended. Otherwise, the graph is too uniform and random sampling is used as a fallback.

Weight Computation

Raw curvature values are transformed into sampling probabilities through a shifted-inversion + normalization pipeline:

Step 1: Shifted inversion --- Flip the sign so negative curvature (bridges) maps to high weight:

$$w(e) = \max!\bigl(\varepsilon,; -\kappa_F(e) + \kappa_{\max} + 1\bigr)$$

Where $\kappa_{\max} = \max_e \kappa_F(e)$ and $\varepsilon$ is the weight floor (default $0.01$) ensuring every edge retains a nonzero sampling chance.

Step 2: Normalize to probabilities:

$$P(e) = \frac{w(e)}{\sum_{e'} w(e')}$$

Effect on sampling distribution:

When all edges have identical curvature (e.g., a uniform hub-spoke), all weights are equal and sampling degrades gracefully to uniform random.

Weighted sampling: Edges are drawn without replacement using the computed probabilities. The sampler iteratively selects an edge proportional to its probability, removes it from the pool, renormalizes, and repeats. This ensures each perturbation set contains unique edges.

How It Feeds Into Attribution

The curvature pipeline replaces random perturbation selection in the KG-SMILE verification gate:

flowchart TD
    A[SPO Triples for Query] --> B[Compute Forman-Ricci Curvature]
    B --> C{Bridge % >= 15%?}
    C -->|Yes| D[Curvature-Guided Sampling]
    C -->|No| E[Random Sampling Fallback]
    D --> F[Select Perturbation Sets]
    E --> F
    F --> G[Run Perturbations<br/>Remove selected edges, re-synthesize]
    G --> H[Measure Response Similarity]
    H --> I[Train Surrogate Model<br/>Weighted Linear Regression]
    I --> J[Attribution Scores per Triple]

Why this matters:

• Fewer perturbations needed: Bridge edges produce the strongest attribution signal. By sampling them preferentially, each perturbation is more informative. Where random sampling might need 20 perturbations to achieve stable attributions, curvature-guided sampling can achieve comparable fidelity with fewer runs.
• Each perturbation is more informative: Removing a bridge edge disconnects knowledge clusters, producing a measurable drop in response quality. Removing a cluster edge changes almost nothing --- it wastes an LLM call that produces near-zero signal.
• Cost reduction: Each perturbation requires an LLM re-synthesis call. Targeting structurally important edges reduces the total number of LLM calls needed for reliable attribution.
• Context edge selection: Beyond perturbation sampling, the curvature pipeline also selects which edges to include in the LLM context window. Bridge edges (sorted by curvature ascending, most negative first) are prioritized. If there are at least min_bridges_for_selection bridge edges, only bridges are included. Otherwise, edges below the mean curvature are used as a fallback. The result is capped at max_context_edges.

Configuration

All curvature behavior is controlled by CurvatureConfig:

Full pipeline entry point: compute_curvature(triples, config) runs the complete pipeline --- Forman-Ricci computation, structure analysis, weight transformation, and context edge selection --- returning a CurvatureResult with edges sorted by curvature ascending (bridges first).

Attribution Scoring

Triple attribution via weighted linear regression:

S_i = B_0 + B_1*W_1*P_1 + ... + B_k*W_k*P_k + e

Where:

• S_i = Response similarity for perturbation i
• P_k = Binary indicator (triple k present?)
• W_k = Kernel weight (Gaussian: exp(-(C_i)^2 / sigma^2))
• B_k = Attribution score (higher = more influential)

Configuration

Schema

-- Sessions track both gates
CREATE TABLE attribution_sessions (
    id UUID PRIMARY KEY,
    tenant_id UUID NOT NULL,
    query TEXT NOT NULL,
    competence_passed BOOLEAN,
    spo_coverage REAL,
    spo_connectivity REAL,
    verification_passed BOOLEAN,
    fidelity REAL,
    faithfulness REAL,
    stability REAL,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Triple-level attribution scores
CREATE TABLE triple_attributions (
    id UUID PRIMARY KEY,
    session_id UUID REFERENCES attribution_sessions(id),
    triple_id UUID REFERENCES spo_triples(id),
    attribution_score REAL,
    weight REAL,
    component_group TEXT
);

-- Raw perturbation data
CREATE TABLE perturbation_results (
    id UUID PRIMARY KEY,
    session_id UUID REFERENCES attribution_sessions(id),
    removed_triple_ids UUID[],
    graph_similarity REAL,
    response_similarity REAL
);

Attribution Session Persistence

Every gate run is durable, not just in-memory. Each call into handle_query persists a row to attribution_sessions and --- when triples were ranked --- batch-inserts the per-triple curvature/weight into triple_attributions.

The write happens via fire-and-forget (tokio::spawn), so query latency is unaffected and a failed insert never propagates to the caller.

Persisted on every gate run:

• query, tenant_id, agent name
• spo_coverage and spo_connectivity (the curvature analysis bridge percentage)
• competence_passed and competence timing
• per-triple attributions (weight = curvature)

Verification-side columns (fidelity, faithfulness, stability, verification_passed) are left NULL until the verification gate is invoked from a higher-level pipeline.

Querying persisted sessions:

The list endpoint is paginated and filterable by workload_id, agent_name, date range (from/to), and the gate-pass booleans (competence_passed, verification_passed). Getting a single session returns it together with its triple_attributions.

Metrics: each gate run emits a tracing event with the target kg_smile.gate_run, carrying gate, result, coverage, and connectivity. These flow through the standard OTLP --> OTel Collector --> Loki pipeline and feed a KG-SMILE Grafana dashboard that charts pass/fail rates and average coverage/connectivity over time --- no new exporter or metrics endpoint required.

Groundedness Validation

Purpose

Detect hallucinations by checking if claims in an agent's response can be traced back to the provided context. Groundedness validation complements the Two Gates model as a third layer of quality assurance.

Validation Flow

Agent Response
    |
    v
Extract Claims (LLM)
    "NVIDIA has market cap of $3T"
    "Revenue grew 200% YoY"
    |
    v
For each claim:
    --> Find supporting triple or chunk
    --> Check entailment (SUPPORTED / CONTRADICTED / NOT_MENTIONED)
    --> Score confidence
    |
    v
Aggregate: { grounded: bool, score: 0-1, ungrounded_claims: [] }

Scoring

#![allow(unused)]
fn main() {
pub struct ValidationResult {
    pub is_grounded: bool,
    pub score: f32,              // 0-1, fraction of claims grounded
    pub total_claims: usize,
    pub grounded_claims: usize,
    pub ungrounded_claims: Vec<String>,
}
}

Integration

Called after synthesis, before returning a response:

• If grounded: return as-is
• If partially grounded: attach warning listing ungrounded claims
• If poorly grounded: trigger gap detection and autonomous research (see 5.9), or regenerate with stricter prompt

Relationship to KG-SMILE

The three quality layers form a chain:

1. Competence Gate (pre-synthesis): Do we have knowledge about this topic?
2. Verification Gate (post-synthesis): Did we stay grounded in that knowledge?
3. Groundedness Validation (post-synthesis): Are specific claims individually supported?

Claim Validation Node

Groundedness validation runs inside a single agent's synthesis. Claim validation lifts the same idea into a flow graph: a ClaimValidation node grounds every claim produced by prior-stage agents against the SPO triples in the Knowledge Core.

This is the cross-stage check for pipelines where one agent's output feeds another --- for example, validating Scarlet's research claims and Ayana's analysis against known facts before a downstream stage builds on them.

What it does:

1. Read the prior-stage agent outputs from the flow state (the input_keys)
2. Extract individual claims from each output
3. Ground each claim against SPO triples in the Knowledge Core
4. Label each claim SUPPORTED, CONTRADICTED, or NOT_MENTIONED
5. Write a per-claim result list plus a validation summary back to state under output_key

Configuration:

The node surfaces contradictions and unsupported claims explicitly rather than silently passing them downstream, giving the pipeline a hard checkpoint before unverified claims propagate into an investment memo.

Entity Aliases

The Problem

The same entity can appear under different names: "Apple", "Apple Inc.", "Apple Inc", "AAPL". Without alias resolution, SPO queries miss related triples.

Approach

Entity aliases are stored as SPO triples themselves:

(Apple Inc., alias_of, Apple)
(AAPL, alias_of, Apple)
(Apple Computer, alias_of, Apple)

During query processing, the system resolves aliases before SPO lookup:

1. Query entities are checked against alias_of predicates
2. If an alias is found, the canonical entity is used for the SPO query
3. Results from all aliases are merged

This uses the existing SPO infrastructure --- no separate alias table needed.

Gap Detection and Autonomous Research

When to Research

After initial synthesis, the system analyzes the response for signs of insufficient knowledge. Research is triggered (not on every query) when any of these conditions are detected:

1. Uncertainty phrases in the response:

"i don't know", "i'm not sure", "cannot find",
"don't have enough information", "not in the context",
"unable to answer", "no relevant", "insufficient"

2. Weak attribution signal: Average absolute attribution score < 0.001 across all attributed triples. Indicates no triples strongly influenced the response.

3. Missing bridges: Bridge edge ratio < 10% when the graph has 5+ edges. The graph lacks structural connectors between knowledge regions.

Research Flow

Gap Detected
    |
    v
Exa Neural Search (3 results)
    Returns: URLs + full text content
    |
    v
For each source:
    --> Chunk text (500 chars, 50 overlap)
    --> Embed each chunk into pgvector
    --> Extract triples via LLM (with epistemic metadata)
    --> Store triples in SPO index
    --> Link provenance (document_triples, document_chunks)
    |
    v
Re-synthesize with enriched context

Re-Synthesis

After research, context is rebuilt from BOTH the original and new knowledge:

EXISTING KNOWLEDGE:
(Apple, founded_in_year, 1976) [fact]
(Apple, headquartered_in, Cupertino) [fact]

NEW RESEARCH:
(Apple, revenue_2024, $391B) [claim, asserter: Reuters]
(Apple, ceo, Tim Cook) [fact, asserter: SEC filing]

SOURCE EXCERPTS:
[chunk text from newly ingested documents]

Constraints

• Only trigger research when gaps are detected (not every query)
• Limit to 3 Exa results per research cycle to control cost
• All new triples carry epistemic metadata (asserter, assertion_type)
• All new content links to source documents for provenance
• Research triggers are logged for debugging and cost monitoring
• Research results go through the same ingestion pipeline (human-gated for background research, automatic for in-flight gap filling during agent execution)

Conflict Detection

The Problem

When new triples contradict existing ones, the system must detect and handle the conflict rather than silently storing both.

Example:

Existing: (Tesla, founded_by, Elon_Musk)      confidence=0.95  fact
New:      (Tesla, founded_by, Martin_Eberhard)  confidence=0.80  claim
--> Contradiction detected: different founders asserted for the same subject+predicate

How Contradictions Are Detected

Two triples contradict when they share the same subject and predicate but assert different objects. The detect_contradictions function in qadra-epistemic-core implements this:

1. Normalize and group: All triples are grouped by normalized (subject, predicate) pairs. Normalization lowercases and collapses whitespace, so "Tesla" and "tesla" group together.
2. Check for distinct objects: Within each group, object values are normalized and compared. A contradiction exists only when 2+ distinct objects appear for the same subject+predicate.
3. Same object, different confidence: Two triples asserting (Tesla, founded_by, Elon Musk) at different confidence levels are NOT a contradiction --- they reinforce each other.
4. Parallel analysis: Groups are analyzed in parallel using rayon, since each (subject, predicate) group is independent.

Resolution Scoring

Each competing assertion receives a resolution score that determines the winner. The score combines four factors:

resolution_score = confidence * assertion_type_weight * source_bonus * trust_factor

Assertion type weights:

Source bonus: When multiple independent sources assert the same object value, the representative gets a source bonus of 1.0 + (source_count - 1) * 0.1. Two sources asserting the same object = 1.1x multiplier. This rewards corroboration.

Source trust scores: An optional source_trust map (source name to 0.0-1.0 score) can downweight unreliable sources. For example, {"sketchy-blog.com": 0.3, "reuters.com": 1.0} causes a high-confidence fact from a sketchy blog to lose to a lower-confidence claim from Reuters:

sketchy-blog.com: 0.9 * 1.0 (fact) * 0.3 (trust) = 0.27
reuters.com:      0.8 * 0.6 (claim) * 1.0 (trust) = 0.48  --> winner

Unknown sources default to trust factor 1.0 (full trust until proven otherwise).

Winner selection: Assertions are sorted by resolution score descending. The highest-scoring assertion wins. The Contradiction struct records the winner's triple ID and all competing assertions with their scores.

Contradiction Result

#![allow(unused)]
fn main() {
pub struct ContradictionResult {
    pub has_contradictions: bool,
    pub contradictions: Vec<Contradiction>,   // Each with subject, predicate, assertions, winner
    pub triples_analyzed: usize,
    pub conflict_count: usize,                // Number of unique S+P pairs with conflicts
}

pub struct Contradiction {
    pub subject: String,
    pub predicate: String,
    pub assertions: Vec<CompetingAssertion>,  // Sorted by resolution_score descending
    pub winner: Option<Uuid>,                 // Triple ID of the highest-scoring assertion
    pub detection_confidence: f32,            // 0.9 when scores are computable, 0.5 otherwise
}
}

Multi-Way Contradictions

The system handles contradictions with more than two competing assertions. For example, three different CEO assertions for the same company:

(Company_X, ceo, Alice)   [fact,    confidence=0.9]  --> score: 0.9 * 1.0 = 0.9  (winner)
(Company_X, ceo, Bob)     [claim,   confidence=0.7]  --> score: 0.7 * 0.6 = 0.42
(Company_X, ceo, Charlie) [opinion, confidence=0.5]  --> score: 0.5 * 0.3 = 0.15

All three assertions are recorded in the Contradiction with their resolution scores. Alice wins.

Integration with Ingestion Pipeline

During triple extraction, before storing a new triple, the pipeline checks for existing triples with the same subject and predicate but different object. When found:

1. Same assertion type: Flag for human review --- which is current?
2. Higher-confidence new triple: Replace old triple, archive the original
3. Lower-confidence new triple: Store as alternative, mark as contested
4. Temporal context available: Both may be correct at different points in time --- store both with temporal metadata

Contested Triples

When a conflict cannot be automatically resolved, both triples are stored and marked as contested. Synthesis prompts include both with appropriate hedging:

CONTESTED:
(Acme Corp, ceo, John Smith) [fact, asserter: 10-K filing, 2024-01]
(Acme Corp, ceo, Jane Doe) [claim, asserter: Bloomberg, 2024-06]
Note: CEO may have changed. Verify with latest source.

Implementation: crates/qadra-epistemic-core/src/contradictions.rs

Usage Examples

Ingesting a Document

POST /v1/documents
Content-Type: application/json

{
  "title": "Acme Corp Overview",
  "content": "Acme Corporation was founded in 1990 by John Smith. The company
              is headquartered in San Francisco and specializes in enterprise
              software. Acme acquired DataCo in 2015 for $50 million...",
  "source_type": "web",
  "source_url": "https://example.com/acme"
}

Response:

{
  "document_id": "uuid",
  "routing": "both",
  "triples_extracted": 4,
  "chunks_created": 2,
  "analysis": {
    "routing": "both",
    "content_type": "text/plain",
    "has_extractable_facts": true,
    "estimated_chunks": 2,
    "confidence": 0.87,
    "reasoning": "Mixed content with facts and narrative"
  }
}

Querying the Knowledge Graph

#![allow(unused)]
fn main() {
// Find all companies founded by a person
let companies = repository.query_spo_sp(
    tenant_id,
    "John Smith",
    "founded"
).await?;

// Find who founded a specific company
let founders = repository.query_spo_po(
    tenant_id,
    "founded_by",
    "Acme Corporation"
).await?;

// Semantic search for relevant context
let context = vector_index.search(
    tenant_id,
    embed("Acme acquisition history"),
    k: 5
).await?;
}

Combined SPO + Vector Query

#![allow(unused)]
fn main() {
// 1. SPO gives us structured facts about the entity
let facts = repository.query_spo_s(tenant_id, "Acme Corporation").await?;

// 2. Vector search gives us narrative context
let narrative = vector_index.search(
    tenant_id,
    embed("Acme Corporation competitive position"),
    k: 5
).await?;

// 3. Combine for synthesis
let context = format!(
    "STRUCTURED FACTS:\n{}\n\nNARRATIVE CONTEXT:\n{}",
    format_triples(&facts),
    format_chunks(&narrative),
);
}

Financial Data Extraction

A specialised sub-agent that queries the SPO index for financial metrics on a named entity. Instead of free-form synthesis, it pulls structured financial facts directly from the knowledge graph and returns them with confidence and provenance.

Recognised predicates:

revenue, ebitda, market_cap, founded_in_year,
headcount, funding_round, valuation

How it works: given an entity (e.g. "Acme Corporation"), the sub-agent looks up SPO triples whose subject matches the entity and whose predicate is one of the financial predicates above. Each match becomes a metric carrying its object value, confidence, and asserter.

#![allow(unused)]
fn main() {
pub struct FinancialMetric {
    pub metric: String,        // predicate name, e.g. "revenue"
    pub value: String,         // object value, e.g. "$391B"
    pub confidence: f32,       // from the source triple
    pub asserter: Option<String>,
    pub low_confidence: bool,  // true when confidence < 0.6
}
}

Metrics with confidence below 0.6 are flagged low_confidence so a reviewer can treat them with caution rather than excluding them outright.

NATS subject: qadra.{tid}.epistemic.extract_financial_metrics

Request { entity } --> response { entity, metrics: [...], retrieved_at }. Because the extraction is pure SPO lookup over a fixed predicate set, it is fast and deterministic --- no LLM call is needed to surface the figures.

Semantic Verification Cache

When verifying LLM responses against the knowledge graph, we can avoid redundant LLM calls by caching verification results for semantically similar queries.

The Problem

Every LLM response verification requires:

1. LLM call to generate/parse the response (~500ms-2s, ~$0.01-0.10)
2. SPO lookup to ground facts against triples (~5ms)

For similar queries ("When was Apple founded?" vs "What year did Apple start?"), we repeat the expensive LLM call even though the SPO verification would be identical.

The Solution: Perturbation Cache

Cache verification results keyed by semantic similarity, not exact string match.

Key insight: The SPO lookup is fast (~5ms). Even on cache hit, we re-validate against current facts. We only skip the expensive LLM call.

Cache Schema

Redis Stack with RediSearch provides vector similarity search:

FT.CREATE perturbation_cache ON HASH PREFIX 1 pcache:
  SCHEMA
    tenant_id TAG
    embedding VECTOR HNSW 6
      TYPE FLOAT32
      DIM 1536
      DISTANCE_METRIC COSINE
    query_hash TAG
    created_at NUMERIC SORTABLE
    ttl NUMERIC

Similarity Threshold

Start at 0.93, monitor cache hit rate and verification accuracy.

Performance