AI Fluency & Response Calibration

Overview

Qadra adapts how it responds based on each user's AI fluency --- their skill at interacting with an AI system. The system observes interaction patterns, infers a fluency level, and calibrates every response accordingly: terse bullets for experts, thorough walkthroughs for novices.

The core principle: Fluency is about HOW users interact, not WHAT they know.

• A novice investor asking about DCF is still a novice AI user.
• An expert prompter asking about stocks is still an expert AI user.

Domain expertise and jargon are not fluency indicators. A user who fluently discusses discounted cash flows but accepts the first AI answer without refining it is interacting like a novice. Fluency lives in the interaction loop --- iteration, format specification, chaining --- never in the subject matter.

Fluency Detection

The system scores each query for signals that reveal interaction skill, weights them, and feeds them into a rolling window. Never use jargon or topic complexity as a signal.

Expert Indicators

High-weight signals that the user is steering the AI deliberately:

Novice Indicators

Lower-weight signals that the user is accepting output passively:

Pattern Detection

Signals are detected with compiled regular expressions over the raw query text. Format specification, context provision, and query chaining each have their own pattern set:

#![allow(unused)]
fn main() {
lazy_static! {
    static ref FORMAT_SPEC_PATTERNS: Vec<Regex> = vec![
        Regex::new(r"(?i)\b(in json|as json|json format)\b").unwrap(),
        Regex::new(r"(?i)\b(as a (list|table)|bullet points)\b").unwrap(),
        Regex::new(r"(?i)\b(step.?by.?step|break.?(it )?(down|into))\b").unwrap(),
    ];
}

lazy_static! {
    static ref CONTEXT_PROVISION_PATTERNS: Vec<Regex> = vec![
        Regex::new(r"(?i)\b(for context|some background)\b").unwrap(),
        Regex::new(r"(?i)\b(i'm (working on|trying to|building))\b").unwrap(),
        Regex::new(r"(?i)\b(the goal is|my goal is)\b").unwrap(),
    ];
}

lazy_static! {
    static ref CHAINING_PATTERNS: Vec<Regex> = vec![
        Regex::new(r"(?i)\b(now|next|then).*\b(that|this|the above)\b").unwrap(),
        Regex::new(r"(?i)\b(take that|use that|apply that)\b").unwrap(),
    ];
}
}

Iteration and chaining are also detected relationally --- comparing the current query against the previous one --- not just by single-query keywords.

Fluency Levels

Every assessment resolves to one of four levels:

#![allow(unused)]
fn main() {
pub enum FluencyLevel {
    Novice,
    Intermediate,
    Advanced,
    Expert,
}
}

Level Determination

An expert score is computed as a weighted blend of the expert signals, then normalized and compared against thresholds:

#![allow(unused)]
fn main() {
// Expert: iteration + format specs + examples + chaining
let expert_score =
    counts.iterates_prompt * 0.35 +
    counts.specifies_format * 0.25 +
    counts.uses_examples * 0.2 +
    counts.chains_queries * 0.2;

// Thresholds
if norm_expert > 0.4 { FluencyLevel::Expert }
else if norm_expert > 0.25 || norm_intermediate > 0.5 { FluencyLevel::Advanced }
else if norm_novice > 0.4 { FluencyLevel::Novice }
else { FluencyLevel::Intermediate }
}

Rolling Window & Confidence

Fluency is not judged from a single query. The assessment maintains a rolling window of recent signals:

• Tracks the last 50 signals (configurable window size).
• Uses an exponential moving average for complexity, so recent behavior weighs more.
• Confidence increases with signal count, reaching its maximum at ~15 signals --- early in a session, confidence is low and the system stays cautious.

#![allow(unused)]
fn main() {
pub struct FluencyAssessment {
    signals: VecDeque<FluencySignal>,
    window_size: usize,
    profile: FluencyProfile,
}

impl FluencyAssessment {
    pub fn process_query(
        &mut self,
        query: &str,
        previous_query: Option<&str>,
    ) -> &FluencyProfile {
        let signals = self.analyze_query(query);
        let is_iteration = self.detect_iteration(query, previous_query);
        let is_chaining = self.detect_chaining(query);

        // Add signals to rolling window
        // Update profile
        // Return updated profile
    }

    pub fn needs_more_guidance(&self) -> bool {
        matches!(self.profile.level, FluencyLevel::Novice)
            || (matches!(self.profile.level, FluencyLevel::Intermediate)
                && self.profile.confidence < 0.5)
    }

    pub fn prefers_brevity(&self) -> bool {
        matches!(self.profile.level, FluencyLevel::Expert)
            || (matches!(self.profile.level, FluencyLevel::Advanced)
                && self.profile.confidence > 0.6)
    }
}
}

Both level and confidence are always checked together --- a high-fluency guess with low confidence should not produce terse expert-style output.

Response Calibration

Once a fluency level is established, the response is tuned across several dimensions:

#![allow(unused)]
fn main() {
pub struct ResponseCalibration {
    pub verbosity: Verbosity,
    pub use_headers: bool,
    pub use_bullets: bool,
    pub technical_level: TechnicalLevel,
    pub anticipate_followups: bool,
    pub should_simplify: bool,
    pub should_elaborate: bool,
}

pub enum Verbosity {
    Minimal,
    Concise,
    Detailed,
    Comprehensive,
}

pub enum TechnicalLevel {
    Layman,
    Intermediate,
    Technical,
    Expert,
}
}

Calibration by Level

Expert users get dense, concise delivery --- no headers (they know what they're looking for), no hand-holding, no anticipated follow-ups (they'll ask if needed). Novice users get comprehensive, layman-level explanations with headers, bullets, simplification, and suggested next steps.

Confidence Blending

When confidence is low, the calibration blends toward Intermediate rather than committing to an uncertain extreme:

#![allow(unused)]
fn main() {
impl ResponseCalibrator {
    pub fn calibrate(
        &self,
        fluency_level: FluencyLevel,
        confidence: f32,
    ) -> ResponseCalibration {
        // Low confidence = blend toward intermediate
        if confidence < 0.5 {
            self.blend_calibrations(
                &self.get_calibration(fluency_level),
                &self.get_calibration(FluencyLevel::Intermediate),
                confidence,
            )
        } else {
            self.get_calibration(fluency_level)
        }
    }
}
}

Calibration → Synthesis Prompt

Calibration is not cosmetic post-processing --- it is injected directly into the synthesis prompt so the model generates the right shape of answer from the start:

#![allow(unused)]
fn main() {
fn build_prompt(
    query: &str,
    context: &str,
    calibration: &ResponseCalibration,
) -> String {
    let mut instructions = String::new();

    match calibration.verbosity {
        Verbosity::Minimal => instructions.push_str("Be extremely brief. "),
        Verbosity::Comprehensive => {
            instructions.push_str("Provide thorough explanation with examples. ")
        }
        _ => {}
    }

    if calibration.should_simplify {
        instructions.push_str("Use simple language, avoid jargon. ");
    }

    if calibration.anticipate_followups {
        instructions.push_str("Suggest what the user might want to explore next. ");
    }

    format!("{}\n\nContext:\n{}\n\nQuestion: {}", instructions, context, query)
}
}

Real-Time Adaptation

Because the rolling window updates on every interaction, fluency --- and therefore calibration --- shifts within a single session. As the user demonstrates more sophisticated interaction patterns, the responses tighten accordingly:

Query 1: "What is NVIDIA worth?"
-> Fluency: intermediate (simple question)
-> Response: Detailed with explanation

Query 2: "Break it down as a table"
-> Fluency: advanced (format spec detected!)
-> Response: Concise table format

Query 3: "Now compare to AMD, bullet points only"
-> Fluency: expert (chaining + constraints)
-> Response: Minimal bullet comparison

Note that none of these shifts depend on the topic (NVIDIA, AMD valuation) --- they are driven entirely by the interaction patterns: a bare question, then a format specification, then chaining plus an explicit constraint.

Constraints

• Never use jargon or keywords as fluency indicators --- domain expertise is not AI fluency.
• Update the profile on every interaction.
• Confidence requires sufficient signal history; blend toward Intermediate when confidence is low.
• Always check both level and confidence.
• Respect explicit format requests --- they override calibration defaults.
• Don't over-explain to experts; don't under-explain to novices.