Agentic AI Evaluation Platform

Problem

Reviewing model-monitoring anomalies is a judgment-heavy, easy-to-get-wrong task. A reviewer has to separate true anomalies from ordinary noise, recognize seasonality and recovery effects, notice when a metric's definition changed rather than its underlying behavior, assemble the evidence that actually supports a conclusion, write the finding up consistently, and leave a clear record of who decided what.

Intended user

Designed as a prototype for analysts or model reviewers who need to evaluate monitoring anomalies, trace findings back to specific evidence, compare architectures, and retain a clear human decision trail. No real analyst has used it in production — it's a working demonstration of the workflow, not a deployed tool.

Workflow before the tool

No measured baseline is claimed here. Qualitatively, this kind of review tends to be manual and inconsistent: an analyst reads a metric history, applies informal heuristics for what counts as an anomaly, and writes up a finding with no standard way to trace which evidence was used, compare one reviewer's judgment to another's, or check it against a transparent rule.

What I built

• A synthetic data generator producing 600 cases across 8 fictional models, 15 documented scenario types, and 4 difficulty levels — with anomaly labels deliberately not recoverable from any single field.
• An evidence store that restructures each case's data into stable, ID-addressable items (metric snapshot, historical baseline, validation rule, release notes, segment comparison, seasonality/recovery indicators, triggered deterministic checks) — so a finding can cite exactly which evidence supports it, and that citation can be checked, not just trusted.
• Seven evidence-retrieval tools (list/get by kind, get-by-ID), each argument-validated and logged, bounded to 6 calls per attempt.
• A real Claude tool-use loop, verified against the installed Anthropic SDK's actual type shapes and end-to-end unit-tested with a stubbed client — plus a deterministic mock-mode equivalent that drives the same tool executor against the same evidence store.
• Two architectures run on identical cases: single-shot (evidence pre-rendered into one prompt) and agentic/tool-using (evidence retrieved on demand) — a controlled comparison, not a quality contest (see Evaluation below).
• An independent reviewer agent with a genuinely distinct prompt and role — it checks the first agent's evidence support and severity, it doesn't repeat the task.
• Deterministic output validation — 7 checks independent of any LLM: schema conformance, evidence-ID existence, numeric consistency, high-severity-requires-review, unsupported causal wording, reviewer compatibility, tool-call limit.
• An explicit, single-function escalation policy covering 9 possible triggers (severity, confidence, abstention, insufficient evidence, unsupported claims, non-approve review, failed validation, excessive tool calls).
• 127 pytest tests, entirely offline — verified by uninstalling anthropic/openai and re-running the full suite.

How it works

Where the LLM acts, where deterministic logic acts

Only two calls in the whole pipeline touch a language model per case — the QA agent's finding and the reviewer's check. Everything downstream of those two calls (existence-checking evidence IDs, numeric consistency, the escalation decision) is plain code with no model involved, so it behaves identically on every run and is unit-tested without needing a model at all.

Single-shot vs. agentic: what the comparison shows

Both architectures draw from the identical per-case evidence — they differ only in delivery: single-shot renders it all into one prompt; agentic retrieves it via bounded tool calls. In mock mode, both conditions deliberately share the same underlying decision policy, so the comparison isolates the evidence-delivery mechanism, not decision quality.

The accuracy difference is not statistically significant (p=0.503) — exactly what the controlled design predicts, since both share the same decision policy. This result should not be read as evidence that the agentic architecture improves, or degrades, model quality. What it demonstrates is that the tool-retrieval, evidence-citation, and tool-call-limit machinery all work correctly end to end under a controlled comparison. Whether a live agentic Claude run would find different or better findings than single-shot is a separate question this mock comparison cannot answer.

Inputs and outputs

The agent's response is validated against a Pydantic schema (src/schemas.py) with extra = "forbid". Key fields, including the new evidence_ids:

AgentFinding {
  anomaly_detected: bool
  severity: "none" | "low" | "medium" | "high"
  anomaly_type: str
  summary: str
  observations: [str]              # facts only
  possible_explanations: [str]     # hypotheses only, never asserted as fact
  supporting_evidence: [EvidenceItem]
  evidence_ids: [str]              # must reference real items — checked
                                    # deterministically, not by the schema
  confidence_score: float          # validated in [0, 1]
  evidence_sufficiency: "insufficient" | "limited" | "adequate" | "strong"
  unsupported_claim_risk: "low" | "medium" | "high"
  requires_human_review: bool
  abstained: bool
  abstention_reason: str | None
}

A real logged case (case_0000, from outputs/sample_agent_outputs.json — a mock-mode run, not a live model):

case_id: "case_0000"     model_id: "model_08"     metric: "score_mean"
scenario_type: "normal_variation"

pred_anomaly: false             confidence: 0.5985
pred_severity: "none"           evidence_sufficiency: "adequate"
unsupported_claim_risk: "low"   requires_human_review: false

deterministic_validation_passed: true
reviewer_decision: "approve"

Normal variation, correctly predicted as no anomaly, validation passed, reviewer approved, no escalation — the routine case. The Screenshots section below shows one where evidence, finding, reviewer output, and escalation are all visible together for a single case.

Error handling and validation

• Schema validation (Pydantic, extra = "forbid"): a finding missing a required field, with the wrong type, or with an unexpected extra field is rejected before it reaches any downstream step.
• Evidence-ID existence: checked against the real evidence store, not by the schema — a citation to an evidence ID that doesn't exist for that case fails validation even though the schema itself can't know that.
• Numeric consistency: a finding's stated confidence and severity are cross-checked against each other and against the evidence for internal contradictions.
• Tool-call limit (agentic only): bounded to 6 calls per attempt; hitting the limit is a validation failure, not a silent truncation.
• Unsupported-claim detection: a surface-pattern check flags causal or definitive language not backed by cited evidence — documented as an approximation, not ground truth (see Limitations).
• What none of this catches: whether a claim is factually correct. Validation confirms structure, citation existence, and internal consistency — not truth. That's exactly why the escalation policy and human review exist downstream.

Tech stack

• Python 3.11+
• Pydantic
• Anthropic SDK (tool use)
• pandas / numpy / scipy
• Streamlit
• Plotly
• SQLite
• pytest

Evaluation

Calibration: expected calibration error 0.120, Brier score 0.261.

Design decisions

• A deterministic, rule-based baseline runs alongside the agent on every case as a transparent reference point — never as a target the agent is judged against in isolation.
• Evidence IDs are checked deterministically (against the real evidence store), not by the Pydantic schema itself, because existence-checking needs external state a single model instance can't see.
• alert_type is deliberately excluded from agent-facing evidence — it's derived 1:1 from the internal scenario type for several scenarios and would leak the ground-truth answer if ever shown to the agent.
• Single-shot and agentic share one decision policy in mock mode specifically so the architecture comparison isolates evidence delivery, not decision quality.
• The reviewer agent has a genuinely distinct prompt and role rather than a second call to the same task — an "independent" reviewer that repeats the same task with the same context isn't actually checking anything.

Adoption & documentation

Quick start (mock mode, no API key required):

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python -m pytest                     # 127 tests, entirely offline

python -m src.experiment_runner      # single-shot + agentic + 20 more conditions
python -m src.reporting              # regenerate reports/research_summary.md
streamlit run app.py                 # 9-page dashboard shown below

Live-model experiments are optional, require anthropic and an API key, and are never run by default.

Screenshots

Responsible AI

The system enforces evidence-based conclusions and an explicit observation-vs-hypothesis distinction, requires uncertainty and abstention fields, flags cases for human review via an explicit escalation policy, versions prompts and models, logs provenance, and uses only synthetic data. The agent is positioned as an assistant to analyst judgment, never as a replacement.

Limitations

• All reported results are simulation under a documented mock process and do not estimate any real model's behavior.
• The Anthropic tool-use loop has never been executed against the live API — no key was available in this environment.
• The deterministic baseline, the surface-pattern unsupported-claim detector, and the escalation policy are documented approximations, not ground truth.
• The human-review store is a single local SQLite file — not multi-user, not persistent across environments, not access-controlled.

What I'd improve next

• Execute the live Anthropic tool-use path and compare paired results against the mock predictions.
• OpenAI tool-use parity (currently Anthropic-only for the agentic condition).
• Add multimodal evidence (chart images) and measure its effect on grounding.