ToolFault Atlas
Simulator ready
OPEN-SOURCE RELIABILITY LABv1.0

Break the tools.
Test the agent.

Inject production-style failures into an AI agent workflow. Replay every decision and measure whether recovery logic actually works.

Run an experiment ↓
06fault
scenarios
03recovery
strategies
100%reproducible
runs
LIVE TRACE fault proxy armed
A
Recovery agentadaptive · policy v3
RUNNING
01
customer_lookup112 ms · 200 OK
✓
02
order_api1,842 ms · TIMEOUT
!
retry + jitter
02
order_api226 ms · RECOVERED
✓
03
policy_searchqueued
··
trace_84df3 / 5 steps2.18 s
01EXPERIMENT LAB

Design an incident.
Watch the agent respond.

The same seeded episodes run against a naive baseline and your selected recovery strategy, making every comparison reproducible.

LATEST RUN · COMPLETE
TIMEOUT CASCADE

Adaptive recovery

VERDICTRESILIENT
CONTROL GROUP

Strategy comparison

50 paired runs
REPRESENTATIVE EPISODE

Decision trace

02SYSTEM DESIGN

Built to test behavior,
not decorate a chatbot.

01
DSL

Scenario layer

Describes target tools, fault type, probability, and severity with deterministic seeded sampling.

  • 6 incident presets
  • Compound faults
  • Reproducible seeds
02
FX

Fault proxy

Intercepts tool calls and returns realistic transport, contract, freshness, or side-effect failures.

  • Timeout + 429
  • Schema + stale data
  • Partial writes
03
AG

Agent strategies

Runs naive, guarded, and adaptive policies through identical workflows and incident samples.

  • Bounded retries
  • Validation + fallback
  • Write verification
04
EV

Evaluation

Aggregates traces into reliability metrics and preserves a representative replay for inspection.

  • Success + recovery
  • Latency + efficiency
  • JSON evidence export
WHY IT MATTERS

Tool-using agents fail at system boundaries: unreliable APIs, ambiguous writes, stale retrieval, and silent contract changes. This lab makes those risks visible before production.

PORTFOLIO SIGNAL

Demonstrates agent architecture, eval design, observability, fault modeling, deterministic testing, and product communication in one inspectable build.

NEXT EXTENSION

Replace the simulated proxy with live HTTP interception and emit OpenTelemetry spans into a production trace backend.