Known-Truth Audit Harness

This harness audits adaptive mastery engines against known truth. The engine under audit loses some of its comparisons. That is the point.

sim2-reg-v1spec v0.236 cells × 50 learners

Explore Results

Browse the full 36-cell factorial. See where the engine wins and where it loses.

Fingerprint Diagnostics

Three patterns that expose broken simulations: uniformity, dead-prior, prior-asymmetry.

Registration Ledger

Pre-registered predictions alongside results. Hypotheses committed before outcomes were known.

What this demonstrates

  • A mastery engine audited against synthetic learners with fully known state trajectories
  • Eight estimators on identical evidence: engine, two ablations, two BKT configs, terminal score, oracle
  • Pre-registered predictions that include the engine losing its primary state-recovery comparison
  • Fingerprint diagnostics distilled from real audit failures during development
  • Event lineage that decomposes mastery error by mechanism, not just aggregate AUC