Synthetic recovery
Intelligent Tutoring Effect Size
VanLehn (2011) · Intelligent Tutoring
All five gates passed
Directionally correct; overshoot from idealized compliance — published value below recovered CI
Description
Recovery of the tutoring-vs-no-tutoring effect size from VanLehn’s meta-analysis. The planted effect was d = 0.76. The engine recovered d = 1.003, an overshoot expected under idealized simulation conditions where compliance is perfect and measurement error is minimal.
Methodology
Simulated 200 learners split into tutored and untutored groups with a planted effect of d = 0.76 on 40 items. The engine estimated proficiency trajectories for both groups and computed the standardized mean difference. Overshoot reflects the absence of real-world noise (attrition, off-task behavior, measurement error).
Reference
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.
This is a synthetic recovery, not a replication of the original study. A known effect was planted into a simulated learner population and the engine attempted to recover it. Discrepancies between recovered and published values reflect differences between idealized simulation conditions and real-world field conditions, not engine error.