Simulation-Based Evaluation of Rule-Based Dynamic Difficulty Adjustment for Game-Based Learning Using Synthetic Bot Agents

Abstract

Dynamic difficulty adjustment (DDA) can regulate challenge in game-based learning, but favorable results against a fixed comparator do not necessarily demonstrate controller superiority. We evaluated a prespecified three-success/two-failure controller (DDA-R32) using Expert, Intermediate, and Novice synthetic agents across 3,000 simulation runs (three million interactions). Under the prespecified target tolerance, DDA-R32 increased target-zone occupancy relative to Fixed-5 by 0.6773, 0.3732, and 0.7694, respectively. Mean absolute probability error (MAE) decreased for Expert and Novice but not for Intermediate. A separately frozen exploratory protocol added 10,800 policy runs, tolerance sensitivity analyses, paired precision analyses, 40-state Markov validation, and horizon extensions. A feedback-only Window-20 policy achieved lower MAE than DDA-R32 in all nine profile-by-slope combinations. At the central slope, its MAE was 0.06948, 0.06553, and 0.06818, compared with 0.10377, 0.10493, and 0.09260 for DDA-R32. For the Intermediate profile, widening the target tolerance from 0.100 to 0.105 reversed the occupancy contrast with Fixed-5. The Markov analysis corroborated that the mechanistic balance reference describes adjustment requests rather than a stationary success-rate target. These findings demonstrate metric-dependent performance and limit claims of superiority for the streak controller. No human learning outcomes were measured.