What Does Privileged Information Add to On-Policy Self-Distillation?

This research investigates the contribution of privileged information in on-policy self-distillation (OPSD) for large language models. To isolate this contribution, the authors construct AMPLE-Math, a suite of 5,319 mathematical problems with six reasoning views sharing the same answer. Results show that reference-free distillation accounts for much of Qwen3-1.7B's improvement under thinking-enabled evaluation, while evidence for additional reference benefits is modest. Benefits depend on the student being trained, and replacing short direct-response rollouts with long thinking-enabled rollouts can turn gains into losses. These findings suggest that OPSD improves access to existing reasoning capabilities through shared parameters.

AMPLE-Math is a reusable suite of 5,319 mathematical problems with six reasoning views that share the same answer.