Scheduling Recursive Reasoning in Looped Transformers

This paper analyzes the terminal loss sensitivity to recurrent update scales in looped transformer models. By decomposing temporal averages into persistent-progress and centered-fluctuation contributions, the authors introduce TAPS to adaptively adjust step sizes online. Experiments demonstrate that TAPS improves terminal accuracy across structured reasoning tasks and yields up to 1.56 times wall-clock speedup at matched baseline accuracy without retraining.

TAPS yields additional accuracy gains with up to 1.56 times wall-clock speedup at matched baseline accuracy.