Decoding Looped Transformers Better for (Almost) Free

The paper introduces LoopCD, a training-free contrastive decoding framework that leverages intermediate states from recurrent loops in Looped Transformers. By contrasting the final prediction with an earlier recurrent pass—either in logit space or hidden-state space—LoopCD guides token selection effectively. This approach delivers substantial decoding performance gains while enabling a reduction of recurrent loops by half, thereby significantly cutting forward FLOPs during inference.

LoopCD-Logits raises Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33%.

LoopCD-Hidden lifts Huginn's HumanEval pass@1 from 22.56% to 31.71%.

The approach enables halving the number of recurrent loops, reducing forward FLOPs by 22.5% to 48.2%.