Learning to Solve Hard Problems in RL for LLMs by Never Giving Up
QUESTION — How can reinforcement learning for LLMs be adapted to improve performance on hard problems without wasting compute on easy ones?
This paper highlights the Matthew Effect in reinforcement learning for LLMs, where RL heavily favors easy problems over hard ones. To solve this, the authors introduce Never Give Up (NGU), an adaptive sampling method that continues generating samples until finding a correct one. By leveraging asynchronous RL, NGU automatically filters out easy problems using fewer samples while reallocating compute to harder problems. Evaluated on math and coding benchmarks, NGU improves compute efficiency and successfully resolves complex tasks that standard GRPO fails to complete.
RL shows large improvements on easy problems that an LLM is already good at solving, but small improvements on hard problems.
mnoukhov · 11 Sept 2026
read the original ↗