RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

The paper introduces Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), which incorporates regularization principles into agent harness evolution to prevent overfitting on training tasks. The proposer uses a temporally annealed budget to limit bundled edits and encourages unexplored trajectories based on history. The selector features a critic to screen proposals and a pruner to remove ineffective or overly expensive changes. Across eight benchmarks spanning coding, workspace, and design tasks, RRSI gains up to 14.1 points on the evolved split, improves up to 4.7 points on out-of-distribution benchmarks, and runs on 30% fewer policy tokens.

RRSI gains up to 14.1 points on the split it evolves against and up to 4.7 points on five out-of-distribution benchmarks.

The resulting harness runs on 30% fewer policy tokens than unregularized evolution.