Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI

The paper investigates test-time AI-for-AI, examining how a Builder model can construct effective execution environments for a Target model while weights remain fixed. To make experience reusable, the authors introduce Meta-Skill: principles determining when support is needed and what resources to supply. The Builder extracts these principles from execution feedback on a development set and applies the frozen skill bank to unseen tasks. Across Harness-Bench and NewtonBench, full-bank meta-skills improve macro-average performance by 8.95 percentage points over no-skill construction and 12.02 points over direct delivery of the bank to the Target.

Full-bank meta-skills improve macro-average performance by 8.95 percentage points over no-skill construction.

They improve performance by 12.02 points over direct delivery of the same bank to the Target.