Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design

This paper introduces a continual adaptation framework where a frozen frontier model operates professional design software through over 230 tools, while accumulating and refining an external procedural memory of natural-language skills from experience. The memory expands by acquiring procedures for uncovered subtasks and deepens by revising existing ones based on execution outcomes, governed by a matched replay gate. Across five rounds using 1,406 real user briefs and 1,869 automatically graded trajectories without weight updates, the approach significantly improves execution success and generation quality across specialized design benchmarks.

Five rounds over 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, grow the bank from 76 documentation-derived skills to 139 and raise GenEval2 execution success on Claude-Sonnet-4 from 72.7% to 99.3% (+11.99 points in generation quality).

On 200 held-out briefs from user-traffic benchmark, widening or deepening alone reaches a 49.4% / 48.6% win rate over the no-skill agent, while their combination reaches 58.5% (p = 0.025).