Agent as Policy for Robotic Manipulation

The study introduces Agent as Policy (AGP), a framework allowing a general-purpose agent to directly control physical robots by interpreting visual evidence, writing executable programs, issuing motion commands, and revising actions based on physical outcomes. AGP integrates the agent's reasoning and programming capabilities into continuous interaction with the physical world without task- or environment-specific training. Tested across tasks like assembly from human videos, block construction, die reorientation, and bimanual towel folding, AGP achieves success rates of 100%, 100%, and 80% on three block construction configurations.

AGP achieves success rates of 100%, 100%, and 80% on three block construction configurations.