AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines

The paper introduces AgSpec, a framework designed to overcome the limitations of retrieval-based speculative decoding in coding agent pipelines. By leveraging session, workspace, and global corpora while indexing opened files in the agent's specific emission format, AgSpec addresses missing text issues. Furthermore, it bounds draft lengths using offline-profiled caps and adapts them online via verification feedback. Evaluated on two repository-level multi-agent coding benchmarks, AgSpec outperforms five retrieval-based drafters and EAGLE-3, raising generation throughput over autoregressive decoding up to 4.37 times at batch size 1 and 4.76 times at batch size 16.

AgSpec raises generation throughput over autoregressive decoding up to 4.37times at batch size 1 and 4.76times at batch size 16.

AgSpec outperforms five retrieval-based drafters and EAGLE-3 in most evaluated settings on two repository-level multi-agent coding benchmarks.