Context Language Models

01 — architecture 26 upvotes

Context Language Models

QUESTION — How can large language models natively manage their own context by treating it as a file?

The paper introduces Context Language Models (CLMs), which natively manage context by treating it as an updatable file allowing unrestricted modifications. This intrinsic behavior shifts context management away from external harness controls and naturally extends to multi-agent file sharing. Zero-shot CLMs outperform state-of-the-art context strategies across tasks, yielding 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, and 5% higher scores with 59% fewer FLOPs on EdgeBench. Furthermore, online reinforcement learning improves Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs, and Suffix Cache Reuse reduces server-side compute by 35% relative to standard SGLang.

Achieves 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus.

Yields 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench.

Online reinforcement learning improves Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs.

rulins · 29 Sept 2026 read the original ↗

Worth reading closely

Context Language Models