PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control

The authors propose PhysStream, an autoregressive model for physics-grounded image-to-video synthesis. The model incorporates structured scene memory—comprising positional and object tracking maps generated online—and supports fine-grained motion control via sparse velocity-increment signals encoding physical quantities. Training is executed in two stages: a bidirectional model is first fine-tuned with motion-control conditioning, followed by a causal autoregressive model trained with structured scene memory. Experimental results demonstrate that PhysStream reduces motion distribution distance and trajectory error significantly on synthetic benchmarks, while being preferred by human evaluators in a vast majority of in-the-wild comparisons.

PhysStream reduces motion distribution distance (FVMD) by 33% over the strongest baselines on synthetic benchmarks.

PhysStream reduces trajectory error by 12% over the strongest baselines on synthetic benchmarks.

PhysStream is preferred by human evaluators in over 85% of in-the-wild comparisons.