GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay
QUESTION — How can the gameplay capabilities of AI models be comprehensively evaluated across multiple time horizons?
The authors present GameHorizon Suite, a comprehensive data and evaluation suite designed to measure gameplay capabilities across multiple temporal horizons. The suite consists of GameHorizon-Annotator for automated instruction annotation, GameHorizon-Data comprising 5,000 hours of recordings from 21 games with multi-horizon instructions, and GameHorizon-Bench for offline and stepwise online testing. Evaluating 47 models through over one million invocations establishes a clear task difficulty hierarchy.
GameHorizon-Data comprises 5,000 hours of recordings from 21 games, collected by 100 human expert players.
The study evaluates 47 models through more than one million model invocations to identify task hierarchies.
taesiri · 21 Sept 2026
read the original ↗