Evaluating LLMs Playing Text Adventures

An evaluation of language models in text adventure games revealed significant performance differences. Using an achievement-based scoring system, researchers tested models across seven games. Gemini 2.5 Flash emerged as the most cost-effective performer, matching premium models at a fraction of the cost. Testing showed high variability in model performance across games, with linear-beginning games providing more consistent evaluation metrics than open-ended ones.