A new benchmark study evaluating AI performance in the real-time strategy game StarCraft: Brood War reveals that current models struggle to progress beyond a beginner level of play.
Key Points
- Codex Astra emerged as the top-performing model, achieving an undefeated 100% win rate in the round-robin tournament.
- Claude Fable demonstrated the most sophisticated gameplay by successfully climbing the tech tree and managing complex unit production.
- Grok 4.6 models consistently failed to execute basic combat loops, often spending excessive time reasoning without issuing game commands.
- Most AI agents struggled with macro-management, frequently creating disconnected subagents that failed to coordinate army production and economic growth.
- All tested models were unable to defend against simple strategies, such as a basic photon rush, that a human beginner could easily execute.