AUTO-UPDATED

Google just put the latest AI models through a brutal coding test — here's how they did

Google has launched Android Bench 2.0, an updated evaluation tool designed to test the performance of large language models on complex, multi-day Android software development tasks and projects.

Key Points

  • Android Bench 2.0 introduces long-horizon tasks that simulate real-world engineering challenges like building apps from scratch or upgrading complex dependencies.
  • The benchmark replaces binary pass-fail grading with continuous scoring to provide more nuanced data on how models handle incomplete development tasks.
  • GPT-6 Astra currently leads the leaderboard with a 28% pass rate, significantly outperforming Gemini 3.8 Flash, which recorded an 8% pass rate.
  • The testing suite includes various industry-leading models, such as Claude Fable 5.1, GPT-5.6 Sol, and Claude Opus 5.

Why it Matters

This benchmark provides a more realistic assessment of how AI agents can assist professional developers with time-intensive, multi-step coding projects. By shifting focus toward long-horizon tasks, the tool helps the industry identify which models are truly capable of handling complex software engineering workflows.
Android Central Published by sanujb6@gmail.com (Sanuj Bhatia) , Sanuj Bhatia
Read original