AI Generated Image
Google is changing how it judges AI models for Android coding
9to5Google • • technology
Key Points:
- Google is updating its Android Bench ranking system for AI models, shifting from a proprietary mini-swe-agent v1 benchmark to the standardized Harbor framework to better evaluate AI coding performance for Android development.
- The new approach allows Android developers to submit their own development tasks for evaluation, encouraging community collaboration and transparency by enabling users to shape the Android Bench dataset and share benchmark results.
- The refreshed Android Bench rankings place Claude Fable 5 at the top with a score of 84.5, outperforming GPT-5.5, which scored 80.2, and demonstrating the effectiveness of Anthropic's AI model despite existing usage restrictions.
- Most AI models have been reevaluated under the new framework, leading to updated scores that reflect the revised benchmarking criteria and provide a more standardized comparison for developers.
