What's the largest software project AI can complete on its own?
SMRTR summary
A single AI model worked for 19 days straight, spent $2,600, and never once asked for help. That's the scale of ambition behind MirrorCode, a new benchmark developed by Anthropic and METR to test whether AI can tackle serious, long-horizon coding tasks.
Unlike most benchmarks that cap spending at $10, MirrorCode asks AI models to completely reimplement entire programs from scratch, with no internet access, no original source code, and no way to cheat.
The results are striking. Claude Opus 4.7 recreated a bioinformatics toolkit called gotree, roughly 16,000 lines of code with over 40 commands, in just 14 hours for $251. A human engineer, working alone, might need up to 17 weeks.
Researchers caution that AI models may have encountered some of these open-source codebases during training, which could inflate scores. Still, the benchmark isn't fully solved, and the leaderboard is updated regularly as models keep pushing further.
SMRTR provides this summary for quick context. The original article belongs to Hacker News.
Read the original article