How to Build an AI Agent That Runs its Own LLM Experiments with autoresearch

Andrej Karpathy's open-source tool autoresearch lets an AI agent autonomously run LLM training experiments on a single GPU — editing code, measuring results, and deciding what to keep without human input. Over roughly 700 experiments on a small model, the agent found about 20 genuine improvements that stacked together and transferred to a larger model, cutting a key benchmark by 11%. A fixed 5-minute training budget and locked scoring metric ensure every result reflects real progress rather than gaming the system.