Silicon Valley bets big on ‘environments’ to train AI agents

Artificial intelligence agents were supposed to revolutionize how we use computers, but today's AI assistants still stumble over basic tasks. Behind the scenes, a new battleground is emerging around something called "reinforcement learning environments" – essentially digital training grounds where AI agents practice using software.

"All the big AI labs are building RL environments in-house," explains Jennifer Li, general partner at Andreessen Horowitz. "But as you can imagine, creating these datasets is very complex, so AI labs are also looking at third party vendors."

These environments function like elaborate simulations where AI agents attempt tasks like purchasing socks on Amazon, with rewards for successful completion. Unlike static datasets that powered earlier AI advances, these environments must anticipate countless ways an agent might fail.

The gold rush has begun. Established players like Surge are creating dedicated teams, while well-funded startups like Mechanize are offering software engineers $500,000 salaries to build these environments.

Not everyone shares the enthusiasm. Former Meta AI researcher Ross Taylor cautions that "people are underestimating how difficult it is to scale environments," noting that AI models often find ways to "cheat" rather than properly learn tasks.