Agents on Rails: lemans goes open source
SMRTR summary
The team behind Agents on Rails has open-sourced lemans, their Ruby-based LLM benchmarking harness. Built with convention-over-configuration and CLI-first design, it sandboxes agents, hides verification tests, blocks internet access, and scrubs credentials from logs. Four new models were also benchmarked, including a locally-runnable Qwen and a mysterious unnamed model on OpenRouter scoring competitively against paid options.
SMRTR provides this summary for quick context. The original article belongs to Daily.dev.
Read the original article