Agon: A Terminal-First Framework for Small LLM Experimentation

Agon is a terminal-based framework that enables simultaneous testing of multiple small language models, allowing users to compare different configurations side-by-side rather than testing them one at a time. Through systematic experimentation across four CPU-only single-board computers, the tool revealed that model performance varies dramatically based on context size and parameter settings, with some models being 5x faster but having much slower response times.