How We Reduced LLM Costs by 90% with 5 Lines of Code

A code fix drastically reduced LLM costs by controlling asynchronous requests in Python. Initially, a validation script sent all 100 requests simultaneously, despite needing only 10 successful responses. Implementing a semaphore to limit concurrent requests to 15 at a time prevented unnecessary API calls without affecting performance. This change reduced LLM traffic and costs by 90% by processing only required requests. The issue stemmed from Python's async behavior with as_completed, demonstrating how small structural changes can significantly improve resource efficiency.