Bringing FinOps to Your LLMs: Understanding and Tracking OpenAI Spend

Companies using OpenAI's large language models for customer support, search, and other features are facing unexpected budget spikes due to poor cost tracking, with simple changes like including full customer history in prompts causing 10x cost increases overnight. Unlike traditional cloud services, OpenAI lacks flexible tagging and metadata capabilities, forcing teams to rely on strict naming conventions since usage is billed by tokens rather than compute hours. Many organizations are adopting a "proxy pattern" - middleware that stamps API calls with metadata before sending them to OpenAI - to gain better visibility into spending across teams and features.