Would I Use LLMs to Rebuild Twitter's Dynamic Product Ads? Yes and No!

A former Twitter engineer who once boosted ad click-through rates by 18% using traditional machine learning reveals a surprising truth about today's AI revolution: even now, he'd only use large language models for about 20% of the system. The engineer's Dynamic Product Ads matched millions of users to hundreds of millions of products in real-time, but the fundamental challenges remain unchanged regardless of whether you're using 2022 embeddings or 2026 AI models. While LLMs excel at understanding that "running shoes" and "sneakers for jogging" are semantically related, the real bottlenecks haven't shifted—understanding user intent from noisy signals, maintaining sub-millisecond response times, and managing data quality issues still dominate the engineering challenge. At Twitter's scale, the difference between 1ms and 10ms inference time translates to millions in infrastructure costs, making classic models essential for the heavy lifting while AI handles the nuanced embedding generation. The unpopular truth? Revolutionary AI approaches might deliver 5% improvements while being 10x slower and 100x more expensive than systems that already work and make money.