How big are our embeddings now and why?

Embedding dimensions have grown dramatically in AI models, from 200-300 dimensions years ago to today's 4096-dimension models. The growth follows technological advances like BERT's 768 dimensions (divisible among 12 attention heads), OpenAI's 1536-dimension embeddings, and now Qwen-3's 4096 dimensions. This evolution reflects shifts from custom embeddings to API-based commodities, with increasing dimensions enabling better performance while researchers explore efficiency techniques like matryoshka representation to balance size with computational demands.