Before Q, K, and V: Reconstructing the Transformer

The Transformer architecture's key components, including queries, keys, and values, are not arbitrary design choices but instead arise naturally from solving specific engineering problems. Starting from the limits of older recurrent neural networks, which compressed memory inefficiently, attention mechanisms emerged as a way to give models direct access to all past inputs. The Q, K, and V matrices then followed from the need to break symmetry, reduce memory usage, and keep computations manageable on GPUs.