Latest story• Nov 30, 2025
Why We’ve Been Optimizing the Wrong Thing in LLMs for Years
Traditional large language models waste computational resources by spending equal effort predicting common filler words like "the" and "and" as meaningful words, despite filler words comprising over 50% of English text. Meta researchers developed Multi-Token Prediction (MTP), which trains models to predict multiple future tokens simultaneously rather than just the next word. MTP models achieve up to 17% better...