Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models

Language models are trained on real text but must generate from their own outputs at inference time, causing errors to compound over long sequences. A new method called Energy-Based Fine-Tuning (EBFT) addresses this by checking whether a model's generated outputs statistically match real completions in feature space, rather than just rewarding correctness. Tested on coding and translation tasks, EBFT matched or outperformed existing reinforcement learning methods on accuracy while also improving cross-entropy loss, without requiring any task-specific verifier or reward signal.