Why Language Models Are So Hard To Understand

Researchers are using neuroscience-inspired techniques to understand how large language models work. They study models by observing responses to prompts and examining internal components. Early efforts have revealed how models represent concepts and perform tasks, but also exposed surprising complexities. Challenges include arbitrary procedures, redundant components, and "emergent self-repair" phenomena. Despite difficulties, researchers remain optimistic about progress in understanding AI systems, which currently operate more like growing plants than engineered machines.