LLMs’ “simulated reasoning” abilities are a “brittle mirage,” researchers find

Chain-of-thought AI models perform well on familiar problems but fail dramatically when faced with new scenarios that require logical generalization. University of Arizona researchers found that these models aren't true reasoners but merely simulate reasoning-like text, with performance degrading significantly when tasks deviate from training examples. This "brittle mirage" of intelligence creates a dangerous false impression of dependability, raising concerns about using such systems in high-stakes fields like medicine or law.