Real-Time Introspective Compression for Transformers
A new technique called introspective compression allows transformer models to save and replay their internal thought states. This enables capabilities like backtracking in reasoning, reinforcement learning over thought trajectories, and causal debugging of model errors. The approach uses a sidecar encoder-decoder system to compress transformer states into a compact latent representation.