RWKV-7: Advancing Recurrent Neural Networks for Efficient Sequence Modeling

RWKV-7 "Goose" is a new sequence modeling architecture that achieves state-of-the-art performance for multilingual tasks at 3 billion parameters. It maintains constant memory usage and inference time per token while matching the performance of larger models trained on more data, introducing innovations like vector-valued state gating and adaptive in-context learning rates.