Soul Player C64 – A real transformer running on a 1 MHz Commodore 64

Developers have successfully implemented a miniature 2-layer transformer neural network—the same architecture powering ChatGPT and other AI models—on a 1982 Commodore 64 computer using hand-written assembly code. The breakthrough required clever optimization including 8-bit quantization and a specialized softmax implementation, allowing the 25,000-parameter model to run on the vintage 1 MHz machine at roughly 60 seconds per token.