Introducing Bolmo: Byteifying the next generation of language models

Ai2 has introduced Bolmo, the first fully open byte-level language models that process text as raw UTF-8 bytes rather than traditional subword tokens, addressing longstanding issues with character-level understanding and multilingual support. Instead of expensive training from scratch, Bolmo "byteifies" existing Olmo 3 models through a two-stage process that preserves the original transformer backbone while adding byte-processing capabilities, achieving comparable performance to subword models while excelling at character-focused tasks with nearly twenty-point accuracy improvements on specialized benchmarks.