Chonkie – A Fast, Lightweight Text Chunking Library for RAG

Chonkie is a new lightweight and fast library for text chunking in RAG (Retrieval-Augmented Generation) applications. It offers various chunking methods including token, word, sentence, and semantic-based approaches. Chonkie boasts smaller install size and faster performance compared to alternatives. The library aims to provide easy-to-use, efficient text chunking without unnecessary bloat or dependencies. It supports multiple tokenizers and includes detailed documentation for users.