Fine-tune an 8B model on a 4 GB laptop GPU

Soup is an open-source tool that makes fine-tuning large language models simple, using a single config file and one command. Its key feature, layer streaming, keeps a model's frozen base in regular RAM and feeds it to the GPU one layer at a time, allowing an 8B model to run on just 4 GB of VRAM at 119.6 tokens per second. The latest update extends this to preference-based training methods like DPO and KTO.