exo-explore/exo – Run your own AI cluster at home with everyday devices ️⌚
SMRTR summary
exo is an experimental software project that allows users to run AI models across multiple everyday devices, creating a distributed GPU cluster at home. It supports various models like LLaMA, Mistral, and LlaVA, and can automatically discover and connect devices. exo offers dynamic model partitioning, a ChatGPT-compatible API, and device equality. The software requires Python 3.12.0 or higher and enough combined memory across devices to fit the chosen model. It can run on diverse hardware configurations, including Macs, PCs, and even Raspberry Pis. exo is still in development, with ongoing improvements and known issues being addressed.
SMRTR provides this summary for quick context. The original article belongs to Github.
Read the original article