Make your own Darkbloom – The Expert Network pools Macs into one model
SMRTR summary
Expert Sniper is an open-source tool that lets multiple ordinary Macs pool their storage and memory to run large AI models none could handle alone. By splitting model components across machines and streaming only what's needed, a 17GB model runs on a Mac with just 3.6GB of RAM, reaching 19.3 tokens per second across two networked nodes.
SMRTR provides this summary for quick context. The original article belongs to Hacker News.
Read the original article