Running a 28.9M parameter LLM on an $8 microcontroller
SMRTR summary
A 28.9 million parameter language model now runs entirely on an $8 ESP32-S3 microcontroller, generating text at 9.5 tokens per second with no internet connection. By storing 25 million parameters in flash memory instead of RAM, using Google's Per-Layer Embeddings technique, this setup holds roughly 100 times more parameters than any previous chip of this size.
SMRTR provides this summary for quick context. The original article belongs to Daily.dev.
Read the original article