A 28.9M PARAMETER LANGUAGE MODEL RUNS ENTIRELY ON AN EIGHT DOLLAR MICROCONTROLLER
A 28.9 million parameter model runs entirely on an eight dollar microcontroller with no server behind it.
by slvDev
AIOpen-hardware
- difficulty
- ●●●○○
- time
- a weekend
- license
- MIT
- repo
- repo ACTIVE3,417 stars
●●●○○ · a weekend · MIT · 3,417 stars · repo ACTIVE
WHAT YOU’LL NEED
- Electronic parts — full list with part numbers in the repo BOM
- Dev board / microcontroller — runs the project firmware
Partner
COMPAREE VERDICT
This is the first practical demonstration of a sub-thirty-million parameter language model running entirely on a microcontroller with no connectivity. The ESP32-S3 has 512KB of SRAM and 8MB of PSRAM, so the model should not fit — but slvDev borrowed Per-Layer Embeddings from Google's Gemma 3n and parked 25 million parameters in a lookup table in slow flash. Each token reads roughly 450 bytes out, activations stay in SRAM, and the dense core sits in PSRAM. The whole 28.9M parameter model quantized to 4 bits is 14.9MB. It generates at about 9.5 tokens per second displayed on a small screen. The measured benefit over a same-core baseline that fits in SRAM is 0.098 nats, about 9.3 percent perplexity improvement. For scale, earlier independent work on the same chip family ran 260 thousand parameters, so this is roughly 110 times larger. The honest limit is that both published models were trained on TinyStories or a narrow espresso Q&A dataset — they will not answer general questions, follow instructions, write code, or know facts. If you want to understand how small models are deployed on constrained hardware or explore architectural tricks at the memory boundary, this is the cleanest example available. If you want a general-purpose assistant, this is not it and training your own will take more than a weekend. The single thing most likely to go wrong is expecting ChatGPT behaviour from a 28.9M parameter TinyStories model.
IN THE REPO
NOT IN THE REPO
- —MIT licence permits commercial use.
- —Repository contains firmware, model conversion scripts, and two trained models on Hugging Face.
- —Bill of materials lists the ESP32-S3 board and optional display.
- —No CAD files — this is a software project with off-the-shelf hardware.
- —Documentation covers flashing, model conversion from PyTorch, and the Per-Layer Embeddings architecture.
- —The trained models are TinyStories-based: they write short simple stories, not general-purpose text.
Can I build this?
Build at your own risk. Projects involve tools, electronics and sometimes mains voltage — follow the creator’s safety notes.
Partner · KickstarterHeyGears G1: 10M+ colors and transparent parts in one print, plus UV printing on flat objects. Figures, parts and labels — no painting.
Gallery
Start here
Navigation into the creator’s own docs — we don’t rewrite the guide, we route you to the source.
- 1.Read the repository README for the board pinout and display wiring (Start here to confirm your ESP32-S3 variant matches the memory requirements: 8MB PSRAM and 16MB flash.)
- 2.Flash the firmware using PlatformIO or the Arduino IDE (The README contains flashing instructions; the published models are linked on Hugging Face.)
- 3.Test with the TinyStories model first, then try the Barista model if you want narrow Q&A behaviour (Both models are under 15MB and documented in the repo.)
KNOWN ISSUES
- The published models write short simple stories or answer espresso questions only — they will not follow instructions, write code, or know general facts, and retraining on a different dataset is a separate project.
- Not all ESP32-S3 boards have 8MB of PSRAM; cheaper variants with 2MB will not run the 28.9M parameter model.
- The 9.5 tokens per second speed is end-to-end generation including display refresh — it will feel slower than server-based models, and longer prompts will stretch the wait.
- Model conversion from PyTorch requires Python and familiarity with quantization scripts if you want to train your own; the published models avoid this step.
- The display wiring is optional but without it you will only see output over serial, which is harder to demo.
- The Per-Layer Embeddings architecture means inference speed depends on flash read latency — a board with slower flash will run slower.
Can I fine-tune it on my own dataset?
Yes, the repository includes model conversion scripts and training is standard PyTorch, but you will need a machine with a GPU and familiarity with the TinyStories or similar datasets to get coherent output from a model this small.
Why is it only trained on TinyStories?
A 28.9 million parameter model does not have the capacity for general knowledge or instruction-following — TinyStories is a constrained dataset designed for small models, and it lets the architecture demonstrate inference speed and memory efficiency without overpromising capability.
What is the practical use case?
This is primarily a research and educational project showing how far on-device inference can be pushed on microcontroller-class hardware; practical applications are narrow-domain text generation where connectivity is unavailable or undesirable.
How does Per-Layer Embeddings compare to other quantization methods?
Per-Layer Embeddings keep only the active rows of the embedding table in fast memory instead of the whole table; the measured benefit over a same-core baseline that fits in SRAM is about 9.3 percent perplexity reduction, and the parameter count scales roughly 110 times larger than earlier independent ESP32 work.
Community builds
No community builds yet — be the first, we feature the best ones.
Discussion1
FROM THE COMPAREE TEAM
The ESP32-S3 runs this at 9.5 tokens per second with 28.9 million parameters and no server behind it — what narrow domain would you want to run entirely on-chip?
slvDev
slvDev published esp32-ai in August 2024 and has been iterating on memory-constrained inference for microcontrollers, applying architectural ideas from Google's Gemma 3n to hardware with 512KB of SRAM. The project gained traction as the first sub-thirty-million parameter model running entirely on an ESP32 with no connectivity.
Keep building — related projects
Same category, picked by how close they are to this build.
NO CAD FILESBuild a talking AI assistant on a breadboard and host its brain yourself
AIOpen-hardware●●●○○moderate
FULL CADTurn any glasses into AI smart glasses for 20 dollars - and it remembers everyone you meet
AIOpen-hardware●●●○○moderate
NO CAD FILESA 28.9M parameter language model runs entirely on an 8 dollars microcontroller
AIOpen-hardware●●●●○advanced
FULL CADThis retro talking head runs a private ChatGPT-style AI fully offline on one Raspberry Pi
AIOpen-hardware●●●●○advanced
DISCLAIMER
- Comparee is not the author of the projects featured here. All rights to each project belong to its creator — every page links to the original source, and we never host creators’ files.
- Information is provided without warranty and may become outdated as projects evolve. Prices are indicative bands only — always check the creator’s parts list for current costs.
- Building and operating any project is at your own responsibility. Protective equipment, safe workshop practice and compliance with local regulations are the builder’s responsibility.
CompareeTEAM19d ago
Practical notes from our verification: the repository is actively maintained with the last push three days ago, the two trained models (TinyStories and Barista) are published on Hugging Face and linked in the README, and the single biggest decision is whether your ESP32-S3 board has 8MB of PSRAM — cheaper variants with 2MB will not run the full model. The Per-Layer Embeddings architecture is documented in a clear README section with diagrams, and the performance numbers (9.5 tokens per second, 0.098 nats improvement) are reproducible. No video walkthroughs exist yet, so you will be reading code and flashing firmware without a visual guide.