A 28.9M PARAMETER LANGUAGE MODEL RUNS ENTIRELY ON AN 8 DOLLARS MICROCONTROLLER
A chip with 512KB of RAM is writing coherent stories with a 28.9 million parameter model, nothing sent to a server, and the whole thing costs about 8 dollars.
by slvDev
AIOpen-hardware
- difficulty
- ●●●●○
- time
- a weekend
- license
- MIT
- repo
- repo ACTIVE2,627 stars
●●●●○ · a weekend · MIT · 2,627 stars · repo ACTIVE
WHAT YOU’LL NEED
- Electronic parts — full list with part numbers in the repo BOM
- Dev board / microcontroller — runs the project firmware
Partner
COMPAREE VERDICT
This is the most interesting constraint problem on a microcontroller in years. Developer slvDev got a 28.9 million parameter language model running on an ESP32-S3 by keeping 25 million of those parameters in slow flash and pulling only what each token needs — about 450 bytes — into fast memory. The method is Per-Layer Embeddings from Google's Gemma models, and at 4-bit quantization the whole thing is 14.9MB. It generates text at roughly 9 tokens per second, fully offline, on hardware costing about 8 dollars. The model was trained on TinyStories, so it writes simple coherent narratives and absolutely nothing else. No questions, no code, no facts. If you want a pocket ChatGPT, this is not it. If you want to see what fits on a chip with 512KB of SRAM when you stop pretending everything has to live in RAM, this is the build. The one thing most likely to go wrong is expecting general assistant behaviour — the output is closer to a grammar demo than a useful tool, and that limitation is baked into the training data.
IN THE REPO
NOT IN THE REPO
- —MIT licensed, no commercial restrictions.
- —Complete firmware and model conversion scripts in the repo.
- —Bill of materials lists the ESP32-S3, display and power components.
- —README includes flash steps and memory layout explanation.
- —No pre-built binaries — you compile from source.
- —The model writes short stories only; it cannot answer questions, follow instructions, write code or retrieve facts.
Can I build this?
Build at your own risk. Projects involve tools, electronics and sometimes mains voltage — follow the creator’s safety notes.
Partner · KickstarterHeyGears G1: 10M+ colors and transparent parts in one print, plus UV printing on flat objects. Figures, parts and labels — no painting.
Videos
End-to-End (small) LLM Fine-tuning Tutorial (from data to model to live ...
No official walkthrough video in the payload. The X post and Reddit thread show the output but do not walk through the build steps.
Gallery
Start here
Navigation into the creator’s own docs — we don’t rewrite the guide, we route you to the source.
- 1.Read the README and check your ESP32-S3 has 8MB PSRAM (The memory layout depends on PSRAM being present; without it the model will not fit.)
- 2.Wire the display to the SPI pins listed in the schematic(Pin assignments are in the README; get this step right before flashing anything.)
- 3.Run the model conversion script to quantize and partition the weights(Python script in the repo; this is where the 25M embedding rows get split for flash storage.)
- 4.Flash the firmware and model partitions using PlatformIO or esptool(Two separate flash steps — firmware first, then the model data.)
Resources
Documentation, files and community threads for this build — we link straight to the original sources and never rehost the creator’s files.
Partner · KickstarterCORE POWER BANK S: 0.2" thin, 2.8 oz, 5000 mAh with 22.5W fast charging. Semi-solid battery, carbon fiber, IPX7 waterproof.
KNOWN ISSUES
- Buying an ESP32-S3 board without PSRAM — the repo specifies 8MB PSRAM and the model will not fit without it.
- Expecting assistant behaviour — this model writes short stories and cannot answer questions, follow instructions, write code or retrieve facts.
- Skipping the model conversion step — the weights must be quantized to 4-bit and partitioned for flash; you cannot use raw model files.
- Wiring the display to the wrong SPI pins — the pin assignments are in the README and must match exactly.
- Assuming 9 tokens per second is fast enough for interactive use — it is not; this is a demo of what fits, not a conversational interface.
- Comparing this to ChatGPT or any general LLM — the training data is TinyStories, a dataset of simple narratives, and the output reflects that constraint.
Can I train it on a different dataset?
Yes, but you would need to retrain the model from scratch and the repo does not include training code — only inference and model conversion scripts.
Will it answer questions or write code?
No. The model was trained on TinyStories, a dataset of short simple narratives, and it cannot answer questions, follow instructions, write code or retrieve facts.
How fast is 9 tokens per second?
Roughly one word every 130 milliseconds, which is readable but far slower than typing. It is fast enough to watch text appear, not fast enough for real-time conversation.
Does it need an internet connection?
No. The entire model runs on the chip and nothing is sent to a server.
What is Per-Layer Embeddings?
A memory layout from Google's Gemma models where the embedding table stays in slow storage and each token pulls only the rows it needs into fast memory. This project uses it to keep 25 million parameters in flash while the compute core stays in 512KB of SRAM.
Community builds
No community builds yet — be the first, we feature the best ones.
Discussion1
FROM THE COMPAREE TEAM
The repo hit 1,010 stars in three days and the technique — Per-Layer Embeddings with most parameters in flash — is the first time anyone has tried it on a chip this small. What would you use a fully offline story generator for?
slvDev
Developer slvDev built this to see how large a language model could fit on an ESP32-S3 by treating slow flash as the primary storage for embeddings. The project uses Per-Layer Embeddings from Google's Gemma models and quantizes the weights to 4-bit to get a 28.9M parameter model into 14.9MB.
Keep building — related projects
Same category, picked by how close they are to this build.
NO CAD FILESBuild a talking AI assistant on a breadboard and host its brain yourself
AIOpen-hardware●●●○○moderate
FULL CADTurn any glasses into AI smart glasses for 20 dollars - and it remembers everyone you meet
AIOpen-hardware●●●○○moderate
NO CAD FILESA 28.9M parameter language model runs entirely on an eight dollar microcontroller
AIOpen-hardware●●●○○moderate
FULL CADThis retro talking head runs a private ChatGPT-style AI fully offline on one Raspberry Pi
AIOpen-hardware●●●●○advanced
DISCLAIMER
- Comparee is not the author of the projects featured here. All rights to each project belong to its creator — every page links to the original source, and we never host creators’ files.
- Information is provided without warranty and may become outdated as projects evolve. Prices are indicative bands only — always check the creator’s parts list for current costs.
- Building and operating any project is at your own responsibility. Protective equipment, safe workshop practice and compliance with local regulations are the builder’s responsibility.
CompareeTEAM28d agoedited
Practical notes from our verification: the model writes coherent short stories at roughly 9 tokens per second, which is slow enough to watch each word appear but far too slow for interactive use. The README is clear about the TinyStories training data and the inability to answer questions or write code, but several discussion threads still assumed this was a general assistant — it is not. The most interesting part is not the output, it is the memory layout: 25 million embedding parameters stay in flash and each token pulls about 450 bytes into SRAM, which is how a 28.9M parameter model fits on a chip with 512KB of fast memory. No pre-built binaries, so your first weekend is compiling firmware, wiring the display, running Python model conversion scripts and flashing two partitions. The bill of materials is minimal — ESP32-S3 dev board with 8MB PSRAM, a small SPI display, USB-C cable — and the whole thing costs about $8 plus shipping.