Browser SLM Lab
Run real small language models on your own device, right here in this page. Pick a model below, let your browser download the weights once, then chat with it offline-capable and 100% private: WebGPU does the inference on your GPU, and nothing you type is sent to any server.
1. Can your device run this?
2. How a language model runs in a browser tab
Download quantized weights
The model ships as 4-bit quantized weights (q4f16), shrinking a 1.5B-parameter model from ~3 GB to under 1 GB. Your browser fetches the shards from a public CDN.
Cache locally
Weights are stored in the browser cache, so the download happens only once per model. Reloads and later visits start in seconds.
Compile WebGPU kernels
WebLLM (the MLC project) compiles GPU compute shaders in the page, the same idea as CUDA kernels but portable across NVIDIA, AMD, Apple, and Intel GPUs.
Generate tokens on-device
Your prompt is tokenized, runs through the transformer layer by layer on your GPU, and tokens stream back into the chat. No API key, no server, no cost.
3. Pick a model to load
Start small: SmolLM2-135M downloads in under a minute on most connections and proves the pipeline works. Then step up to Qwen 2.5 1.5B or DeepSeek-R1 Distill for noticeably better answers. Sizes are approximate one-time downloads.
4. Playground
5. The models, side by side
| Model | Maker | Params | What it teaches you |
|---|---|---|---|
| SmolLM2 (135M / 360M / 1.7B) | Hugging Face | 135M-1.7B | How far careful data curation goes: the 135M model is smaller than many phone photos yet holds a conversation. The 1.7B version is a genuinely useful assistant. |
| Qwen 2.5 (0.5B / 1.5B) | Alibaba | 0.5B / 1.5B | Best-in-class multilingual coverage and instruction following at tiny sizes. The 1.5B model is a strong default pick for browser inference. |
| Gemma (1B class) | 1-2B | A distilled slice of the Gemini lineage. Shows how a big lab compresses frontier training recipes into an open, on-device model. | |
| Phi-3 Mini | Microsoft | 3.8B | The "textbooks are all you need" thesis: train on filtered, textbook-quality data and a 3.8B model rivals much larger ones. The biggest brain in this lab, and the heaviest download. |
| TinyLlama 1.1B | Open community | 1.1B | A 2023 classic: Llama 2 architecture trained on 3 trillion tokens. Great baseline for feeling how far SLMs have come since. |
| DeepSeek-R1 Distill 1.5B | DeepSeek | 1.5B | Reasoning distillation: a big reasoning model taught a small Qwen student to "think out loud". Watch its thinking traces stream in the chat. |
| MobileLLM | Meta | 125M-1B | Designed for phones: deep-and-thin layers, embedding sharing, grouped-query attention. No WebGPU build exists yet, so it is a study-only entry here; it runs on-device via ExecuTorch. |
6. What to take away
- Privacy by architecture: nothing you type here leaves your device. That is not a policy promise, it is how the system is built. For regulated data (health, legal, finance), on-device SLMs remove a whole class of risk.
- Cost goes to zero: after the one-time download there is no per-token bill. Features like autocomplete, summarization, or form-filling can run unlimited volume for free.
- Quantization is the enabler: 4-bit weights cut memory ~4x versus fp16 with a small quality loss. That is the difference between "needs a server GPU" and "runs in a tab".
- Small models need tight prompts: SLMs follow simple, explicit instructions well but drift on vague or multi-step asks. Compare the same prompt across sizes above and you will feel the capability curve directly.
- The pattern to remember: route easy, high-volume tasks to a local SLM and escalate hard ones to a frontier API model. That hybrid is how production teams cut AI costs 10x.