smolmaxxing Runs a 16-Billion-Parameter Model on a 1GB iPad Air
An open-source project called smolmaxxing modifies llama.cpp so large AI models run on devices with very little memory. On a Xiaomi TV Box S with 1.9GB, the 16-billion-parameter Ling-mini-2.0 ran at 5.1 tokens per second.
1 source 6h ago
Reporting from 1 source: GIGAZINE.
Why it matters
The project shows local AI performance is now limited as much by model structure and memory handling as by raw device memory, since a 9-billion-parameter model on the same TV box managed only 0.27 tokens per second.
Key Facts
- smolmaxxing is an open-source project that modifies llama.cpp to run AI models on devices with limited memory.
- The 16-billion-parameter Ling-mini-2.0 ran at 5.1 tokens per second on a Xiaomi TV Box S with 1.9GB of memory.
- Ling-mini-2.0 ran at 4.5 tokens per second on a first-generation iPad Air with 1GB of memory, which required jailbreaking.
- The 1.7-billion-parameter Bonsai-1.7B ran at 23.7 tokens per second on the first-generation iPad Air.
- The developer ran about 160 experiments and published the program and results on GitHub, with models on Hugging Face.
On a first-generation iPad Air with 1GB, Ling-mini-2.0 ran at 4.5 tokens per second and the 1.7-billion-parameter Bonsai-1.7B at 23.7 tokens per second. The iPad Air test required jailbreaking.
The developer ran about 160 experiments and published both the program and the results on GitHub, with optimized models hosted on Hugging Face. The memory approach keeps frequently used data in memory and reads the rest from storage as needed, with processing tuned for the CPU.
Speed depends heavily on model structure. On the Xiaomi TV Box S, the 9-billion-parameter Qwen3.5-9B managed 0.27 tokens per second, and the developer names Qwen3.5-2B as the model with the best balance of speed and quality. The stated aim is to reach smartphones and routers next.
Written with AI by Yomimono from the 1 cited source below, including Japanese-language reporting where cited. See our editorial policy.
Add Yomimono as a preferred source on Google