Colibrì Inference Engine Runs 744B-Parameter GLM-5.2 on a Consumer PC with 25GB of Memory
Colibrì demonstrates that even a 744B-parameter open model can be run on a consumer PC by trading speed for memory, potentially broadening access to large language models beyond users with expensive hardware.
Reporting from 1 source: GIGAZINE.
A new inference engine called Colibrì can run the 744-billion-parameter GLM-5.2 model on a consumer PC with 25GB of RAM by streaming expert data from an SSD. The engine exploits the model's Mixture-of-Experts architecture, activating only about 40 billion parameters per token. Speed is very slow-0.05 to 0.1 tokens per second-but the approach makes large open models practical on limited hardware.
The developer vforno built Colibrì around GLM-5.2's Mixture-of-Experts structure, which activates only a subset of parameters per token. The engine keeps about 9.9GB of always-needed data in RAM in 4-bit format, while 21,504 experts totaling 370GB reside on an NVMe SSD. An LRU cache and a learning-based cache reduce SSD reads by keeping frequently used experts in RAM. The engine is a single C file with no external dependencies and does not require a GPU. The trade-off is speed: on a 12-core CPU with 25GB RAM, generation runs at 0.05-0.1 tokens per second when the cache is empty, meaning a short response can take several minutes.
Synthesized by Yomimono from the 1 cited source below, including Japanese-language reporting where cited, then editorially reviewed before publishing.