Colibrì in one sentence: frontier AI, budget hardware, painful speed
Colibrì is a proof-of-concept system that shows a 1.5‑terabyte, 744‑billion‑parameter frontier‑level language model can run on a modest CPU with only 25GB of RAM and fast NVMe storage, trading extreme latency for the ability to run AI models locally on ordinary home hardware.
The key takeaway is blunt: frontier AI hardware no longer means you must own a data‑center‑class GPU stack. An individual engineer, Vincenzo (aka JustVugg), built Colibrì to run the GLM‑5.2 Mixture‑of‑Experts model—normally a 1.5TB heavyweight—on a setup with 25GB RAM and a 1 GB/s virtual NVMe drive. The cost is speed: on this configuration, Colibrì outputs only about 0.05–0.1 tokens per second, so one decent reply can take hours. That sounds unusable for chat, but it proves a point: the wall between hobbyist PCs and frontier‑level models is architectural, not absolute. If you care more about access, privacy, and experimentation than instant response, Colibrì is a glimpse of your future.
How Colibrì squeezes a 1.5TB model into 25GB of RAM
Colibrì works because it refuses the usual rule that the whole model must fit in memory. Instead, it treats GLM‑5.2 like a parts bin. This Mixture‑of‑Experts architecture includes hundreds of specialized sub‑models that are selected per token, not per question. Colibrì exploits that: it loads only the experts needed for each token into RAM, then evicts them again, pulling the next slice from NVMe as generation continues. In plain language, your SSD becomes a slow, gigantic extension of VRAM.
To make this limited RAM AI more feasible, the model is also quantized—compressed with a lossy encoding—so the on‑disk footprint shrinks before any slicing starts. The expert‑selection logic itself is written as a single C file with minimal dependencies, keeping overhead down. The bottleneck shifts from memory capacity to data shuffling: NVMe throughput and latency now dictate everything. Quote‑ready: “Colibrì’s speed on Vincenzo’s setup is only about 0.05 to 0.1 tokens per second on average, a measure that’s unusable for practical conversation.” But the brilliance is that it works at all, on hardware that looks more like a home lab than a server rack.
Old PCs, slow models, and why access still beats speed
Colibrì lands in a world where people already strain to run AI models locally on modest machines. One home user describes running Qwen3‑4B on a Beelink Mini S13 Pro—a mini desktop with an Intel N150 CPU and 16GB of DDR4 RAM—to generate daily briefings and newsletters. Even these small models crawl; generating a morning briefing can take around 10 minutes. Yet the setup is useful because prompts are scheduled at 5 AM, so latency is hidden behind automation.
This is the philosophy Colibrì extends to the extreme. The problem with running an LLM on an old PC is not only that “even fairly small LLMs run slowly” but also that their capabilities lag behind cloud giants. “The small local LLMs that run on my hardware are never going to match the capabilities of ChatGPT, Gemini, or Claude.” Colibrì flips that trade‑off: you keep the frontier‑level model quality, at least in theory, and sacrifice speed instead. For tasks that can batch overnight—analysis jobs, content generation, offline agents—hours‑long latency is annoying but acceptable.
Democratizing frontier AI hardware without a server rack
The appetite for home labs and local LLMs is rising because subscription costs add up and people dislike sending private data to remote servers. Local models already help with tasks like stripping names and account numbers from financial documents before sending anonymized data to stronger cloud models. The roadblock has been that using anything close to a frontier model seemed to demand a prohibitively large GPU cluster—think an NVL72‑class rack—far beyond hobbyist budgets.
Colibrì represents a different answer to frontier AI hardware. Rather than chase bigger GPUs, it asks: can smart storage access, quantization, and algorithm design let an LLM on an old PC punch above its weight? So far, yes—with severe performance penalties, but with credible model quality thanks to GLM‑5.2’s Mixture‑of‑Experts design. Colibrì does not make limited RAM AI feel snappy, but it widens who can experiment. Enthusiasts no longer have to accept that their only option is a small, weaker model; they can now choose slow access to a powerful one.
From proof-of-concept pain to practical local LLMs
No one should pretend Colibrì is ready for mainstream use. It is explicitly a proof‑of‑concept, still CPU‑only, and even moving it to GPUs will not erase the bottleneck of moving slices to and from the card. In this architecture, NVMe speed is the first major bottleneck, but the “funnel” changes across configurations. Today, you will not hit the 20–30 tokens per second usually considered comfortable for interactive use. That is acceptable for experiments, not for replacing a commercial chatbot.
Yet the momentum is clear. The project was barely released and it is already popular; benchmark data and fixes are actively collected, with community contributions encouraged. According to the Colibrì repository, “Maybe at some point it’ll be feasible to run a really clever model on high‑end consumer hardware at a decent enough clip.” Combine this direction with practical tactics from today’s local LLM users—scheduling long jobs overnight, tolerating slow generations, and being selective about what must stay private—and a pattern emerges. The future of powerful LLMs on limited hardware will be less about chasing raw speed and more about giving ordinary users control: choosing privacy, deciding when to wait, and experimenting with frontier AI on machines they already own.
