190 tokens per second: a 35-billion-parameter model on a laptop A 35-billion-parameter language model shouldn’t fit in 16 gigabytes of video memory, let alone generate at nearly two hundred tokens per second. And yet it happens, provided you accept two architectural constraints and treat the key-value cache as a design variable. Here is what we measured, with the complete numbers and the parameters that mattered. ...
Seven dense days, in which the “blog” project went from being a static site with a makeshift Flask panel to a small ecosystem: a well-written CMS, images generated by the local model, and a direct line between the two machines at home that now also records what I do. I retrace everything in order, with links to the posts already written in detail. ...
Until today, every article on this blog had an aesthetic problem: no cover image. PaperMod shows covers on the home page, in lists, and at the top of each post — without an image, the layout loses half of its impact. Drawing them by hand every time, though, was the quickest way to stop publishing anything. ...
Over the past few days I have spent my time optimizing the communication between the two main machines of my setup: my gaming laptop, the Strix Scar 16, and the home server GX10. The goal was clear: create a smooth workflow that lets me leverage the server’s compute power to run heavy local models while using the laptop’s interface for everyday interaction. ...
On the ASUS Ascent GX10 I finally have the setup I wanted: Qwen3.8 Flash Next running locally, with a systemd service that starts at boot and a script to switch models on the fly. This post documents the current installation and closes the Qwen3.6 chapter. ...