I wanted to understand where local models stand today compared to frontier models. What kind of hardware is needed? What are inference options? And what does local vs open-weight hosted on data centers look like?
According to my own benchmarks running local models in my custom harness, Qwen3.5:122b was already better than Haiku and about equal with Sonnet 4.5 on coding tasks. Qwen3.8:27b-gsq-iq3s is better than both of them and also pretty dang fast on local hardware (~50-60tk/s on GB10 128Gb VRAM).
I wanted to understand where local models stand today compared to frontier models. What kind of hardware is needed? What are inference options? And what does local vs open-weight hosted on data centers look like?
According to my own benchmarks running local models in my custom harness, Qwen3.5:122b was already better than Haiku and about equal with Sonnet 4.5 on coding tasks. Qwen3.8:27b-gsq-iq3s is better than both of them and also pretty dang fast on local hardware (~50-60tk/s on GB10 128Gb VRAM).
The harness is just as important as the model.