Over 107tok/s Qwen3.8-Flash-Next on M3 Ultra
Our fastest recorded configuration reaches 107.16 tok/s versus 55.74 tok/s for llama.cpp with MTP3, on a Rust sorted-slice merge task with 4,096 prompt tokens and 512 generated tokens.
Latest Posts
Mac mini as 10 GbE firewall
A Mac mini with the 10 GbE option is an interesting firewall box. The hardware has more memory bandwidth than a 10 Gbit/s link can consume, it is quiet, and Apple Silicon gives the CPU and GPU access to the same physical …
Using Apple Metal to accelerate DPI recognition
GPU acceleration for DPI sounds attractive, but the wrong design can make a router slower. The expensive part is not just “running math.” The expensive part is moving packet data, synchronizing CPU and GPU …
Thoughts on BBR congestion control in QUIC under high loss
BBR is a natural fit for QUIC, but it is not magic. It works well when packet loss is a bad congestion signal. It can work poorly when loss corrupts the measurements that BBR itself needs. That distinction matters on …
GRE over WireGuard mesh with BGP and BIRD
A WireGuard mesh gives each site a secure underlay. BGP gives each site a way to tell the rest of the network which prefixes it can reach. GRE, placed inside WireGuard, gives each adjacency a normal point-to-point tunnel …
Using DPI to distinguish QUIC, HTTP/3, and HTTP/2
Deep packet inspection for modern web traffic is less about reading application payloads and more about making fast decisions from the few bytes that are still visible. That is especially true for QUIC. After the …