Engineering1 min read
DeepSeek V4.1 Flash on a single B300
Custom quantization and inference optimization on a single B300
By Geometric Team
Our optimization tools discovered a quantization algorithm and the surrounding inference stack that enable DeepSeek-V4.1-Flash to run on a single NVIDIA B300.
In internal testing of the newer DeepSeek-V4.1-Flash-1017 checkpoint, we measured 260 output tokens per second and 0.48 seconds to first token with one request at a time and a 10,000-token input.
In a separate high-concurrency benchmark on an earlier checkpoint, we measured 4,722 aggregate output tokens per second across 256 concurrent requests on one B300.
Our internal evaluations of the 1017 checkpoint recorded scores of 91 on GPQA and 79.2 on Tau2 Verified Airline.