Muse-Glimmer-30B-geoquant-GGUF
Two GGUF quantizations of Muse-Glimmer-30B, with benchmark results: muse-v4 runs on mainline llama.cpp; muse-lowbpw-r1 uses geo-lucebox or the documented llama.cpp fork.
Two GGUF re-quantizations of meta-models/Muse-Glimmer-30B
(dense 29.8B agentic VLM, LM tower only), built by allocating bytes from
measured per-tensor damage rather than from a named preset.
Table 1
muse-v4sits 812 KB under the official 17 GB build and scores above it
on every repeat run. Stock K-quants only — no custom loader.muse-lowbpw-r1matches the official build's measured band using 69 %
of its bytes, by letting two custom low-bit formats compete against stock
K-quants per tensor. Needs one of the two runtimes below — mainline
llama.cpp rejects the file rather than misreading it.
Both are text-only on their own. For image input, pair with the vendor'smmproj-kquant.gguf from the official GGUF repo.
What we compared against
Every number below is our measurement of their published artifact through
the same harness, not a vendor-reported score. We did not re-quantize anyone
else's model or tune against their choices.
Table 2
The first-party build is the one the vendor ships and the one our byte ceiling
is sourced from. Unsloth is included as an independent, widely-used community
quantizer so the comparison is not purely against the model's own authors.
UD-IQ2_XS is within 0.11 % of muse-lowbpw-r1's size (theirs is 13 MB
larger), which makes that pair the cleanest head-to-head in the set.UD-Q4_K_XL is the nearest Unsloth build to muse-v4 but is 5.2 % smaller,
so it is not a byte-matched comparison and is not reported as one.
Benchmark
A 122-item agentic golden suite — MATH 40 / CODE 40 / TOOL 30 / AGENT 12 —
scored by a deterministic grader. Corpus perplexity and token agreement are
not used as gates; they are smoke tests that do not move with agentic ability.
Every artifact is served identically and scored by the same grader:
- llama.cpp
llama-server, masterdd1ea52, CUDA arch 90 -ngl 99 --ctx-size 16384 --parallel 4 --special- greedy sampling; chat template rendered client-side from the HF tokenizer
- per-axis
max_new_tokensrecorded in each results file
--special is load-bearing: serving with control tokens stripped measures
TOOL 0/30, because ATEM tool calls are delimited by them.
Controls
The gate is only worth reading because it has been shown to move in both
directions:
- Positive control — official kquant-17gb scores 101/122.
- Negative control — a uniform Q2_K crush (
--pure, 2.63 bpw) scores
1/122. The gate fails hard on a broken artifact. - Instrument validation — grader
--self-check122/122 against canonical
answers, plus bf16 anchor reproduction re-run after every grader change.
Reported ranges are repeat runs of the same artifact at the same config
(n given), not confidence intervals. Single-run numbers are labeled n=1.
Results
Table 3
Claims, each stated at the strength the data supports:
- muse-v4 beats the official build at byte parity. Worst run (103) ≥ best
official run (101), and it is 812 KB smaller. TOOL exceeds even the bf16
reference (27 vs 26) in all three runs. - muse-lowbpw-r1 beats the size-matched Unsloth build. 101–103 against
96–98 at 13 MB less. The bands do not overlap: our worst run clears their
best by 3 items. The gap is mostly MATH (32·30·31 vs 28·29·29). - muse-lowbpw-r1 matches the official band on 69 % of the bytes. Worst run
(101) equals the official build's best. A match claim, not a beat claim. - muse-lowbpw-r1 also matches UD-Q4_K_XL on 72 % of its bytes (101–103 vs
100–102). The bands overlap, so this is a match, not a win — but the byte
asymmetry runs in our favor, so it is not confounded. - AGENT is the standout. muse-lowbpw-r1 holds bf16 parity (8·8·8) and is
the only quantized build here that does — official 6·7, UD-Q4_K_XL 6·6·6,
UD-IQ2_XS 7·7·7, muse-v4 6·7·7. That is the axis that normally degrades
first.
Stated fairly in the other direction: Unsloth's UD-Q4_K_XL matches the
first-party 17 GB build (100–102 vs 100–101) while being 5.2 % smaller — a
good result for it. muse-v4 scores above it, but muse-v4 is 878 MB (5.5 %)
larger, so that particular pairing is not byte-matched and we do not claim
it as a win.
Serving throughput
Measured with bench_muse_decode in geo-lucebox
(prefill once, time N single-token steps, no sampling), batch 1:
Table 4
muse-lowbpw-r1 decode on CUDA was 26.0 tok/s until a kernel fix landed
upstream of this release (2.06×, bit-identical output). On AMD the same change
is neutral, so those figures reflect the current kernels on both.
With speculative decode against the vendor DFlash drafter (dflash_server --draft, full 122-item suite on H200), muse-lowbpw-r1 reaches 81.0 tok/s —
1.54× its own autoregressive 52.6 — at 103/122, inside the ±2 the suite is
noisy to. The gain is workload-dependent: it tracks draft acceptance, which is
higher on reasoning and code traffic (0.32 on the suite) than on open-ended
chat (0.15–0.24), so short conversational turns will see less than 1.54×.
On the geo-lucebox runtime, batched multiplies for qtypes 105/106 are routed
per multiply on batch width: narrow batches (speculative verify) take the
quantized MMQ kernels, wide ones (prefill) take dequantize + dense GEMM, because
neither wins both regimes. It is automatic — there is no flag to set — and it is
why the speculative figure above does not come at the cost of prefill. Details inserver/docs/MUSE_GLIMMER.md.
The llama.cpp fork below loads and runs the same artifact, but does not carry
that routing — its 105/106 multiplies take the dequantize path at every batch
width. The throughput figures in this section were measured on geo-lucebox.
Running them
muse-v4 — mainline llama.cpp, nothing special:
llama-server -m muse-v4.gguf -ngl 99 --ctx-size 16384 --specialmuse-lowbpw-r1 — requires a runtime that understands qtypes 105/106
(Q3_1_ROCMFP3_MIX / Q2_1_ROCMFP2_MIX) and the geoquant.dmix2.sidecar KV.
Mainline llama.cpp will reject the file rather than misread it. Two public
options:
- llama.cpp fork —
GeometricAGI/llama.cpp, branchmuse-rocmfpx-cuda
(CUDA + ROCm):
llama-server -m muse-lowbpw-r1.gguf -ngl 99 --ctx-size 16384 --specialGeometricAGI/geo-luceboxdflash_server— a standalone CUDA/HIP server with native support for this
model family (ATEM chat format, tool-call parsing, DFlash speculative decode
against the vendor drafter). Seeserver/docs/MUSE_GLIMMER.mdthere.
- Full GPU offload of every mix layer is mandatory. Decode for these qtypes
is GPU-only; the loader refuses partial offload rather than silently
producing wrong numbers. On unified-memory parts (Strix Halo) a host pointer
can read something valid-but-wrong instead of faulting, which is why the
refusal is by name and not by fault. - The sidecar KV must exactly cover the resident mix tensors; any mismatch
refuses the file.
Chat template
ATEM: <|start|>role[ to=recipient]<|message|>…<|eom|>/<|eot|>, with<atem:function_calls> blocks and a to=self reasoning channel. Serve with--special. <|eom|> ends a segment, not the turn — treating it as a stop
token truncates every reply at the end of its reasoning.
Method, briefly
- Allocation is measured, not hand-picked. Every byte decision comes from
a per-tensor damage table (relative layer-output error on 128×2048-token
activations of a domain calibration bundle) fed to a knapsack under a sourced
byte ceiling. Formuse-lowbpw-r1the custom low-bit rungs compete in that
same knapsack and are selected only where measurement favors them — which is
why they appear on 71 of 418 tensors rather than everywhere. - Calibration is domain-representative and contamination-audited. 400
agentic traces (math reasoning, code, ATEM tool episodes, chat), audited
against the golden suite by 60-char shingle containment: zero hits. - No external imatrix files are consumed. Rungs ≥4 bpw are encoded
stock-unweighted; sub-4 bpw rungs are weighted by per-channel calibration
moments captured in-process (worth +3 gate items at identical bytes here).
Limitations
- LM tower only. Vision needs the vendor
mmproj; we did not quantize or
evaluate the perception encoder. - Single model family, single scale. These results are a strong prior for
this model, not a general law about the formats. - The 122-item suite is small enough that ±2 items is noise; that is why bands
from repeat runs are reported instead of single numbers, and why the claims
above are phrased against worst/best runs rather than means. muse-lowbpw-r1needs a non-mainline loader (two public options above). If
that is a problem, usemuse-v4.
Reproducing
The serving side is public: the
llama.cpp fork branch
carries the qtype-105/106 decode kernels and sidecar registration, and
geo-lucebox carries the native
server plus its test suite. The quantization harness that produced these
files (measured per-tensor damage → knapsack allocation under a sourced byte
ceiling, golden-suite gating) is not public; the Method section above states
what it does, and the numbers in this card are reproducible against the
published artifacts with any harness that scores at the stated serving
geometry.
License
Apache 2.0, inherited from the base model — a quantization is a derivative and
cannot be relicensed. Sourced frommeta-models/Muse-Glimmer-30B,
whose card, LICENSE file, and model-card body all state Apache 2.0. Users
remain subject to the base model's
usage policy,
which travels with the weights and is not altered by re-quantization.