Mixed-precision allocation · measured, not guessed
The right format for every weight matrix.
Most quantization picks one format and applies it everywhere. PrismaQuant decides per Linear — how many bits, and in which hardware format — by measuring what each choice actually costs the model end to end. The result ships as a stock checkpoint that vanilla vLLM serves with no forked runtime and no custom kernels.
Qwen3.6-27B PrismaAURA · 5.5 bpp · the shipped artifact
loading…Every cell is one Linear layer, coloured by the format the allocator chose for it. Columns are the model's 64 blocks, left to right; rows are projection types. This is read directly out of the published checkpoint's own metadata, so it is the allocation vLLM actually loads. Hover any cell for its assignment.
Two questions, and only one of them was open.
The local question — well studied
Given a fixed format, how do you round this matrix best?
GPTQ, AutoRound, scale sweeps, rotations. A deep and crowded literature, and PrismaQuant uses it: this toolkit runs underneath whatever format gets chosen. It is not where the remaining headroom is.
The global question — PrismaQuant's contribution
How many bits should each Linear get, and in which format?
A per-Linear allocation of a total bit budget across {BF16, FP8, NVFP4, FP8_SOURCE}. Sensitivity is not uniform down a model — roughly 5% of Linears carry most of the damage — and a single format cannot express that. Measurement finds them; intuition does not.
The method in one line: surrogates generate, real KL selects. A cheap cost model proposes candidate allocations; expensive, faithful end-to-end KL on a held-out split decides which one ships. Cross-layer interactions stop being quantities you must model and become quantities you observe.
What measurement buys.
Every number here is a served-artifact measurement — vLLM KL against the BF16 source, or a deterministic benchmark run against the served endpoint. Never a local screen. Where a KL was taken against a truncated teacher distribution it is marked as a lower bound on the artifacts page, because that is not the same quantity as a full-vocabulary one.
ToolEvalBench hardmode, temp 0, same seed — above the BF16 source at 86, and the prior 5.31 bpp flagship at 85. At 5.5 bpp and 23 GB.
KL-vs-BF16 on the served artifact — a −40.9% reduction over the prior AURA build at the same bit-rate, from adding the FP8 middle rung to the menu plus render/export fidelity fixes.
Confident KL-vs-BF16 0.0143, top-1 agreement 98.6%, with a grafted MTP head reaching 91.3% draft acceptance at position 0.
Against uniform NVFP4, on the same model
RedHatAI's uniform NVFP4 build of Qwen3.6-35B-A3B keeps 342 hand-picked Linears in BF16. The measured allocation ships 2 GB smaller with ~90 fewer Linears in BF16, and lands about 4× closer to the BF16 source on mean zero-shot delta: −0.56 pp versus −2.21 pp, winning 8 of 9 metrics. Uniform precision collapses the sensitive minority. Measurement finds them.
Where full precision actually goes
In the flagship above, 57 Linears stay at BF16 — but they account for 0.4% of the parameters. Full precision is spent where it is cheap and the model is fragile, not where the tensors are large. That is the entire argument for allocating rather than picking a format, and it is visible in the map.
Shipped, and served by stock runtimes.
Three containers, none of which require a forked engine: compressed-tensors loads in vanilla vLLM, gguf loads in llama.cpp and vLLM, and codebook loads through gridbook, an out-of-tree vLLM plugin with zero core patches.
| Artifact | Container | bpp | Size | Measured |
|---|---|---|---|---|
| Qwen3.8-27B PrismaAQUA | ct | 5.50 | 23.6 GB | KL 0.0338 mean, p99 0.292 (lower bound — truncated teacher) · PPL 9.580 vs BF16 9.361 |
| Qwen3.6-27B PrismaAURA | ct | 5.5 | 23 GB | KL 0.0342 · ToolEvalBench 91/100 |
| Qwen3.6-27B PrismaSCOUT (DOI) | ct | 5.31 | 20.17 GB | Held-out KL 0.0151 |
| Ornith-1.0-35B-A3B | ct | 4.75 | 23 GB | KL 0.0143 · top-1 98.6% · MTP 91.3% |
| Qwen3.6-35B-A3B | ct | 4.75 | — | 8 of 9 zero-shot metrics vs uniform NVFP4 |
| Tencent Hy3-295B-A21B | gguf | 2.80 | 103.7 GB | no quality claims — see Limits |
These rows are not a ranking. The KLs are measured on different models against different teachers, one of them truncated, so the top two figures being close is a coincidence of scale and not a comparison — and the bit-rate labels are not comparable across artifacts either, only the disk sizes are. The 295B carries no quality claim on purpose: a BF16 teacher cannot be served on one box, so there is nothing to measure KL against. Load and coherent-generation smokes plus bit-exact packing are what is honestly available, and that is what is stated. Full artifact list →
— · — in the last 30 days
No forked runtime, no custom kernels on the
compressed-tensors lane. vllm serve and it loads.
Built and measured end to end on a single NVIDIA GB10 / DGX Spark, Blackwell sm_121.