Published on Hugging Face · rdtand
Everything shipped, and what stands behind it.
Quality claims here are served-artifact measurements: vLLM KL against the BF16 source, or deterministic benchmark runs against the served endpoint. Where no such measurement is possible, the row says so instead of substituting a weaker one. Where a KL was taken against a truncated teacher distribution rather than the full vocabulary, the row says that too — such a number is a lower bound on the divergence, not an equivalent measurement.
Dense and hybrid models
| Model | Artifact | bpp | Size | Measured |
|---|---|---|---|---|
| Qwen3.8-27B | PrismaAQUA first activation-aware allocation | 5.50 | 23.6 GB | KL-vs-BF16 0.0338 mean over 4088 positions, p99 0.292, max 1.329 · WikiText-2 test PPL 9.580 against the BF16 source's 9.361 on the same corpus — a cost of +2.34%, or +0.023 nats/token ‡ § |
| Qwen3.6-27B | PrismaAURA current flagship | 5.5 | 23.6 GB | Served KL-vs-BF16 0.0342 · ToolEvalBench 91/100 vs BF16 86 |
| Qwen3.6-27B | PrismaSCOUT DOI 10.57967/hf/8656 | 5.31* | 20.2 GB | Held-out KL 0.0151 · ToolEvalBench 85/100 |
| Qwen3.6-27B | v1 | 5.5 | 22.7 GB | Superseded by the two rows above; no served measurement republished here. |
| Gemma4-31B-IT | 6 bpp · 5.5 bpp | 6.0 | 27.2 GB | −24% confident-position KL-vs-BF16 and +5.9 pp top-1 agreement over the 5.5 † |
| Mistral-Medium-3.5-128B | 4.75 bpp | 4.75 | 84.1 GB | — |
‡ That KL is a lower bound, not a full-vocabulary measurement. The BF16 teacher's distribution was stored as its top 1024 logprobs per position plus one lumped bucket for the remaining mass (coverage mean 0.988, minimum 0.573). Coarse-graining a distribution can only reduce a KL, so the true divergence is at least this and never less. It is reported because it is what was measured, and labelled because it is not the same quantity as the exact figures elsewhere on this page. Restricted to the 2067 of 4088 positions where the teacher is confident — its top-1 token above 0.5 — the same measurement reads 0.0187. § No downstream task suite has been run on this artifact. No GSM8K, IFEval, MMLU or ToolEvalBench number exists for it, so none is quoted. What it did clear is the pre-ship gate — serve, generation sanity, perplexity and MTP acceptance all pass, against thresholds of 25.0 perplexity, 3.0 mean and 6.0 p99 per-prompt NLL — and it loads and generates in vLLM in both eager and CUDA-graph modes. The gate's own measured values are not quoted here because the artifact's shipcard records that slot as a pass without stamping them, and an unstamped number is not evidence.
One comparison this artifact does not support, and it is the one a reader most wants: there is no matched-bit-rate uniform build of Qwen3.8-27B here — no uniform NVFP4 arm, no uniform FP8 arm, and no weight-only-cost arm that was actually served. The numbers above say this artifact is good. They do not, on their own, say that per-Linear allocation is why. The argument for that rests on the Qwen3.6-35B-A3B comparison against uniform NVFP4, which was run.
Mixture-of-experts models
| Model | Artifact | bpp | Size | Measured |
|---|---|---|---|---|
| Ornith-1.0-35B-A3B | PrismaAURA + MTP | 4.75 | 24.6 GB | Confident KL 0.0143 · top-1 98.6% · MTP acceptance 91.3% |
| Qwen3.6-35B-A3B | 4.75 bpp | 4.75 | 22.9 GB | Wins 8 of 9 zero-shot metrics vs uniform NVFP4; −0.56 pp vs −2.21 pp mean delta |
| Qwen3.5-122B-A10B | 4.75 bpp | 4.75 | 76.7 GB | — |
| MiniMax-M2.7 | 3.20 bpp | 3.20 | 95.7 GB | — |
| LFM2.5-8B-A1B | 6.5 bpp | 6.5* | 8.3 GB | ToolEvalBench at BF16 parity † |
| Tencent Hy3-295B-A21B | 2.8 bpp GGUF gguf | 2.80 | 103.7 GB | No quality claims. Serves on a single Spark. |
| Tencent Hy3-295B-A21B | 5.3 bpp + MTP | 5.3 | 203.6 GB | Two-Spark configuration. |
* Bit-rate labels are not comparable across accounting eras. The 5.31 artifact's body rate is ≈4.76 under the current convention, and the LFM2.5 repo labelled 6.5 achieved ≈6.58. † Recorded in project notes and not re-verified against code for this page; treat as a lead rather than a citation.
Why the 295B rows are blank where others are not: at that scale the BF16 teacher cannot be served on the target hardware, so there is nothing to measure KL against. Validation is load plus coherent-generation smokes and bit-exact packing. Publishing a weaker proxy in that column and letting it read like a quality result is exactly the failure this project tries not to commit.
Codebook artifacts
Served by gridbook, an out-of-tree vLLM plugin with zero core patches.
| Model | bpp | Size | Measured |
|---|---|---|---|
| Qwen3.6-27B | 5.5 | 23.0 GB | ALL-KL −58.3% vs matched-bpp scalar baseline |
| Laguna-S-2.1 (117B MoE) | 6.0 | 89.4 GB | No quality claims — no servable BF16 teacher at this scale |
| Hy3-295B-A21B | 2.9 | 105.7 GB | No quality-vs-teacher claim; prefill ~2.6× the matched-byte GGUF build |
Serving them.
The compressed-tensors lane needs nothing but vLLM:
vllm serve rdtand/Qwen3.6-27B-PrismaAURA-5.5bit-vllm \
--host 0.0.0.0 --port 8000One operational warning worth repeating, because it silently corrupts
results: if vLLM is serving with --speculative-config, a perplexity run against
/v1/completions with echo and logprobs returns the draft model's NLL,
not the target's. Measure perplexity on a serve without speculative decoding.