When four bits is still too many.

Codebooks buy the quality back. The problem is speed.

The classical answer to that regime is vector quantization: stop rounding each weight independently and store an index into a learned codebook instead. It works — and it is usually slow, because decoding a codebook index produces values that then have to be fed to a matrix-multiply kernel that wanted native hardware types all along. The dequantization tax lands hardest on prefill, where the arithmetic is dense.

gridbook's move is to remove the tax rather than pay it. Its product codebooks are constrained so that every codeword already lies exactly on the NVFP4 or FP8 hardware grid. A decoded tile is therefore not converted into a native tile — it is one, and feeds the same CUTLASS tensor-core GEMM the scalar path uses.

It ships as an out-of-tree vLLM quantization plugin with zero core patches, JIT-building its CUDA and CUTLASS kernels at model load. It attests every required operation and shape before the first forward and fails closed if the native path is unavailable — there is no Triton fallback and no slow dispatch arm to silently land in.

The allocator gets a much finer menu.

A codebook size K is a continuous knob, not a discrete format. Where the scalar lane chooses among three rungs, the codebook lane is offered dozens — and the allocator spreads across them rather than settling on one. This is read from the published 27B codebook checkpoint.

What one rung stores — and why it is called gridbook.

Every format in the family works the same way. Weights are taken eight at a time, and each group of eight is replaced by a single k-bit index into a codebook of eight-dimensional codewords. The name says the rest: grid + book. The book's entries are not free-floating floats — every codeword is a vector of values on one of the machine's own numeric grids, E2M1 for the NVFP4 family or E4M3 for the FP8 family. That constraint is the entire design: a decoded tile does not approximate a native tensor-core tile, it is one, bit-for-bit the tile a scalar checkpoint would have supplied. The format is named for how it lines up with the exact formats native to the machine.

A full 2^k table is only feasible up to k≈14, so the shipping formats are product codebooks: the NVFP4 family splits its eight weights into two 4-dim halves with a 2^(k/2)-entry book each; the FP8 family splits into four 2-dim pairs with 2^(k/4) entries each. Concretely, FP8_CB_K40 stores a 40-bit index per 8 weights, split 10/10/10/10 across four 1,024-entry pair-books: a 256-weight superblock is 160 B against 256 B for plain FP8, and the four books are a 16 KB fp16 sidecar serialized once and shared by every tensor that references them — about 0.008 bits per weight on a single 4096² Linear, and less when shared across a role.

The mechanism, animated for NVFP4_CB_K16: each group of eight weights is matched to its nearest codeword and only the index is stored — two bytes per eight weights, plus group-16 scales in the two-tier coding (9 B per superblock, against the 16 B plane scalar NVFP4 carries). A 256-weight superblock lands at 73 B where scalar NVFP4 needs 144 B and BF16 needs 512 B. Synthetic illustration; every measured number on this page is in the tables.

The ladder, in bits.

The two families present differently, and the difference matters. The NVFP4 family is every integer rung, K12–K24 — thirteen of them, odd widths included: the k % 4 law does not apply there. Its fused lane decodes through a shared-memory value LUT that takes the ceil-first ragged split (k−⌊k/2⌋ / ⌊k/2⌋ bits) natively, so no rung is privileged, and the whole ladder fits the 16 KiB LUT budget — K24 fills it exactly. The FP8 family is presented as its step-4 fused lane — the k % 4 law is a property of that family's decode geometry alone (next section). FP8 interior rungs (K29–K47 off multiples of four) remain legal in the producer and appear in older artifacts — the published 27B above spreads across five of them — but they are permanently served through the expand path and are omitted here. Chips mark current policy per rung: which codebook source ships, and the rung's fused-prefill status.

Bits per weight include everything the format itself carries: the NVFP4 rows are index + two-tier scales (4k+9 B per 256-weight superblock); the FP8 rows are the 4k-byte index body, with the per-row fp32 scale and the shared fp16 book sidecars adding ≈0.01 bpw. "vs scalar" compares each family to its own scalar format at the same grid. Learned-book adoption is measured per rung on the DSv4 routed-expert campaign; every artifact published so far — including the 27B map above — ships lattice books throughout. The NVFP4 rows carry the capacity band (≈1.8–3.3 bpw) but do not yet have teacher-backed same-rate quality validation; their fused native-FP4 prefill stays opt-in behind its serving gate.

What is deliberately absent: the signed rungs. A sign-magnitude mode (S13–S16: eight explicit sign bits plus a magnitude book, the GGUF-IQ construction) was built when the first screen suggested sign coverage was the gap. Measured, it gives no benefit on this family: at identical bytes the product mode beats it (0.6B emulated confident-KL 2.21 vs 2.75), it lost 609 of 776 matched weight-MSE comparisons against product rungs, and the rate-distortion study explains why — forcing magnitude books onto the FP4 grid costs +10% where full vector books pay +4.5%. The signed rungs remain registered and decodable for research, and are excluded from the production allow-list.

Lattice books and learned books.

Where do the codewords come from? Two answers, both grid-snapped, both Lloyd's algorithm — differing only in what they train on.

The fixed lattice is a universal book: weighted Lloyd run once on a seeded synthetic sample of the distribution the encoder actually produces after normalization, then snapped to the grid and frozen. It is deterministic, identical for every model, and it is what every published gridbook artifact ships. The one subtlety is load-bearing: the sample must be drawn at the post-scale data distribution — an early version trained at N(0,1) while NVFP4's group-16 normalization actually yields std ≈ 2.9, and reconstruction collapsed so badly (emulated KL ≈ 15) that the bug would have falsely killed the whole family had it not been caught.

A learned book starts from the lattice and runs the same grid-snapped weighted Lloyd on the layer's own decoded weight vectors, imatrix-weighted, pooled per role so the sidecar stays shared. Adoption is per-rung and measured: a learned book ships only where it beats the lattice on held-out error, certified against the production evaluator to 1.2e-8 first.

Same data cloud, both panels; ring markers are codewords, snapped to the grid every step. Left: the frozen universal lattice. Right: Lloyd iterating on the data itself. With a sparse book, placement matters and learning wins. Switch to the dense book and the two converge — the palette saturates and there is nothing left for placement to buy. Synthetic 2-D illustration of the mechanism; the measured ladder is below.

Sweep-matched held-out activation-MSE ratios, DSv4 routed experts. The advantage of learning decays monotonically as the book grows and flips sign between K46 and K47. This is the measured two-regime law: below palette saturation, codeword placement dominates the error and Lloyd wins; above it, the per-row scale dominates — the scale search's value grows from +25–29% at K28 to +398–400% at K48 — and no placement compensates. K48 is a lattice format because learning it makes it worse.

Measure, then interpolate.

A finer menu has a price of its own: before the allocator can choose, it needs a cost for every (Linear, rung) pair — and the honest way to get one is to render the rung with the production encoder and measure the damage. On a nineteen-rung ladder that stops being a detail and becomes most of the compute. This section is how the lane prices ten thousand cells while rendering about a fifth of them, and how it checks itself — including the check it currently fails.

Why not just measure everything. The stock costing path renders every offered format into a cache it keeps. Measured on the Qwen3.8-27B build, one retained rung costs 45.5 GB — the cache holds the rendered weight at ~2 bytes per parameter, independent of K — so the full ladder would want ~865 GB of cache on a 128 GB machine. The easy exit, trimming the menu to what fits, was refused on principle: a render budget would then be deciding what the allocator is allowed to consider, the exact class of heuristic this project keeps out of the platform. The menu stays full; the measurement gets cheaper instead.

What one measurement is. Render the Linear at that rung — the same imatrix-weighted encode, the same scale search, the same codebooks the exporter ships — subtract to get the weight perturbation, and contract it against the KL-adjoint probe. The result is a predicted loss change in the same currency every scalar format is priced in, so the knapsack compares a codebook rung against scalar NVFP4, FP8 or BF16 directly (the activation-side price is added on top in the same currency — it is what actually separates the 4-bit-activation NVFP4 family from the 8-bit FP8 family). The render is then dropped, consumed into that one number, so the 865 GB never exists. Sensitivity enters exactly once, through the probe: the payload contract forbids any second helping of Fisher weight, weight-MSE or activation error from multiplying in later, and every priced cell carries that assertion in its provenance.

Anchors, a panel, and one shared shape. Every Linear gets an anchor: one production render per codebook family it can take, its exact price at that rung. A panel — 32 Linears per role, chosen by hash — is rendered at every rung of the ladder. Within a segment (one family × role × book-basis; a fit never crosses those walls), each panel Linear's own level is subtracted out and what remains is pooled into a single log-linear shape over the rung index — on the NVFP4 ladder also an odd/even term, because the ragged half-book split gives odd rungs their own signature. The price of an unmeasured cell is then its anchor times the shape ratio, nothing else. Two honesty rails are structural: segments whose ladder has one rung are priced by their anchor alone — the code refuses to spend panel renders where the anchor is already strictly more faithful — and every interpolated cell is stamped in provenance as not-a-rendered-measurement.

The mechanism: measure a panel at every rung, subtract each Linear's own level, and the curves collapse onto one shared shape. A new Linear then costs one render — its anchor pins the level, the shape prices the rest of its ladder, and a held-out render checks the transfer. Synthetic illustration; the measured record is the table below.

What the fit found. The shape is strikingly stable: one step of the FP8 fused ladder (four rungs of index) multiplies the predicted damage by 0.51–0.58 — call it half per step — across every role, both book bases, and both campaigns, dense 27B and 284-B MoE alike. The NVFP4 ladder falls faster per model but spreads wider by role (×0.81–0.87 per rung), with a small measured parity wobble of up to ~8% between odd and even rungs. In production this machinery has priced two builds: the dense Qwen3.8-27B (496 Linears; 9,920 prices from 2,144 rendered cells, about three hours on the box — shipped as the 13 GB PrismaAQUA gridbook card) and the DSv4-Flash 284-B MoE (33,325 units; 334,454 prices from ~68,800 rendered cells, the 92 GB build).

The check it fails, said plainly. Both campaigns hold out rendered cells the fit never saw — disjoint Linears, off-anchor rungs, including both NVFP4 ladder endpoints on the 27B — and re-measure them. Error is scored in dex, |log₁₀(predicted / measured)|: 0.05 dex is ±12%, 0.30 dex is ×2. Both campaigns blow the 0.05-dex bar and say so in their own payloads:

Why this is reported rather than gated — by design, not by leniency. The anchor pins every Linear's absolute price exactly; only cross-rung ratios ride the fit, so a typical cell is off by ×1.2, not ×4. Nothing promotes on this surrogate: it proposes, and the artifact still faces the served gates. And the signal ships with the artifact — the 27B model card carries it, with the observed fit error priced against the objective it could bend: worth ≤8.8% of the total, and ≤4.4% of the 4-bit-vs-8-bit activation boundary decision, an order of magnitude below the activation signal that actually drives that boundary. A surrogate allowed to be this wrong in the tails is usable precisely because nothing trusts it past proposing.

How dense must measurement be? That question was itself measured, in the research lane, on a fully-measured bank (every DSv4 layer-21 expert at every rung of the LDLQ chain — ground truth with no interpolation in it). Three anchors per curve fail every fit law tried. Five anchors with a monotone shape-preserving interpolant is the cheapest configuration that passes the pre-registered bar — median ≤5%, p95 ≤15%, on all three expert projections. (Monotonicity also turns anchors into brackets: the truth at an unmeasured rung is bounded by its neighbors, so an allocation found on bounds can be re-measured along its own path until the bound is exact.) The registered follow-up burn was not started: its measured cost came in at 1.861× against a pre-registered 1.5× ceiling, and the winner sits registered, unburned. None of those interpolants is in the production path above — production remains the anchored log-linear fit, carrying its own bad-tail report.

Why the ladder stops at K48.

Three converging reasons — two measured, one structural.

The palette is saturated. At K48 each pair-book already holds 4,096 codewords over a two-element E4M3 pair. The two-regime law above shows what that means: by K48 the residual error is scale-dominated — placement is so far past saturation that learning the book makes it 54–98% worse. Index bits above 48 would be spent where bits no longer buy placement.

The ceiling above is a native format. K48 is 6.0 bits per weight. Two more rungs of index would land at 6.5–7 bpw — closing on plain FP8 at 8.0 bpw, which is exact on the same grid, served natively, with no index to decode and no sidecar to load. The band between K48 and FP8 prices itself out: a shrinking quality discount, bought with a decode stage the scalar format simply does not have.

The grid was never the constraint. The rate-distortion study measured the cost of forcing codewords onto the E4M3 grid at under 1% against unconstrained codebooks. The codebook's whole value at the top of the ladder is index-rate against the scalar format's fixed 8 bits — and that argument runs out at 6.

The prefill problem, and the k % 4 law.

A codebook weight cannot be multiplied. Before any matrix math, indices must be decoded back into grid values — and where that decode happens is the entire performance story of the format. At decode time (a handful of tokens in flight) the GEMV absorbs it. At prefill, thousands of tokens hit a dense tensor-core GEMM that wants a dense weight tile, and the decode has to happen somewhere first. Three schedules have been built and measured:

• Transient expand — decode the whole tile to memory once, then run the dense GEMM. The shipping answer for large prefill: after the CUDA expander landed, the dense-lane decode tax is ~10% of prefill time.
• Fused decode-in-prologue — each GEMM block decodes the sub-tile it needs in shared memory, so decoded weights never touch main memory. Wins in the mid-batch window (one block per column, no redundancy); at prefill widths it re-decodes the same tile ~11× and loses.
• Persistent-N — decode once, hold resident, stream all tokens through. Built, bit-exact, and measured 2–5.7× slower than expand at 27B dense shapes — a clean negative, quarantined, because the expander had already cut the tax it was designed to remove. The MoE variant remains the open target: expert expand is still ≈35% of a MoE layer's time.

The fused lane is where the k % 4 law comes from, and it is a law, not a preference. Two independent hardware facts converge on the same predicate: the tensor memory accelerator fetches a superblock's 4k-byte index body only if it is a 16-byte multiple (4k % 16 == 0 ⟺ k % 4 == 0), and the fused decoder extracts sub-indices at a single width k/4 — an off-law rung like K47 splits 12/12/12/11, and a uniform decode of a ragged split is wrong, not merely slow. So {28, 32, 36, 40, 44, 48} is the whole fused lane, permanently: interior rungs stay legal to produce and are expand-served forever. The law is the FP8 family's alone. The NVFP4 family decodes its two half-books through a per-rung shared-memory value LUT that takes the ceil-first ragged split as the real layout, so every rung K12–K24 is fused-eligible there; its gate is a capacity budget — the LUT must fit 16 KiB of shared memory, which the whole ladder does, K24 exactly — not an alignment law. The published 27B above is the honest illustration — its most-used rung is K47, allocated before the law was established, riding the expand path; the 35B run that followed used the six law rungs exclusively.

The measured faces of the prefill tax, spanning its range: on Hy3-295B the native-tile design is the win — ~2.6× faster prefill than the matched-byte GGUF build, which pays its dequant on general CUDA cores. On the 27B dense artifact the residual tax shows as 1.44× the scalar baseline's time-to-first-token. And on the 35B MoE it is the open problem: 3.46 s vs 0.325 s native, because grouped expert decode has not yet been fused. One format, three honest numbers.

Measured, at matched bytes.

Three things stated plainly, because they cut against the pitch. Base decode still trails the GGUF build — 14.6 vs ~18 tok/s. After three kernel rounds the wall is measured, not guessed: the 4-bit codebook decode chain is compute-bound at GEMV shapes under the bit-exact decode contract, not bandwidth-bound, so better staging will not fix it. Speculative decoding, which the GGUF build cannot carry, closes most of the gap at 16.1 tok/s. The tool-use score is read as parity, not a win: the same bytes measured 85–88 across serving configurations, and the GGUF family's own band is 86–87, so a one-point read sits inside the noise. And the 35B's prefill is the open problem, not a win: 3.46 s to first token vs 0.325 s for the native build, because grouped expert decode is not yet fused — the MoE counterpart of the expand tax the dense lane already paid down.

One further disclosure: the 27B model card reports this comparison as −77% ALL-KL where the benchmark page reports −58.3%. They are two different sessions and two different builds; the project documents the discrepancy rather than picking the flattering number. The conservative figure is the one used here.

The full testing record.

Everything the lane has measured, in order, with the tier of evidence each result carries. The house rule ranks them: an emulation screen can green-light an experiment, only a served A/B on the exported artifact can promote one. Negative results are listed with the same weight as wins — several of them are why the current design looks the way it does.

Where it stands.