AURA · production-faithful KL–Fisher allocation

Surrogates generate. Real KL selects.

An allocator does not need a perfect cost model if every candidate it proposes can be cheaply re-scored end to end on a held-out split. That one sentence is the whole design. It turns cross-layer interaction from something you must model into something you observe.

1 · A cost that prices the bytes that actually ship.

The classical per-Linear cost is ½ · H_trace · MSE_W — a Fisher-diagonal trace times weight round-trip error. AURA replaces both halves.

Sensitivity from the real objective

The sensitivity term comes from KL-Fisher probes of the full model — the adjoint of the end-to-end KL objective — rather than a layer-local proxy. A Linear matters because of what it does to the output distribution, not because its own reconstruction error is large.

Error measured on the production render

The error term is measured on the production-rendered weights: GPTQ, joint scale optimization, activation ordering — the exact render that ships, not a raw round-trip. The surrogate, the KL validation and the exported bytes are the same rendering. Otherwise an A/B has a rendering confound.

And, since August 2026, the activation side

Both halves above are weight-space, and the shipping formats are not: NVFP4 is W4A4. A second Δloss term for what quantizing the activations costs is added to the first (AQUA-AURA). One published artifact — Qwen3.8-27B PrismaAQUA — was allocated with it; every earlier one was not. What it changed, and what it has not yet proven.

On served A/Bs at matched bit-rate this cost beats a strong h_trace × output_mse baseline by −38% and −39.5% confident-KL at the 4B knee across two calibration corpora, and −17.9% at 27B. Both current flagships were produced with it.

2 · One cost level, not three.

PrismaQuant was described for most of its life as a three-level cascade: additive Fisher, then a perturbed-activation fixed point, then propagated end-KL over a bounded neighbourhood. That framing was retired, and the honest reason is worth stating.

The second level beat the additive baseline by −1.5%. A better single cost beat it by −38.5% — roughly 25× the return for less machinery.
The pairwise cross-layer residual is diffuse: 3 of 1180 pairs reached significance. The apparent non-additivity turned out to be a bf16 differencing artifact — per-Linear KLs do add in fp32.
Per-Linear costs measured under one context do not sum to true end-KL when many Linears flip at once. Measured, one-at-a-time coordinate descent is the form that survives.

This is why the literature's answer — model the cross-layer bias with pairwise integer programs, second-order ILPs or Shapley games — is not the answer here. The measurements say there is little there to model, and the full quadratic program was rejected: O(N²) per-pair measurement for an optimum the O(N) path recovers to within 1–2%.

3 · The solve, and the gate.

Multi-choice knapsack

Each Linear is one decision unit with a menu of (format) options, each with a cost and a byte count. A multi-choice knapsack DP allocates the total budget across all of them. The target is a byte budget — fit the card — not a curve heuristic. Serving reality constrains the solve directly: fused siblings (q/k/v, gate/up) and packed MoE experts must share one format, enforced by union-find promotion, because vLLM will not load them otherwise.

Real KL decides what ships

The allocator's Pareto candidates are rendered and scored with real KL on a held-out split — text the surrogates never saw. An earlier audit found a "validation" set that was in-sample; that is now a hard gate. The empirical frontier, not the surrogate's opinion, picks the shipping point.

Why the gate is not ceremony: on a 27B model the surrogate's own knee picks 5.857 bpp at KL 0.056. The validated frontier picks 5.31 bpp at KL 0.015 — smaller and better. Outside the additive trust region, bit-rate order is simply not KL order.

4 · Polish that cannot make things worse.

After a frontier point is chosen, coordinate descent tries single-unit format flips and accepts one only if it strictly reduces measured real KL. That is a contractual guarantee under the fixed polish-time evaluator — provably no worse than the chosen point — and explicitly not a claim of optimality.

The gate earns its keep by rejecting things. On a Qwen3.6-4B microbenchmark at 4.5 bpp, a DP-based polish regressed KL from 0.371 to 0.461 and was rolled back by the gate; coordinate descent then recovered and surpassed it, reaching 0.245 with 6 of 101 attempted flips accepted. None of those six would have been chosen by the additive surrogate.

5 · Mixture-of-experts needs a different measurement.

On MoE models the smooth cost is route-flip-blind for routed experts: it cannot see that changing a format changes which tokens arrive. So packed experts are priced with measured empirical unit-KL instead, merged with the smooth cost into one hybrid payload.

A subtle correctness detail, recorded because getting it wrong inverts the answer: every Fisher row — dense or expert — is normalised by the same global calibration token count. Tokens never routed to an expert contribute zero gradient to the mean-Δloss objective, so dividing by an expert's own routed-token count inflates rarely-routed experts by the ratio of global to routed tokens. That is inverted importance weighting, and it shipped for a while before it was found and removed.

6 · What "measured" is allowed to mean.

The metric hierarchy is explicit, and lower rungs cannot promote anything on their own.

#	Metric	Standing
1	Exact full-vocab vLLM KL-vs-BF16 on the served artifact, matched bpp	The gold metric. Contract is n=8 × seqlen 512.
2	Direct WikiText perplexity on the served artifact	Can veto a candidate even when a narrow KL screen improves.
3	Mean NLL; KL-vs-BF16 for instruct models	Raw perplexity is meaningless on instruction-tuned models.
4	Downstream suite — GSM8K, IFEval, MMLU, ToolEvalBench	Tool use is a deep reason to optimise KL: a small probability shift at a decision point flips a call.
5	Cheap last-token "hook KL" screens	Triage only. Never final selection.

KL is a screening metric, not a standalone promotion metric. A candidate that improves calibration KL but regresses held-out perplexity or a downstream task stays research-only. Lower mean KL can hide a heavier tail — and has.

7 · The pipeline, end to end.

incremental_probe          → per-Linear KL-Fisher sensitivity (streaming)
   │
aura_cost + expert_empirical_cost
   │                       → per-(Linear, format) cost on production-rendered weights
   │                         MoE: measured empirical unit-KL for packed experts
allocator + solver         → layer_config.json + Pareto candidates
   │                         multi-choice knapsack; fused-sibling and packed-expert
   │                         union-find promotion
build_production_cache     → the one rendered-weight cache
   │
validate_assignments_kl    → real held-out KL per Pareto point
select_validated_frontier  → the shipping point
   │
export_native_compressed   → compressed-tensors / GGUF / codebook checkpoint
   │
validate_native_export     → vLLM eager + graph load, greedy smoke
validate_quantized_model   → perplexity, p99 per-prompt NLL, MMLU, MTP acceptance

The p99 per-prompt NLL gate exists because a broken 27B once passed on the mean while 80% of prompts were broken. Every ship gate on this list was added by something getting through.

See it allocate What doesn't work