{"id":"5f076278-51cb-4310-ac2b-b906bf62e933","arxiv_id":"2606.04115","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"dMX learns continuous per-layer MXFP bit-width offsets with temperature annealing and target-aware regularization, yielding Pareto-better mixed-precision LLMs than uniform or KL-heuristic baselines.","lead":"dMX learns per-layer floating-point bit-widths for MXFP formats by optimizing a continuous offset with temperature annealing and a target bit-width regularizer. It produces better accuracy–bit-width trade-offs than uniform MXFP or KL-based layer selection on small LLMs.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Pareto claim rests on single-run, small-scale PTQ without multi-seed variance or larger-model checks, so intermediate-bit-width gains may not be robust.","rationale":"The reader correctly identifies the narrow evaluation regime (1–2B models, 400 steps, bit-width as sole cost proxy, no multi-seed stats) as the weakest assumption underwriting the Pareto claim. My concern is a concrete instantiation of that same assumption: without variance estimates it is impossible to know whether the intermediate-bit-width gains over KL heuristics (Fig. 5) and ROUND+STE (Fig. 8) are reliable. The method itself (continuous β, annealing, target-aware R) is clearly described and the ablations are useful; the issue is not internal inconsistency but insufficient statistical and scale support for the strongest empirical claim. Hence the verdict remains CONDITIONAL and agreement with the reader is full. A multi-seed re-run on the critical intermediate targets would settle whether the claim lands inside the reported regime; larger models remain desirable but are secondary to first establishing reproducibility of the existing curves.","tokens_in":18677,"tokens_out":668,"duration_ms":5635,"concrete_test":"Re-run the Llama-3.2-1B MXFP8/MXFP4 target-aware series (targets 5.0 and 6.0) for 3 independent random seeds (data order + β init), report mean±std of WikiText-2 PPL and zero-shot accuracy, and compare the gap to the corresponding KL-preselected points; if the mean gap shrinks below ~0.2 PPL or the std bands overlap the KL curve, the intermediate-bit-width Pareto claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim is that dMX yields Pareto-dominating models vs homogeneous MXFP and KL heuristics at intermediate average bit-widths (roughly 5–7) on three 1–2B models. That claim is load-bearing on the empirical curves in Tables 1–2 and Figures 3–5/7–8. Those curves come from single calibration runs (400 steps on 3200 FineWeb samples, joint with SpinQuant-style rotations) with no multi-seed error bars, no reported variance, and no models beyond ~1.7B. The continuous β + temperature annealing + target-aware regularizer can produce different discrete assignments under modest seed or hyper-parameter changes (T schedule, Tratio=60%, λ=5, SGD lr=1 for β). If the intermediate-bit-width advantage over KL pre-selection or ROUND+STE is smaller than run-to-run noise, or vanishes at larger scale where cross-layer interactions differ, the Pareto-dominance statement does not hold beyond the reported regime. The paper itself flags scale as future work (Sec. 5) and uses average bit-width as the sole cost proxy, so the central empirical claim is currently under-supported for robustness.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes dMX, a gradient-based post-training framework for mixed-precision assignment over OCP microscaling floating-point formats (MXFP8/MXFP6/MXFP4). Each layer is parameterized by a single continuous offset β that defines a format of the form E(2+β)M(1+β) (or the MXFP6/MXFP4 variant), with weight and activation bit-widths tied. Continuous formats are used during calibration; a temperature-annealed sigmoid progressively maps offsets to the discrete hardware endpoints. A target-aware regularizer on average bit-width (simple or size-weighted) steers the budget. Calibration jointly optimizes β and SpinQuant-style rotations on 3200 FineWeb samples (400 steps). On Llama 3.2 1B, Qwen3 1.7B and SmolLM2 1.7B the method reports lower WikiText-2 perplexity and higher average zero-shot accuracy than homogeneous MXFP baselines and a KL-divergence layer-selection heuristic at intermediate average bit-widths, with ablations of regularization form, continuous vs ROUND+STE, and format pairs. Appendix A supplies closed-form STE gradients that match autograd.","tokens_in":19078,"tokens_out":1326,"duration_ms":11603,"significance":"If the empirical Pareto claim holds under modest robustness checks, the work is a useful and timely contribution: it is the first gradient-based bit-width allocation method specialized to MX floating-point formats, supplies a clean continuous parameterization plus annealing that avoids the oscillations of naive rounding+STE, and is compatible with existing PTQ pipelines (Brevitas, SpinQuant rotations). The closed-form gradients in Appendix A and the systematic ablations (target-aware vs simple penalty, continuous vs ROUND+STE, learned vs KL) are concrete strengths. The practical impact is currently limited by the 1–2B model scale and the use of average bit-width as the sole cost proxy, both of which the authors flag as future work.","major_comments":[{"comment":"Tables 1–2 and Figures 3–5/7–8: the central Pareto-dominance claim (vs homogeneous MXFP and vs KL pre-selection at intermediate bit-widths ~5–7) rests on single calibration runs with no multi-seed error bars or reported variance. Given free parameters (T schedule, Tratio=60%, λ=5, SGD lr=1 for β, 400-step budget) and the discrete nature of the final assignment, modest seed or hyper-parameter variation could move the intermediate-bit-width advantage inside run-to-run noise. At least a small multi-seed study (or sensitivity sweeps) on one model is needed before the claim can be treated as robust.","section":"Tables 1-2, Figures 3-5"},{"comment":"Sec. 5 and experimental setting: all results are on models ≤1.7B. Cross-layer quantization interactions and the value of end-to-end β optimization may change at larger scale or for MoE architectures. The paper correctly lists scaling as future work, but the abstract and introduction state the Pareto claim without that qualifier; either add a clear scope statement or provide at least one larger-model check so the claim is not over-generalized.","section":"Section 5, Abstract"},{"comment":"Sec. 2.3: average bit-width (simple or size-weighted) is used as the sole proxy for inference cost. On real MX hardware, latency/energy can depend on format-specific throughput, memory hierarchy, and activation vs weight traffic in ways that a scalar average does not capture. The paper acknowledges this as a coarse proxy; a short discussion of when the proxy is expected to be adequate (or a simple hardware-aware alternative) would strengthen the deployment claim.","section":"Section 2.3"}],"minor_comments":[{"comment":"Abstract and Sec. 2.1: repeated word 'format format' and occasional missing spaces around equations; a light copy-edit pass would help.","section":"Abstract, Section 2.1"},{"comment":"Figure 1 caption and surrounding text: the continuous-grid illustration is useful but the discrete E2M1 reference is only briefly mentioned; a short sentence clarifying what continuous e,m produce would aid readers unfamiliar with the construction.","section":"Figure 1"},{"comment":"Appendix B: hyper-parameter values (T from 8 to 400, Tratio=60%, λ=5, bit-width SGD lr=1) are listed; stating whether any of these were tuned per model or held fixed would improve reproducibility.","section":"Appendix B"},{"comment":"Related Work (Sec. 4): MicroMix is cited for channel-wise MX assignment; a one-sentence contrast of layer-wise continuous β vs channel-wise heuristics would make the novelty claim sharper.","section":"Section 4"},{"comment":"Tables 1–2 report only a subset of targets (4.5/5/6/8); the full curves are in the appendix. Cross-referencing the appendix figures more explicitly in the main text would help readers locate the complete Pareto fronts.","section":"Tables 1-2"}],"recommendation":"major_revision","confidential_remarks":"The technical core (continuous MXFP(β) + temperature annealing + target-aware regularizer) looks sound and the ablations are informative. The main risk is over-claiming from single-run 1–2B results; a major-revision request focused on multi-seed checks and clearer scope language should be sufficient. Fit for a solid ML systems / efficient-LLM venue is good if robustness is addressed."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The new piece here is a continuous β-offset that folds MXFP formats into E(2+β)M(1+β) (or the MXFP6/MXFP4 variant), plus temperature-annealed sigmoid discretization that avoids the ROUND+STE oscillations you get when the gap between FP4 and FP8 is large. That combination, plus a target-aware average-bit-width regularizer, is the actual contribution; integer mixed-precision learning already existed, and MicroMix is heuristic.\n\nWhat they do well: the parameterization is clean, Appendix A gives closed-form STE gradients that match autograd, and the ablations are useful (target-aware vs simple penalty, continuous vs ROUND+STE, learned vs KL pre-select). On Llama-3.2-1B, Qwen3-1.7B and SmolLM2-1.7B the intermediate-bit-width (roughly 5–7) points sit above the homogeneous MXFP baselines and above the KL heuristic, and they hit the requested average bit-width reasonably well. Joint optimization with SpinQuant-style rotations is a practical plus. Citations look honest; they do not over-claim novelty relative to FracBits/SDQ/etc.\n\nSoft spots, in proportion: the stress-test note is right that everything is single-run, 400 steps on 3200 FineWeb samples, no multi-seed bars, and nothing past ~1.7B. Average bit-width is only a coarse cost proxy. The paper itself flags scale as future work. That does not kill the method, but it means the “consistently Pareto-dominating” claim is currently supported only inside a narrow regime. Hyper-parameters (T schedule, Tratio=60%, λ=5, β lr=1) are free and could move the discrete assignments. No public code is mentioned.\n\nThis is for people already shipping MXFP PTQ on recent accelerators who want a better layer assignment than KL or uniform. It is not foundational, but it is a competent, reproducible-looking engineering advance. I would send it to peer review; the core idea is clear enough and the empirical package is already better than many PTQ notes. Engage if you care about MX mixed precision; otherwise skim the ablations and move on.","headline":"Solid first gradient-based MXFP mixed-precision PTQ method; Pareto gains look real on the reported 1–2B models, but single-run small-scale evidence is the main soft spot.","tokens_in":19672,"tokens_out":578,"would_cite":true,"duration_ms":5361,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A single continuous offset per layer lets LLMs learn mixed MXFP bit-widths that beat uniform low-precision and greedy heuristics.","keywords":["mixed-precision quantization","MXFP","differentiable bit-width","temperature annealing","LLM post-training quantization","floating-point formats","microscaling"],"falsifier":"On the same 1–2 B models and calibration budget, either a KL-ranked mixed-precision assignment or a rounding-plus-STE baseline matching or beating dMX’s Pareto front at intermediate average bit-widths (roughly 5–7).","tokens_in":19556,"feed_emoji":"📉","tokens_out":596,"duration_ms":5630,"temperature":0.7,"pith_summary":"Uniform low-precision floating-point formats such as MXFP4 hurt LLM quality, yet choosing different formats per layer is a hard combinatorial search. dMX turns that search into ordinary gradient descent: each layer is given one continuous scalar that smoothly interpolates between hardware-legal MXFP formats, a temperature schedule slowly hardens the choice into a discrete format, and a simple average-bit-width penalty steers the model toward a user budget. On 1–2 B models the resulting mixed-precision assignments sit above the uniform MXFP baselines and above KL-based layer ranking on both perplexity and zero-shot accuracy, especially in the intermediate bit-width band where the budget must be spent carefully. The practical payoff is a short post-training calibration that produces Pareto-better quantized models without full retraining or hand-crafted sensitivity metrics.","feed_headline":"One continuous number per layer picks better MXFP bit-widths","feed_subtitle":"Gradient descent plus temperature annealing beats uniform low-precision and greedy heuristics on small LLMs","key_machinery":"The shared offset β that defines the format E(2+β)M(1+β) (or the MXFP6/MXFP4 variant), mapped through a temperature-annealed sigmoid so that continuous values used in the forward pass gradually collapse onto the two hardware-legal endpoints.","core_discovery":"Parameterizing every layer’s MXFP format by a single continuous offset β, annealing it with a temperature-controlled sigmoid, and regularizing the average bit-width yields mixed-precision models that dominate both homogeneous MXFP baselines and KL-divergence layer-selection heuristics on perplexity and zero-shot accuracy for Llama, Qwen3 and SmolLM2.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["One continuous offset per layer assigns mixed MXFP bit-widths","dMX anneals continuous β to Pareto-better MXFP mixes on LLMs","Learnable offsets plus annealing beat uniform and KL MXFP picks","Single continuous parameter yields dominating mixed-precision MXFP","Temperature schedule turns continuous offsets into hardware MXFP"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That one continuous offset per layer plus a temperature schedule and a simple average-bit-width penalty is enough to capture how quantization errors interact across layers and still land on legal hardware formats.","fun_headline_variants_meta":{"raw":{"variants":["One continuous offset per layer assigns mixed MXFP bit-widths","dMX anneals continuous β to Pareto-better MXFP mixes on LLMs","Learnable offsets plus annealing beat uniform and KL MXFP picks","Single continuous parameter yields dominating mixed-precision MXFP","Temperature schedule turns continuous offsets into hardware MXFP"]},"model":"grok-4.5","effort":"low","cost_usd":0.003598,"raw_usage":{"total_tokens":1227,"prompt_tokens":845,"num_sources_used":0,"completion_tokens":68,"cost_in_usd_ticks":35980000,"prompt_tokens_details":{"text_tokens":845,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":314,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":845,"tokens_out":68,"duration_ms":3298,"temperature":1.0,"reasoning_tokens":314,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-15T10:56:08.041335+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same 1–2 B models and calibration budget, either a KL-ranked mixed-precision assignment or a rounding-plus-STE baseline matching or beating dMX’s Pareto front at intermediate average bit-widths (roughly 5–7).","supporting_citations":[],"review_version":2}