{"id":"72d3ee91-f3da-44b6-a3c1-f1328c3f801f","arxiv_id":"2509.02512","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Assigning 2, 3, or 4 bits to MoE experts by Hessian trace sensitivity keeps VLM accuracy close to uniform 4-bit while reducing model size.","lead":"A new recipe decides how many bits each expert in a mixture-of-experts vision-language model should use, aiming to shrink memory without losing accuracy. It ranks experts by a Hessian-based sensitivity score instead of how often they are used.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MoPEQ's per-expert 'Hessian trace' for L=||W||_F is analytically (n-1)/||W||_F, so the bit assignment is inverse weight-norm ranking; whether this correlates with true quantization sensitivity is unvalidated and is the paper's load-bearing assumption.","rationale":"The reader already identified this exact point as the weakest assumption; my read agrees and sharpens it. The identity Tr(grad^2 ||W||_F) = (n-1)/||W||_F is not a speculation about proxy quality: it is a closed-form consequence of the loss chosen in Algorithm 1. Thus MoPEQ's advertised Hessian-trace sensitivity is, as implemented, a ranking by inverse parameter magnitude. The paper does contain real supporting structure--per-expert granularity, layer-wise vs model-wise ablations, and a genuinely smaller 2/3/4-bit model footprint--but none of that validates the central attribution to sensitivity. The proposed concrete test would settle the question: if inverse-norm ranking correlates with measured per-expert quantization damage, the approach survives albeit with a corrected name; if not, the empirical tables are explained by budget or noise and the central claim fails. Because the reader already conditioned acceptance on exactly this validation, I do not move the verdict; I treat the requirement as confirmed and essential.","tokens_in":15336,"tokens_out":9843,"duration_ms":94520,"concrete_test":"Pick one model, e.g., DeepSeek-VL2-Tiny. For each expert independently, quantize only that expert to 4-bit while all other experts stay 16-bit, measure the change in a held-out VLMEvalKit score (or language perplexity on a calibration split), and compute the Spearman correlation between these per-expert drops and MoPEQ's Hessian-trace/bit ranking. Separately, verify analytically that Algorithm 1 returns (n-1)/||W||_F for each gate/up/down matrix. If the correlation with actual quantization impact is not significantly positive--or if the inverse-norm formula reproduces the assignment exactly--the sensitivity mechanism claimed by MoPEQ is not supported and the paper must be reframed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 (Section 3.3) computes the Hutchinson estimate of Tr(H) with L=||W||_F. For an n-parameter weight tensor W, the Hessian of the Frobenius norm is H = (I - ww^T/||W||^2)/||W||, so Tr(H) = (n-1)/||W||_F exactly. The Hutchinson average recovers this closed form; no data or task information enters. Hence Figure 3 and every 'Hessian-sensitivity' assignment in Tables 2-5 are monotone inverse-Frobenius-norm rankings of (gate+up+down) weights. The paper's central claim is that expert sensitivity, not activation frequency, should determine precision, but the quantity named 'sensitivity' is actually weight scale. The MME-perception win and other accuracy comparisons therefore cannot be attributed to the stated mechanism: they may reflect a norm heuristic, the smaller average bit width of the 2/3/4-bit budget, or run-to-run noise (no error bars are reported). This is more than 'proxy needs calibration': the analytic form shows the computed metric is a different quantity from task sensitivity unless an empirical correlation is demonstrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MoPEQ, a post-training mixed-precision quantization method for mixture-of-experts (MoE) layers in vision-language models. The method assigns 2-, 3-, or 4-bit precision to each expert based on a per-expert importance score, which is computed either from expert activation frequency, from a 'Hessian trace approximation' of the Frobenius-norm loss on the expert weights, or from a normalized product of the two. Similar experts are grouped by K-means clustering and bit widths are assigned to clusters by descending mean importance. The authors evaluate MoPEQ on four VLM-MoEs (DeepSeek-VL2 tiny/small/base and MolmoE-1B) across nine VLMEvalKit tasks, comparing against uniform-precision quantization and activation-frequency-based assignment. The main claims are that sensitivity-based assignment achieves competitive or better accuracy than uniform 4-bit and activation-frequency baselines while reducing model size, and that the approach is data-free because the sensitivity score uses only weights.","tokens_in":15615,"tokens_out":4405,"duration_ms":38403,"significance":"If the sensitivity proxy is valid, the paper would make a useful empirical contribution: it is, to my knowledge, the first mixed-precision quantization study targeting expert granularity in VLM-MoEs, and it includes a broad evaluation across four models and nine tasks. The paper also compares against the activation-frequency baseline and releases code, which are strengths. A notable positive is that the expert-importance assignment is computed from weights alone (plus optionally activation frequency) and is not fitted to the target accuracy, so the central comparison is not circular. However, the validity of the proposed 'Hessian trace' sensitivity metric is the load-bearing assumption of the paper, and as shown below it is not supported by the manuscript's derivation or experiments.","major_comments":[{"comment":"The quantity called the 'Hessian trace approximation' is analytically identical to the inverse Frobenius norm of the weight tensor, not a measure of task sensitivity. For L = ||W||_F, the Hessian is H = (I - ww^T/||W||^2)/||W||, so Tr(H) = (n-1)/||W||_F exactly; the Hutchinson estimator in Algorithm 1 merely recovers this deterministic value. Thus Figures 3, 6, and 9 and the 'Hessian sensitivity' assignments in Tables 2-5 are monotone inverse-Frobenius-norm rankings of the expert weights. The manuscript's central claim that this captures 'expert sensitivity' to quantization is unvalidated: no experiment shows a correlation between this quantity and actual per-expert quantization error or task-level accuracy drop. The authors should either replace this proxy with a calibration-based Hessian trace (e.g., on a reconstruction loss) or demonstrate empirically that the inverse-weight-norm ranking correlates with quantization sensitivity. Without that, the observed accuracy differences cannot be attributed to the stated mechanism.","section":"Section 3.3, Algorithm 1"},{"comment":"No error bars or multiple-seed results are reported for any of the VLMEvalKit tasks, several of which are known to be noisy. Many of the reported improvements are small (e.g., Table 3 shows DocVQA 87.334 vs. 87.652 for Hessian layer-wise vs. activation-frequency layer-wise, and Table 5 shows MMMU 37.0 vs. 36.0 for Hessian model-wise vs. activation-frequency model-wise). Without variance estimates, the claim that sensitivity-based assignment 'outperforms' activation frequency in '63 different scenarios' (Section 5.3) is not statistically supported. Please provide standard errors, multiple seeds, or at least significance tests for the main comparisons.","section":"Tables 2-5"},{"comment":"The manuscript repeatedly claims 'a model size reduction of approximately 1.5x' (also phrased as 'substantial improvements in memory footprint' and 'smaller model size'), but the tables show at most a 1.37x reduction compared to the uniform 4-bit baseline (Table 5: 14.353 GB to 10.485 GB), with other models in the 1.12-1.26x range. If the 1.5x figure is relative to the 8-bit uniform baseline, that baseline is not the one used in the accuracy comparisons and should be stated explicitly. The abstract, contributions, and conclusion should be corrected to match the reported numbers.","section":"Abstract, Section 1 Contribution (3), Section 6 Conclusion"}],"minor_comments":[{"comment":"The normalization formula for I_i has no epsilon guard; if max_j H_j equals min_j H_j, the denominator is zero. Please add a small constant or state the assumption that the ranges are non-degenerate.","section":"Section 3.4"},{"comment":"Reference [42] is listed as a survey on multimodal large language models but is used in Table 2 for the MME benchmark; the MME dataset citation should be [16] (Fu et al., 2023). Please correct the citation.","section":"Reference list"},{"comment":"Some cells in the DeepSeek-VL2-Small results show extreme degradations (e.g., activation-frequency layer-wise DocVQA 13.193 vs. uniform 4-bit 62.58, and MMMU 9.333 vs. 34.0). These are not discussed in the text; a sentence explaining whether these reflect a systematic failure mode of layer-wise assignment would help the reader interpret the comparison.","section":"Table 4"},{"comment":"The K-means clustering step requires a pre-specified number of clusters C equal to the number of bit widths, and the mapping from clusters to bit widths is by sorted cluster means; the sensitivity of the assignment to the random initialization of K-means is not reported. Please state the number of runs or the fixed seed used.","section":"Section 4.1"},{"comment":"The paper says the activation-frequency baseline is profiled on the MME dataset and then evaluated on MME among other tasks; this gives the activation-frequency baseline an in-distribution advantage and should be stated explicitly in the setup, since MoPEQ's Hessian-based assignment is data-free.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical study with a potentially useful benchmark contribution, but the core interpretation rests on an unvalidated sensitivity proxy that is analytically a weight-norm ranking. I recommend requiring a validation experiment (e.g., correlation with per-expert quantization error or a calibration-based Hessian) before acceptance. The size-reduction claim also needs correction. No issues with novelty disclosure; the related work appears adequately cited."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful empirical mapping of expert-level mixed-precision quantization for VLM-MoEs, and the first I know of for those model families. But the paper's central 'sensitivity' proxy has a closed form you can derive in three lines, and it is not what the authors think it is. The headline result should be read as a norm heuristic, not a Hessian-based sensitivity analysis.\n\nWhat's new: per-expert bit allocation applied to DeepSeek-VL2-tiny/small/base and MolmoE, with layer-wise vs model-wise assignment and a comparison against activation-frequency routing. They use AutoRound/SignRound consistently, report model sizes and nine VLMEvalKit tasks, and do not oversell the lack of hardware measurements. That is a solid benchmark study for a deployment-focused audience.\n\nThe soft spot is load-bearing. In Algorithm 1, the loss is L=||W||_F. The Hessian of the Frobenius norm is (I - ww^T/||W||^2)/||W||, so its trace is (n-1)/||W||_F exactly. The Hutchinson sampling in Algorithm 1 is therefore estimating a deterministic function of the weight norm. The 'importance' maps in Figure 3 are inverse weight-norm maps. Higher bits go to experts with smaller Frobenius norms. That might be a reasonable heuristic, but it is not sensitivity to quantization on the task, and the paper provides no calibration showing that norm ranks correlate with per-expert quantization loss. The claim in the abstract and conclusion that sensitivity beats activation frequency is therefore not established. The MME-perception wins and similar comparisons could be noise (no error bars), or could be the effect of the norm heuristic, or the smaller average bit width of the 2/3/4 budgets. This needs fixing.\n\nSmaller issues: the claimed '~1.5x' size reduction over uniform 4-bit does not appear in the tables; the best case is about 1.37x, most are 1.1-1.25x. That is still interesting, but the claim should be corrected. Also K-means has free parameters (C, initialization, Hutchinson sample count) with no sensitivity analysis. That is a minor omission.\n\nThe paper deserves a serious referee because the application area is real and the experimental setup is mostly sound. But I would not accept it as is. The authors should either re-run the assignment with a data-aware sensitivity measure (e.g., per-expert Hessian of the actual task loss, or per-expert quantization error on a few calibration batches) or re-frame the paper as a study of norm-based heuristics. The current title and abstract promise more than the math delivers.\n\nI would send it to peer review, but with a clear message: the core metric has to be re-examined before publication.","headline":"Useful first mapping of expert-level mixed-precision quantization for VLM-MoEs, but the 'Hessian sensitivity' metric is analytically just inverse weight norm, so the central interpretive claim does not hold as stated.","tokens_in":16152,"tokens_out":3507,"would_cite":false,"duration_ms":30799,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Per-expert sensitivity, not activation frequency, is the right signal for quantizing mixture-of-experts vision-language models.","keywords":["Mixture-of-Experts","mixed-precision quantization","post-training quantization","vision-language models","Hessian trace sensitivity","expert clustering","model compression","bit-width allocation"],"falsifier":"Measure, for every expert in one of the tested models, the actual output perturbation after rounding its weights to 2 and 3 bits on a held-out calibration set; compute Spearman correlation between that measured error and the paper's Hessian-trace score. If the correlation is near zero or negative, the bit assignments are not tracking true sensitivity, and the reported accuracy advantages would not be explained by the claimed mechanism.","tokens_in":15123,"feed_emoji":"🎯","tokens_out":6538,"duration_ms":57045,"temperature":0.7,"pith_summary":"The paper tries to establish that mixture-of-experts vision-language models can be compressed more effectively by giving each expert its own bit width, chosen by a data-free sensitivity score, than by quantizing all experts uniformly or by allocating bits according to how often experts are used. The proposed algorithm, MoPEQ, estimates each expert's sensitivity with a Hessian-trace proxy on the weights and clusters experts with similar importance into 2-, 3-, and 4-bit groups. On four open vision-language MoE models, the method keeps accuracy within a few points of uniform 4-bit baselines while cutting model size by roughly 1.5x, and on one perception benchmark it improves the score from 1300 to 1338 at a smaller size. The broader claim is that expert specialization, not usage frequency, should drive which experts deserve high precision.","feed_headline":"Sensitivity, not usage, should set each expert's bit width","feed_subtitle":"A Hessian-trace score picks 2/3/4-bit groups and cuts MoE VLM size ~1.5x with accuracy intact.","key_machinery":"The load-bearing object is the per-expert Hessian trace $\\mathrm{Tr}(\\mathbf{H})$ of a proxy loss $L = \\|\\mathbf{W}\\|_F$ for each expert's feed-forward weight tensor, estimated by a stochastic trace estimator that forms Hessian-vector products $\\nabla_W(g_1^\\top v)$ with random sign vectors $v$ and averages $v \\odot \\mathrm{HVP}$. This score is meant to quantify how much an expert's output would move under quantization, without requiring input data. The assignment machinery then K-means clusters experts by the score, either within each layer or across the whole model, sorts clusters by mean importance, and maps the top cluster to 4 bits, the middle to 3, and the bottom to 2.","core_discovery":"The central claim is that per-expert precision assignment for MoE layers improves the accuracy-per-byte trade-off of post-training quantized vision-language models, and that Hessian-trace sensitivity is a better importance signal than activation frequency. For each expert, the method computes a Hessian trace estimate of the Frobenius-norm loss $\\|\\mathbf{W}\\|_F$ of its feed-forward weight matrices, without any calibration data, then K-means clusters experts by those scores and assigns 4, 3, or 2 bits to high-, mid-, and low-importance clusters. Evidence includes that this sensitivity-based scheme beats the activation-frequency baseline on the largest tested model in most tasks, that model-wise assignment outperforms layer-wise assignment in 63 of 105 comparison scenarios, and that the largest model reaches a 10.485 GB size with competitive scores versus 14.353 GB for uniform 4-bit quantization. If this is right, expert importance is discoverable from weights alone, and compressing MoE models need not wait for calibration data.","pith_inferences":["Because $\\mathrm{Tr}(\\mathbf{H})$ for $L=\\|\\mathbf{W}\\|_F$ simplifies, for an $n$-parameter weight matrix, to $(n-1)/\\|\\mathbf{W}\\|_F$, the \"sensitivity\" ranking may effectively order experts by weight norm; a parsimonious test is to compare MoPEQ against magnitude-based bit assignment with the same cluster fractions.","The paper does not measure actual per-expert output perturbation before assigning bits; a direct check is to compute per-expert quantization error at 2 and 3 bits and see whether it matches the Hessian ranking.","If mixed-precision kernels become available, the scheme's habit of assigning low bits to frequently activated experts could cut GPU-CPU transfer in offloaded MoE serving, which the paper notes but does not measure.","The hybrid frequency-times-sensitivity score may be most useful for unevenly routed models, where it trims the model most while keeping accuracy, suggesting the two signals are complementary rather than substitutes."],"forward_implications":["If per-expert sensitivity is the right signal, uniform-precision quantization leaves memory on the table for every MoE vision-language model, and data-free bit assignment can deliver most of the savings.","On models trained with load-balancing losses, activation frequency is nearly flat, so frequency-based methods cannot differentiate experts; sensitivity-based assignment should be the default there.","Model-wide clustering of experts tends to outperform layer-wise clustering because it can concentrate high bits in the layers that matter most to overall output.","At fixed bit budgets, the method gives a family of Pareto points: smaller models with mild accuracy loss or larger models with near-baseline accuracy, selectable by the bit set $\\{2,3,4\\}$.","The approach is orthogonal to the underlying quantizer, so it can ride on any post-training rounding routine that accepts per-module bit widths."],"supporting_citations":[{"why":"Provides the activation-frequency-based bit-allocation baseline that MoPEQ claims to beat.","marker":"[24]"},{"why":"Provides the layer-wise mixed-precision assignment scheme for MoE LLMs that MoPEQ extends to model-wide clustering.","marker":"[18]"},{"why":"Supplies the stochastic estimator of the Hessian trace that underlies the per-expert sensitivity score.","marker":"[19]"},{"why":"Establishes Hessian trace as a layer-importance measure for mixed-precision quantization, the conceptual basis for per-expert sensitivity.","marker":"[12]"},{"why":"Refines trace-weighted Hessian-aware quantization, motivating the use of trace rather than the full Hessian.","marker":"[13]"},{"why":"Supplies the signed-gradient rounding routine used to quantize weights at the assigned precisions.","marker":"[5]"},{"why":"Provides the quantization toolkit that executes the uniform and mixed-precision quantizations in the experiments.","marker":"[20]"},{"why":"Supplies the multimodal evaluation suite whose accuracy numbers appear in the comparison tables.","marker":"[14]"},{"why":"Defines the vision-language MoE model family used as the primary testbed for the method.","marker":"[39]"}],"fun_headline_variants":["Hessian trace, not frequency, picks expert bit width","Per-expert Hessian scores drive 2/3/4-bit MoE quantization","Sensitivity-based expert precision shrinks MoE VLMs ~1.5x","Quantize each expert by Hessian trace, not activation count","Hessian-aware precision allocation boosts MoE accuracy per byte"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the Hessian trace of the Frobenius norm of each expert's weight matrix ranks experts by how much their quantized outputs will damage the model's accuracy, although this proxy is never calibrated against measured quantization error.","fun_headline_variants_meta":{"raw":{"variants":["Hessian trace, not frequency, picks expert bit width","Per-expert Hessian scores drive 2/3/4-bit MoE quantization","Sensitivity-based expert precision shrinks MoE VLMs ~1.5x","Quantize each expert by Hessian trace, not activation count","Hessian-aware precision allocation boosts MoE accuracy per byte"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000509,"raw_usage":{"total_tokens":2500,"prompt_tokens":985,"completion_tokens":1515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":1420}},"tokens_in":601,"tokens_out":1515,"duration_ms":10706,"temperature":1.0,"reasoning_tokens":1420,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:37:17.277605+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, for every expert in one of the tested models, the actual output perturbation after rounding its weights to 2 and 3 bits on a held-out calibration set; compute Spearman correlation between that measured error and the paper's Hessian-trace score. If the correlation is near zero or negative, the bit assignments are not tracking true sensitivity, and the reported accuracy advantages would not be explained by the claimed mechanism.","supporting_citations":[{"cited_title":"A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines","cited_arxiv_id":null,"evidence_quote":"Supplies the stochastic estimator of the Hessian trace that underlies the per-expert sensitivity score."},{"cited_title":"Hawq: Hessian aware quantization of neural networks with mixed-precision","cited_arxiv_id":null,"evidence_quote":"Establishes Hessian trace as a layer-importance measure for mixed-precision quantization, the conceptual basis for per-expert sensitivity."},{"cited_title":"Intel autoround","cited_arxiv_id":null,"evidence_quote":"Provides the quantization toolkit that executes the uniform and mixed-precision quantizations in the experiments."},{"cited_title":"Vlmevalkit: An open-source toolkit for evaluating large multi-modality models","cited_arxiv_id":null,"evidence_quote":"Supplies the multimodal evaluation suite whose accuracy numbers appear in the comparison tables."}],"review_version":2}