{"id":"faa4654e-caf2-4547-82a6-6d4054022fe1","arxiv_id":"2505.04861","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A mixed-precision post-training quantization method for SAM that allocates bit-widths via an integer quadratic program guided by KL-divergence importance scores and a cross-layer synergy heuristic.","lead":"Mix-QSAM assigns different bit-widths to different layers of the Segment Anything Model after training, guided by per-layer importance and a layer-to-layer smoothness score, to shrink the model for edge deployment. The paper reports large accuracy gains for the small SAM variant, but the gains shrink or reverse on larger variants, and the score derivations contain gaps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-layer synergy in Eq. (18) is not the causal-MI interaction of Eq. (17); with the printed typo it is constant, so the paper's central mechanism is unsupported as written.","rationale":"The reader's weakest assumption concerns the undefined sample space in the KL-based layer importance score, Eq. (14), which is related but not identical to my concern: the cross-layer synergy metric in Eq. (18) is not derived from the stated causal-MI definition and, as described, degenerates to a constant. The reader's rationale already flags this; my check sharpens it into a concrete implementation-level test. This is load-bearing because cross-layer synergy is one of the two named contributions and the Table 3 ablation specifically credits it with large gains. If the implemented metric is constant or merely an inverse-difference heuristic, the paper's explanation of why the method works is not supported, even though the empirical results might still hold. The concern is addressable with code inspection or a corrected derivation, so a conditional rather than accept or reject verdict remains appropriate. I do not see an honest non-finding here: the printed math is internally inconsistent enough that the central mechanism is unverified, but the empirical claims could still be salvaged by releasing the implementation and clarifying the metric.","tokens_in":16776,"tokens_out":13599,"duration_ms":139259,"concrete_test":"Obtain the released code or request the implementation of Eq. (18) from the authors. Instrument S_t_lm: if the denominator is computed as 1/(|I(Lm)-I(Lm)|+eps), the printed metric is constant. If it uses Ll and Lm, rerun the Table 3 ablation twice, once with S_t_lm set to a constant and once with the printed inverse-difference form, and compare the resulting W6/A6 and W4/A4 mAP values. If the constant version reproduces Table 3, the reported synergy gains are not attributable to the claimed cross-layer synergy metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's novelty rests on cross-layer synergy. Eq. (17) defines synergy as interaction information, I(Ll,Lm;Y) - I(Ll;Y) - I(Lm;Y). Eq. (18) then replaces it with 1/(|I(Ll;Yt)-I(Lm;Yt)| + epsilon), and the explanatory sentence even writes the denominator as |I(Lm;Yt)-I(Lm;Yt)|, making the score a constant for every neighboring pair. No derivation connects Eq. (17) to Eq. (18); the proposed approximation is simply an inverse difference of scalar importance scores, not a joint causal quantity. If the typo is taken literally, the BTR penalty in Eq. (21) degenerates to a generic smoothness regularizer, so the reported ablation gains in Table 3 cannot be attributed to the claimed cross-layer synergy metric. Even if the intended Ll,Lm form is used, the paper provides no argument that this heuristic approximates the defined interaction information. Without released code or a derivation, the central 'importance + synergy' mechanism that drives the IQP allocation is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Mix-QSAM, a post-training mixed-precision quantization framework for the Segment Anything Model. It introduces a layer-wise importance score computed as a KL divergence between the original output distribution and the output distribution after zeroing a layer's activations, and a cross-layer synergy score intended to capture interaction information between adjacent layers. These scores are used in an integer quadratic programming formulation that allocates bit-widths under model-size and bit-operation constraints. Experiments on COCO instance segmentation and object detection with four detectors compare Mix-QSAM against static and learning-based PTQ baselines at W6/A6 and W4/A4 settings, and an ablation study isolates the effects of the importance score and the bit-transition regularizer.","tokens_in":17059,"tokens_out":8684,"duration_ms":87775,"significance":"If the proposed metrics are valid and the reported results are reproducible, this is a practically significant result: a static, reconstruction-free mixed-precision PTQ method that can match or exceed learning-based PTQ for SAM at a fraction of the calibration cost. The evaluation is not circular in an obvious way, because the importance and synergy scores are computed from 64 calibration images while mAP is measured on held-out COCO data, and the comparison set is reasonably broad. However, the central formal gaps in the definitions of both metrics and a constraint-feasibility inconsistency in the W4/A4 experiments mean that the significance is conditional on repair.","major_comments":[{"comment":"Equation (18) is presented as an approximation to the interaction information in Eq. (17), but no derivation is provided, and the printed denominator contains the typo |I(Lm;Yt)-I(Lm;Yt)|, which makes S_t_l_m equal to 1/epsilon for every pair. Under that literal reading, the BTR term in Eq. (21) is a constant-weight smoothness penalty, so the improvement of \"Synergistic BTR\" over \"Independent BTR\" in Table 3 (e.g., 29.3 to 35.3 for SAM-B W6/A6) cannot be attributed to cross-layer synergy. Even after correcting the denominator to |I(Ll;Yt)-I(Lm;Yt)|, the inverse difference of marginal importance scores is not a special case of Eq. (17) without additional assumptions; please provide the derivation or explicitly reclassify S as a heuristic and validate it separately.","section":"§3.3, Eqs. (17)–(20) and Table 3"},{"comment":"The importance score rests on a formal identification that is not established. Equation (12) is the conditional mutual information I(Y;Li|L¬i), while Eq. (13) states an approximation by a KL divergence. These quantities coincide only after averaging the KL over the joint distribution of Li and L¬i; the paper does not write this expectation, and when the perturbation is implemented by zeroing activations, P(Y|L¬i) is an interventional distribution rather than the conditional distribution of the original model. The sample space of Y is also never defined: SAM's output is a mask or per-pixel probability map, so the summation over y in Eq. (14) is non-operational as written. Please define Y, the perturbed model, and the averaging procedure, and state precisely which quantity Omega_i is intended to estimate.","section":"§3.2, Eqs. (12)–(14)"},{"comment":"The statement in §4 that the mixed-precision models \"match their fixed-precision counterparts in model size and bit-operations\" is inconsistent with the candidate set shown in Figure 1. For W4/A4, if B = {4,5,6,7,8}, the only assignment satisfying a model-size constraint equal to the all-4-bit model size is the uniform all-4 assignment, because no bit-width below 4 is available to compensate for layers assigned above 4. In that case Mix-QSAM should coincide with the fixed-bit PTQ4SAM-S baseline, contradicting the large W4/A4 gains in Table 3 (e.g., 17.4 to 26.9 for SAM-B with YOLOX). Please report the actual candidate sets and constraint values used for each W/A setting, and clarify whether weights and activations are quantized with separate bit-widths, as the text in §3.4 suggests.","section":"§3.4, Eqs. (21)–(24); §4; Figure 1"}],"minor_comments":[{"comment":"The ablation rows are labeled \"Mix-QViT(ours)*\" instead of \"Mix-QSAM(ours)*\"; this appears to be leftover text from a related method and should be corrected.","section":"§4.4, Table 3"},{"comment":"The abstract states \"up to 20% higher average precision\" while the Introduction states \"up to 17% higher\"; please specify whether these are absolute mAP point differences or relative improvements and harmonize the numbers.","section":"Abstract and §1"},{"comment":"The sentence defining the denominator writes |I(Lm;Yt)-I(Lm;Yt)|; this should be |I(Ll;Yt)-I(Lm;Yt)|.","section":"§3.3, text after Eq. (18)"},{"comment":"The caption reads \"SAM- under W6/A6\"; the model name is incomplete, and it should also state whether the candidate set {4,5,6,7,8} is the one used for W4/A4 experiments.","section":"Figure 1 caption"},{"comment":"Equation (21) uses a single bit-width variable b_j for each layer, while the text in §3.4 says the method assigns different bit-widths (bw_l, ba_l) to weights and activations; please clarify the relationship between the two notations.","section":"§3.4, Eq. (21)"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is extensive and the reported gains, if reproducible, would be interesting, and I found no evidence that the calibration metrics were tuned to the test set. My main concern is formal: the mechanism driving the bit allocation is either mis-specified or unproven, and the W4/A4 constraint story is internally inconsistent as written. These issues are substantial but appear fixable with a careful rewrite, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper has a real empirical contribution: it's the first mixed-precision post-training quantization specifically for SAM, and the experiments are reasonably broad (three SAM variants, four detectors, two tasks). The ablation in Table 3 gives some evidence that the bit-transition penalty helps, and the static method avoids reconstruction time. That's worth something.\n\nThe soft spots are significant, though. The 'cross-layer synergy' metric is the centerpiece, but Eq. (18) is not an approximation of the interaction information in Eq. (17) — it's just an inverse difference of individual importance scores. No derivation is offered, and the prose even contains a typo that writes |I(Lm;Yt)-I(Lm;Yt)| in the denominator, which would make the score a constant. Even with the intended Ll,Lm, this is a smoothness regularizer dressed up as an information-theoretic quantity. The KL-based importance score also has an undefined output distribution; summing over y for dense masks is handwavy. And the abstract's claim of 'consistently outperforming' is not backed by Table 1, where several 4-bit results on SAM-L and SAM-H fall below PTQ4SAM-L. No code and no explicit constraint values make reproduction harder.\n\nThat said, these are fixable. The authors can reframe the synergy term as a heuristic, provide the derivation or drop the information-theoretic language, clarify the KL computation, and soften the claims. The empirical finding that mixed-precision helps SAM quantization is plausible and useful.\n\nWho's this for? People working on efficient deployment of segmentation models. It deserves a serious referee, but with major revision.","headline":"Useful engineering results for SAM quantization, but the central synergy metric is mislabeled and the abstract overstates consistency.","tokens_in":17555,"tokens_out":4567,"would_cite":true,"duration_ms":44080,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A static mixed-precision quantizer can match learning-based PTQ on SAM by allocating bits via KL importance and cross-layer synergy.","keywords":["mixed-precision quantization","post-training quantization","Segment Anything Model","layer importance","KL divergence","causal mutual information","cross-layer synergy","integer quadratic programming"],"falsifier":"Quantize each encoder layer one at a time to 4-bit while keeping every other layer at full precision, and correlate the resulting mAP drop with that layer's importance score $\\Omega_l$; if low-importance layers lose as much mAP as high-importance layers, or if reshuffling the bit assignment within the same total model size and bit-operation budget changes mAP by less than a point, then the importance and synergy metrics are not carrying the reported gains.","tokens_in":16553,"feed_emoji":"🖼️","tokens_out":14874,"duration_ms":127388,"temperature":0.7,"pith_summary":"Mix-QSAM tests a straightforward hypothesis: for a prompt-based segmentation model like SAM, low-bit accuracy can be recovered without retraining or reconstruction—you only need to spend bits where they matter. The paper builds a static post-training quantizer that assigns each layer its own weight and activation bit-width by solving a small integer quadratic program. The optimization is driven by two scores: a per-layer importance score derived by zeroing that layer's activations and measuring the Kullback-Leibler divergence of the output, and a cross-layer synergy score that keeps adjacent interdependent layers at similar precision. On COCO instance segmentation and object detection, the method reports 17–20% higher average precision than fixed-bit post-training quantization at 4- and 6-bit settings, matching or beating learning-based reconstruction methods without the reconstruction cost.","feed_headline":"Bit-allocation trick recovers most of 4-bit SAM's accuracy","feed_subtitle":"KL-guided bit allocation matches learning-based quantizers without retraining.","key_machinery":"The load-bearing object is the pair of scores that feed the optimizer. Layer importance uses causal mutual information: zero the activations of layer $l$, measure $D_{\\mathrm{KL}}(P_Y(y\\mid L_l,L_{\\neg l})\\,\\|\\,P_Y(y\\mid L_{\\neg l}))$, normalize by the total over layers, and average over 64 calibration images to get $\\Omega_l$. Cross-layer synergy approximates interdependence by $\\hat{S}_{l,m}=\\log\\bigl(1+\\bar{S}_{l,m}\\bigr)$, where $\\bar{S}_{l,m}$ is the mean over images of $1/(|I(L_l;Y)-I(L_m;Y)|+\\epsilon)$. These two metrics enter a one-hot integer quadratic program: maximize $\\sum_l \\Omega_l\\sum_j \\alpha_{l,j} b_j - \\lambda \\sum_l \\hat{S}_{l,m}\\,|\\sum_j \\alpha_{l,j} b_j - \\sum_j \\alpha_{m,j} b_j|$ subject to exactly one bit-width per layer, a model-size budget, and a bit-operation budget. The optimization is what converts causal-ish scores into a concrete bit assignment for every quantized layer.","core_discovery":"The paper's central claim is that for SAM, per-layer bit-widths can be chosen before quantization, without retraining or reconstruction, by solving a small constrained integer quadratic program. The program's objective is a weighted sum of two causal-information metrics: a layer-importance score $\\Omega_l$, computed by zeroing that layer's activations and measuring the KL divergence between the perturbed and unperturbed output distribution, averaged over 64 images; and a cross-layer synergy score $\\hat{S}_{l,m}$, the log-scaled reciprocal of the difference between adjacent layers' importance scores. Maximizing $\\sum_l \\Omega_l\\, b_l$ while penalizing $\\hat{S}_{l,m}\\,|b_l-b_m|$ assigns high precision to critical layers and keeps interdependent layers at similar precision. On COCO instance segmentation and object detection, the paper reports that this static allocation raises average precision by up to 20% in the abstract (17% in the contributions list) over fixed-bit PTQ at 6-bit and 4-bit settings, and matches or beats reconstruction-based methods while requiring no reconstruction. The claim is specifically about placement of bits, not about introducing new quantization operators.","pith_inferences":["Because the two scores are computed once on 64 images and the bit assignment is solved offline, the same allocation would likely transfer to other prompt distributions or downstream tasks without recalibration; the paper only demonstrates this on COCO instance segmentation and object detection.","The synergy penalty smooths a one-dimensional chain of bit widths. A natural test the paper does not run is to apply the same causal-information construction at finer granularity, such as attention heads or individual blocks, or to allow non-adjacent dependencies.","The reported 4-bit results suggest that bit placement, not rounding or reconstruction, is the dominant source of quantization error for SAM. If this interpretation is right, future PTQ for prompt-based models should focus on allocation metrics rather than reconstruction iterations, but the paper does not isolate that claim."],"forward_implications":["At 6-bit on the larger SAM variants, the mixed-precision static model ties full-precision mAP on several detector/model combinations, so a quantized SAM with no visible segmentation loss is within reach.","At 4-bit, the static method improves on the best reconstruction baseline by a wide margin on at least one SAM-B detector pairing (30.6 mAP versus 14.4 mAP), showing that reconstruction is not required to recover low-bit SAM.","The synergy penalty is worth 5.5–6.0 points of mAP in the SAM-B ablation, so abrupt bit changes between adjacent layers are a measurable source of quantization error.","Calibration stays practical: about 15 minutes for SAM-B, and no 200–400 minute reconstruction phase is needed."],"supporting_citations":[{"why":"Supplies the pretrained Segment Anything Model whose encoder layers are quantized; the method's importance and synergy scores are computed on this model.","marker":"[18]"},{"why":"Provides the fixed-bit quantization recipe, including adaptive granularity for bimodal post-key activations, that Mix-QSAM adopts as its base quantizer and as the main static baseline.","marker":"[34]"},{"why":"The adaptive-rounding fixed-bit baseline that Mix-QSAM aims to beat without reconstruction; its reported 4-bit numbers anchor the comparison.","marker":"[37]"},{"why":"The blockwise-reconstruction PTQ baseline; establishes the learning-based bar that Mix-QSAM matches or exceeds while skipping reconstruction.","marker":"[22]"},{"why":"The dropout-based reconstruction baseline; with the blockwise and adaptive-rounding methods it defines the set of learning-based PTQs Mix-QSAM is compared against.","marker":"[47]"},{"why":"The solver used to run the integer quadratic program that turns importance and synergy scores into a bit-width assignment.","marker":"[7]"}],"fun_headline_variants":["Mixed-precision bit allocation boosts SAM's accuracy by 20%","Find the best bit-widths per layer with a few images and a formula","SAM quantization gets smarter: allocate bits per layer, no retraining","IQP-based bit allocation lifts SAM quantization accuracy by 20%","KL and causal info guide mixed-precision SAM quantization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that zeroing a layer's activations and measuring the KL divergence of the output distribution faithfully estimates that layer's causal contribution; because the output distribution in Eq. (14) is never explicitly defined, the importance scores that drive the entire bit allocation rest on an unspecified probability space.","fun_headline_variants_meta":{"raw":{"variants":["Mixed-precision bit allocation boosts SAM's accuracy by 20%","Find the best bit-widths per layer with a few images and a formula","SAM quantization gets smarter: allocate bits per layer, no retraining","IQP-based bit allocation lifts SAM quantization accuracy by 20%","KL and causal info guide mixed-precision SAM quantization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000414,"raw_usage":{"total_tokens":2175,"prompt_tokens":1014,"completion_tokens":1161,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":630,"completion_tokens_details":{"reasoning_tokens":1071}},"tokens_in":630,"tokens_out":1161,"duration_ms":9387,"temperature":1.0,"reasoning_tokens":1071,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:20:00.499809+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Quantize each encoder layer one at a time to 4-bit while keeping every other layer at full precision, and correlate the resulting mAP drop with that layer's importance score $\\Omega_l$; if low-importance layers lose as much mAP as high-importance layers, or if reshuffling the bit assignment within the same total model size and bit-operation budget changes mAP by less than a point, then the importance and synergy metrics are not carrying the reported gains.","supporting_citations":[{"cited_title":"Segment anything","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained Segment Anything Model whose encoder layers are quantized; the method's importance and synergy scores are computed on this model."},{"cited_title":"PTQ4SAM: Post-training quan- tization for segment anything","cited_arxiv_id":null,"evidence_quote":"Provides the fixed-bit quantization recipe, including adaptive granularity for bimodal post-key activations, that Mix-QSAM adopts as its base quantizer and as the main static baseline."},{"cited_title":"Up or down? adaptive rounding for post-training quantiza- tion","cited_arxiv_id":null,"evidence_quote":"The adaptive-rounding fixed-bit baseline that Mix-QSAM aims to beat without reconstruction; its reported 4-bit numbers anchor the comparison."},{"cited_title":"CVXPY: A python-embedded modeling language for convex op- timization","cited_arxiv_id":null,"evidence_quote":"The solver used to run the integer quadratic program that turns importance and synergy scores into a bit-width assignment."}],"review_version":1}