{"id":"819a1292-d801-4da1-81ba-780c59f727bb","arxiv_id":"2507.19131","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A mixed-precision activation quantization framework that processes unimportant windows of Swin transformers at lower bit width, improving the accuracy-efficiency trade-off versus uniform quantization and activation pruning.","lead":"MixA-Q assigns lower-bit quantization to the less important windows inside Swin vision transformers, instead of skipping those windows entirely. It reports 1.2 to 1.5x bit-operation savings on COCO object detection with little or no accuracy loss, plus better robustness than activation pruning on out-of-distribution images.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'lossless speedup' claim is vulnerable to selection on the evaluation split: compression ratios are tuned against COCO val mAP and then reported on that same val set, with no held-out check.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the compression ratios are optimized against COCO val mAP and then the same COCO val mAP is used to claim lossless speedups. I agree this is the single weakest link because the headline claims are explicitly about 'without accuracy loss' and 'lossless' speedups; if the accuracy comparison is biased by selection, the central quantitative claims are not supported. I considered the BOPs-versus-latency gap, since speedups are never measured as wall-clock time, but that concern is secondary: the paper carefully calls the speedup 'computational' and defines it in BOPs, so the accuracy-equivalence claim is more central to what is being asserted. The circular evaluation is also directly fixable, so the verdict should remain CONDITIONAL rather than REJECT: rerunning the search on a training/validation split and reporting held-out mAP would settle whether the reported points are genuine. Thus no change to the reader's verdict is needed.","tokens_in":13675,"tokens_out":6533,"duration_ms":66767,"concrete_test":"Hold out a random half of COCO val (or use COCO test-dev) before any search. Run the NSGA-II search of Sec 3.3 using only the search half, take the selected compression-ratio configurations at the claimed 1.24x/1.35x operating points, and evaluate mAP on the held-out half. If the held-out mAP falls below the W4A4/W4A8 baseline by more than the reported 'lossless' margin, the lossless claim is an artifact of selection on the evaluation metric.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central accuracy-equivalence claims rest on an evaluation protocol that conflates model selection with evaluation. In Sec 3.3, the evolutionary search (NSGA-II) explicitly optimizes compression ratios using 'mAP is the mean average precision on the COCO val dataset.' The reported mAP numbers in Sec 4.1 (QAT) and Sec 4.3 (PTQ) are computed on the same COCO val split. The search space is large: six compression ratios discretized to {0%, 10%, ..., 80%}, i.e., about 9^6 = 531,441 configurations, from which NSGA-II selects Pareto-optimal points. Selecting configurations that maximize val mAP and then reporting those same val mAP values as 'lossless' introduces selection bias: with a noisy metric like COCO val mAP, the best of hundreds of evaluated candidates can easily appear 0.3-1.0 mAP better than its true held-out performance. That is the same magnitude as the reported 'lossless' margins (e.g., Table 1: 43.2 vs. 43.1 for the 1.24x point, and 42.3 vs. 43.1 for the 1.35x point). No held-out split, no repeated search, and no variance estimates are reported. The PTQ 'training-free 1.35x speedup without accuracy loss' claim is especially exposed because the ratios are searched directly against the reported metric without any fine-tuning to regularize the objective. This does not invalidate the method, but it means the headline accuracy-equivalence claims are not established by the current experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MixA-Q, a mixed-precision activation quantization framework for window-based vision transformers (Swin). For a given uniform-bit quantization configuration, MixA-Q splits each Swin block into high- and low-precision branches, assigns a lower activation bit width to windows deemed less important by an L2-norm importance score, and scatters the results back to the feature map. Compression ratios are chosen by an NSGA-II evolutionary search over a discrete space (six ratios for Swin-Tiny), optimizing bi-objectively for bit-operation savings and COCO val mAP. The method is integrated with QAT (OFQ) and PTQ (RepQ), and optionally combined with activation pruning. Experiments on COCO object detection and panoptic segmentation report a lossless 1.25x speedup with QAT, a training-free 1.35x speedup with PTQ, up to 1.82x speedups with pruning at moderate mAP drop, and an improvement of 0.7 mAP for the W4A4 quantized model attributed to sparsity-aware quantization adaptation (SAQA). The paper also evaluates robustness to COCO-O and to adversarial window selection, where MixA-Q degrades less than SparseViT.","tokens_in":13976,"tokens_out":6036,"duration_ms":55906,"significance":"If the results hold, the paper makes a useful conceptual contribution by transposing activation-sparsity exploitation from pruning to mixed-precision quantization, and the Two-Branch Swin Block is a clean, integration-friendly mechanism. The inclusion of PTQ without training is a genuinely attractive property, and the OOD-robustness comparison with SparseViT is a thoughtful and falsifiable check. However, the headline lossless-accuracy and speedup claims are currently supported only by (i) mAP numbers computed on the same COCO val split that the evolutionary search optimizes, and (ii) bit-operation counts rather than wall-clock latency. These issues are load-bearing for the stated contributions and need to be addressed before the claims can be accepted as established.","major_comments":[{"comment":"The evolutionary search in Sec. 3.3 explicitly optimizes compression ratios using the objective \"mAP is the mean average precision on the COCO val dataset,\" while the mAP numbers reported in Tables 1, 2, and 4 and in Fig. 4 are computed on the very same COCO val split. The search space of six discretized ratios (about 9^6 ≈ 531,000 configurations) makes it likely that the selected Pareto-front points are optimistically biased relative to true held-out performance. The reported \"lossless\" margins are small (e.g., Table 1: 43.2 vs. 43.1 for the 1.24x point, and 42.3 vs. 43.1 for the 1.35x point), so a selection bias of a few tenths of mAP is of the same magnitude as the claimed accuracy equivalence. I ask the authors to report results on a held-out split that was not used during the search, or to provide repeated-search variance estimates; at minimum, the paper should explicitly state that all reported mAP values are the search objective itself and therefore not independent measurements.","section":"Sec. 3.3 and Sec. 4.1"},{"comment":"The abstract and conclusion use the term \"speedup\" (1.25x, 1.35x, 1.53x), but Sec. 4 states that all computation savings are measured in bit operations (BOPs) of the backbone. No wall-clock latency, throughput, or energy measurements are reported. Mixed-precision branches with different bit widths, the gather/scatter operations around each window attention, duplicated layer normalizations, and non-uniform window counts across blocks will introduce overheads that BOPs does not capture. The claims should be rephrased as \"theoretical bit-operation savings\" unless actual runtime measurements are provided, or the limitation should be stated explicitly in the abstract and conclusion.","section":"Sec. 4 (opening) and Abstract"},{"comment":"The abstract claims a \"training-free 1.35x computational speedup without accuracy loss in PTQ configuration,\" but the PTQ results are only presented as a Pareto plot in Fig. 7b, without tabulated mAP values and without an explicit indication of which configuration corresponds to the 1.35x point or its exact mAP. Please report the compression-ratio configuration, the equivalent activation bits, the BOPs, and the mAP for the claimed 1.35x point, together with the baseline W4A8 mAP, so that the lossless claim can be verified numerically.","section":"Sec. 4.3 and Abstract"}],"minor_comments":[{"comment":"\"SpraseViT\" should be \"SparseViT\".","section":"Sec. 1, contribution 3"},{"comment":"The concluding paragraph contains a typo: \"bit widts\" should be \"bit widths\".","section":"Sec. 5"},{"comment":"The output line \"such that Pri = S\" should read \"such that Σ r_i = S\".","section":"Appendix 6.1, Algorithm 1"},{"comment":"\"the weights and bias of of the QKV layer\" contains a duplicated \"of\".","section":"Sec. 4.3"},{"comment":"The table caption says \"Act Bit represents the activations' equivalent bit width,\" but the meaning of the parenthetical speedup column header \"1.33x\" etc. should be clarified to indicate that these are relative to the W4A4 OFQ model, not to the full-precision model.","section":"Sec. 4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the journal and the core idea is interesting, but the val-set selection issue is serious and affects the most important claims. I would like to see a revision that either presents held-out evaluation results or explicitly reframes the accuracy-equivalence claims as being with respect to the search objective. The use of BOPs as a proxy for speedup should also be clearly labeled as theoretical, since otherwise the practical-efficiency claims will be over-read."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"MixA-Q is a sensible, well-motivated extension of SparseViT-style window sparsity to mixed-precision activation quantization. The two-branch Swin block with shared weights is clean, the training-free PTQ story is attractive, and the OOD robustness analysis is a thoughtful addition. The central mechanism—keep less important windows but run them at lower precision—is plausible, and the reported Pareto comparisons against SparseViT are consistent.\n\nThe main issue, and it is a real one, is that the 'lossless' speedup numbers are not independent. Compression ratios are searched by NSGA-II directly against COCO val mAP, and the same val mAP is then reported in Tables 1 and 2. With six ratios discretized to nine values, that is roughly a 531k-configuration search space; selecting Pareto-optimal points against the evaluation metric and then reporting that metric as 'no loss' is selection on the evaluation split. The reported margins are small (43.2 vs 43.1, 42.3 vs 43.1), right in the range that could be selection noise. The PTQ claim, where ratios are searched without any fine-tuning, is the most exposed. This does not invalidate the method, but it means the headline accuracy-equivalence claims are not established by these experiments.\n\nOther soft spots are minor. Speedup is in bit operations, not wall-clock latency; that is a standard proxy, but practical deployment needs hardware verification. The paper cites PMQ and Granular-DQ but does not compare against them, and no code or training hyperparameters are provided.\n\nThe paper is worth engaging with. The two-branch shared-weight block is a nice architectural contribution that could be reused, and the dynamic activation distillation idea is worth understanding. For a serious referee, I would ask for a held-out evaluation: fix the search budget, then evaluate on a different split or run repeated searches and report variance. That would settle the lossless claim. I would send it out.","headline":"MixA-Q is a plausible, well-motivated extension of activation sparsity to mixed-precision quantization, but its 'lossless' speedup claim is weakened by tuning compression ratios on the same COCO val split used for evaluation.","tokens_in":14583,"tokens_out":1981,"would_cite":true,"duration_ms":21061,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MixA-Q shows that assigning lower bit widths to less important windows—instead of pruning them—lets window-based vision transformers run at 1.35x lower computational cost with no measured accuracy loss.","keywords":["mixed-precision quantization","activation sparsity","Swin Transformer","vision transformers","quantization-aware training","post-training quantization","object detection","out-of-distribution robustness"],"falsifier":"Hold out COCO test-dev (or any split unused by the evolutionary search and adaptation), run the same ratio search on training data, and compare MixA-Q at matched bit operations against the uniform-bit baseline; if mAP at 1.25-1.35x bit-operation savings drops by more than a point on the held-out split, the lossless claim is falsified.","tokens_in":1724,"feed_emoji":"⚡","tokens_out":2001,"duration_ms":83484,"temperature":0.7,"pith_summary":"The paper tries to show that activation sparsity inside a Swin Transformer can be exploited through mixed-precision quantization rather than hard pruning: unimportant windows are still computed, but at a lower bit width, while important windows keep full precision. If true, quantized window-based vision transformers would gain a training-free computational speedup in post-training settings, and a retrained speedup plus lower quantization error in quantization-aware settings, while avoiding the information loss and out-of-distribution fragility of window pruning. MixA-Q reports a lossless 1.35x bit-operation speedup in PTQ on COCO object detection, a lossless 1.25x speedup in QAT at W4A4, and a 0.7 mAP improvement on the W4A4 baseline from focusing quantization error away from important windows.","feed_headline":"Swin Transformer gains a lossless 1.35x speedup from mixed bits","feed_subtitle":"Keeping unimportant windows at lower precision, instead of pruning them, saves computation at equal mAP.","key_machinery":"The Two-Branch Swin Block is the load-bearing mechanism: it replaces a standard Swin block with a high-precision branch and a low-precision branch that share the same attention and feed-forward weights but have separate layer normalizations and separate quantization step sizes and zero points, so windows can be processed at different precisions and scattered back into a single feature map. Window importance is the $\\ell^2$ norm of the features, computed once per stage. Per-stage compression ratios are chosen by an evolutionary multi-objective search over bit-operation savings versus COCO mAP, and for QAT a sparsity-aware quantization adaptation trains the model under randomly sampled ratio configurations so that any searched ratio set works without retraining. Dynamic activation distillation—gradients flowing mostly through the high-precision branch for important windows—is what shifts quantization error toward unimportant regions.","core_discovery":"The central claim is that intra-layer activation sparsity—the observation that some local windows of a feature map matter less than others—can be converted directly into precision savings instead of computation skipping. MixA-Q scores each window by the $\\ell^2$ norm of its features, routes high-scoring windows through a high-precision attention and feed-forward branch, routes the rest through a low-precision branch, and searches for per-stage compression ratios that minimize bit operations for a given mAP. On COCO object detection with a Swin-Tiny backbone, this yields a lossless 1.35x speedup when layered on a post-training-quantized W4A8 model, a lossless 1.25x speedup on a QAT W4A4 model, and a 1.53x speedup with roughly 1 mAP point lost when 30% activation pruning is added. The same sparsity-aware adaptation raises the W4A4 model's mAP from 43.1 to 43.8 by redistributing quantization error away from important windows, a 24% reduction in quantization degradation.","pith_inferences":["The principle 'compress rather than discard' may transfer to any hierarchical vision model where regions are scored and processed in batches, not only Swin; the hard requirement is hardware that can execute both branches at different precisions in parallel.","The measured speedups are in bit operations; actual wall-clock gains will depend on kernels that pack 2-bit/4-bit windows efficiently, which the paper does not demonstrate.","The evolutionary search selects compression ratios against COCO val mAP and the same split is used to report mAP, so a held-out evaluation would be needed to confirm the lossless claim generalizes.","Replacing the $\\ell^2$-norm importance score with a learned or attention-based saliency measure could make window assignment more robust to distribution shift and improve both pruning and mixed-precision selections."],"forward_implications":["On COCO object detection, MixA-Q with a PTQ model reaches 1.35x lower bit operations at equal mAP, meaning training-free deployment is possible.","At W4A4 in QAT, MixA-Q gives a lossless 1.25x speedup; combining it with 30% activation pruning gives 1.53x with roughly 1 mAP point lost.","Quantization-aware adaptation alone improves the W4A4 baseline by 0.7 mAP, so better allocation of quantization error, not just lower cost, is part of the payoff.","On out-of-distribution weather images, MixA-Q degrades less than activation pruning, and with deliberately reversed window selection it degrades 30.6% versus 43.4%, because compressed windows are still computed, not discarded.","Because MixA-Q works with both QAT and PTQ and does not change the network's data flow, it can be layered on top of existing uniform-bit quantization methods."],"supporting_citations":[{"why":"Defines the Swin Transformer whose windowed attention MixA-Q splits into high- and low-precision branches.","marker":"[22]"},{"why":"Supplies the window-importance scoring, sparsity-aware adaptation, and pruning baseline that MixA-Q converts from skipping to mixed precision.","marker":"[2]"},{"why":"Provides the W4A4 QAT baseline that MixA-Q is evaluated on and whose mAP improves by 0.7 points after adaptation.","marker":"[19]"},{"why":"Provides the PTQ baseline that MixA-Q is layered on for the training-free 1.35x speedup result.","marker":"[15]"},{"why":"Supplies the COCO dataset used for quantization experiments and Pareto-front comparisons.","marker":"[17]"},{"why":"Supplies the weather-shifted COCO-O images used to test out-of-distribution robustness.","marker":"[24]"}],"fun_headline_variants":["Mixed-precision Windows: lossless 1.35x speedup for quantized Swin ViTs","Sparsity-aware quantization squeezes 1.35x from Swin at same mAP","Two-bit branches: 1.35x faster Swin without accuracy loss","Quantization split by window importance gives Swin 1.35x speedup","Activation sparsity becomes bit savings: Swin gains 1.35x lossless"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The lossless-speedup claims rest on optimizing compression ratios against COCO val mAP and then reporting mAP on that same validation split, so the accuracy equivalence may not survive on a held-out test set.","fun_headline_variants_meta":{"raw":{"variants":["Mixed-precision Windows: lossless 1.35x speedup for quantized Swin ViTs","Sparsity-aware quantization squeezes 1.35x from Swin at same mAP","Two-bit branches: 1.35x faster Swin without accuracy loss","Quantization split by window importance gives Swin 1.35x speedup","Activation sparsity becomes bit savings: Swin gains 1.35x lossless"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000669,"raw_usage":{"total_tokens":3091,"prompt_tokens":1027,"completion_tokens":2064,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":1948}},"tokens_in":643,"tokens_out":2064,"duration_ms":15905,"temperature":1.0,"reasoning_tokens":1948,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:00:44.092843+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out COCO test-dev (or any split unused by the evolutionary search and adaptation), run the same ratio search on training data, and compare MixA-Q at matched bit operations against the uniform-bit baseline; if mAP at 1.25-1.35x bit-operation savings drops by more than a point on the held-out split, the lossless claim is falsified.","supporting_citations":[{"cited_title":"Swin transformer: Hierarchical vision transformer using shifted windows","cited_arxiv_id":null,"evidence_quote":"Defines the Swin Transformer whose windowed attention MixA-Q splits into high- and low-precision branches."},{"cited_title":"Sparsevit: Revisiting activation spar- sity for efficient high-resolution vision transformer","cited_arxiv_id":null,"evidence_quote":"Supplies the window-importance scoring, sparsity-aware adaptation, and pruning baseline that MixA-Q converts from skipping to mixed precision."},{"cited_title":"Oscillation-free quantization for low-bit vision transform- ers","cited_arxiv_id":null,"evidence_quote":"Provides the W4A4 QAT baseline that MixA-Q is evaluated on and whose mAP improves by 0.7 points after adaptation."},{"cited_title":"Repq- vit: Scale reparameterization for post-training quantization of vision transformers","cited_arxiv_id":null,"evidence_quote":"Provides the PTQ baseline that MixA-Q is layered on for the training-free 1.35x speedup result."},{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"Supplies the COCO dataset used for quantization experiments and Pareto-front comparisons."},{"cited_title":"Coco-o: A benchmark for object detectors under natural distribution shifts","cited_arxiv_id":null,"evidence_quote":"Supplies the weather-shifted COCO-O images used to test out-of-distribution robustness."}],"review_version":1}