Pith. sign in

REVIEW 3 major objections 5 minor 35 references

MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MixA-Q shows that assigning lower bit widths to less important windows—instead of pruning them—lets window-based vision transformers run at 1.35x lower computational cost with no measured accuracy loss.

desk verdict MixA-Q is a plausible, well-motivated extension of activation sparsity to mixed-precision quantization, but its 'lossless' speedup claim is weakened by tuning compression ratios on the same COCO val split used for evaluation. read the letter →

arxiv 2507.19131 v1 pith:ZK7O65ZM submitted 2025-07-25 cs.CV

Weitian Wang , Rai Shubham , Cecilia De La Parra , Akash Kumar This is my paper · ORCID
classification cs.CV
keywords mixed-precisionquantizationactivationsparsitySwinTransformervisiontransformersquantization-awaretrainingpost-trainingobjectdetectionout-of-distributionrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that activation sparsity inside a Swin Transformer can be exploited through mixed-precision quantization rather than hard pruning: unimportant windows are still computed, but at a lower bit width, while important windows keep full precision. If true, quantized window-based vision transformers would gain a training-free computational speedup in post-training settings, and a retrained speedup plus lower quantization error in quantization-aware settings, while avoiding the information loss and out-of-distribution fragility of window pruning. MixA-Q reports a lossless 1.35x bit-operation speedup in PTQ on COCO object detection, a lossless 1.25x speedup in QAT at W4A4, and a 0.7 mAP improvement on the W4A4 baseline from focusing quantization error away from important windows.

What carries the argument

The Two-Branch Swin Block is the load-bearing mechanism: it replaces a standard Swin block with a high-precision branch and a low-precision branch that share the same attention and feed-forward weights but have separate layer normalizations and separate quantization step sizes and zero points, so windows can be processed at different precisions and scattered back into a single feature map. Window importance is the $\ell^2$ norm of the features, computed once per stage. Per-stage compression ratios are chosen by an evolutionary multi-objective search over bit-operation savings versus COCO mAP, and for QAT a sparsity-aware quantization adaptation trains the model under randomly sampled ratio configurations so that any searched ratio set works without retraining. Dynamic activation distillation—gradients flowing mostly through the high-precision branch for important windows—is what shifts quantization error toward unimportant regions.

What would settle it

Hold out COCO test-dev (or any split unused by the evolutionary search and adaptation), run the same ratio search on training data, and compare MixA-Q at matched bit operations against the uniform-bit baseline; if mAP at 1.25-1.35x bit-operation savings drops by more than a point on the held-out split, the lossless claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that intra-layer activation sparsity—the observation that some local windows of a feature map matter less than others—can be converted directly into precision savings instead of computation skipping. MixA-Q scores each window by the $\ell^2$ norm of its features, routes high-scoring windows through a high-precision attention and feed-forward branch, routes the rest through a low-precision branch, and searches for per-stage compression ratios that minimize bit operations for a given mAP. On COCO object detection with a Swin-Tiny backbone, this yields a lossless 1.35x speedup when layered on a post-training-quantized W4A8 model, a lossless 1.25x speedup on a QAT W4A4 model, and a 1.53x speedup with roughly 1 mAP point lost when 30% activation pruning is added. The same sparsity-aware adaptation raises the W4A4 model's mAP from 43.1 to 43.8 by redistributing quantization error away from important windows, a 24% reduction in quantization degradation.

Load-bearing premise

The lossless-speedup claims rest on optimizing compression ratios against COCO val mAP and then reporting mAP on that same validation split, so the accuracy equivalence may not survive on a held-out test set.

Editorial extensions

If this is right

  • On COCO object detection, MixA-Q with a PTQ model reaches 1.35x lower bit operations at equal mAP, meaning training-free deployment is possible.
  • At W4A4 in QAT, MixA-Q gives a lossless 1.25x speedup; combining it with 30% activation pruning gives 1.53x with roughly 1 mAP point lost.
  • Quantization-aware adaptation alone improves the W4A4 baseline by 0.7 mAP, so better allocation of quantization error, not just lower cost, is part of the payoff.
  • On out-of-distribution weather images, MixA-Q degrades less than activation pruning, and with deliberately reversed window selection it degrades 30.6% versus 43.4%, because compressed windows are still computed, not discarded.
  • Because MixA-Q works with both QAT and PTQ and does not change the network's data flow, it can be layered on top of existing uniform-bit quantization methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The principle 'compress rather than discard' may transfer to any hierarchical vision model where regions are scored and processed in batches, not only Swin; the hard requirement is hardware that can execute both branches at different precisions in parallel.
  • The measured speedups are in bit operations; actual wall-clock gains will depend on kernels that pack 2-bit/4-bit windows efficiently, which the paper does not demonstrate.
  • The evolutionary search selects compression ratios against COCO val mAP and the same split is used to report mAP, so a held-out evaluation would be needed to confirm the lossless claim generalizes.
  • Replacing the $\ell^2$-norm importance score with a learned or attention-based saliency measure could make window assignment more robust to distribution shift and improve both pruning and mixed-precision selections.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MixA-Q, a mixed-precision activation quantization framework for window-based vision transformers (Swin). For a given uniform-bit quantization configuration, MixA-Q splits each Swin block into high- and low-precision branches, assigns a lower activation bit width to windows deemed less important by an L2-norm importance score, and scatters the results back to the feature map. Compression ratios are chosen by an NSGA-II evolutionary search over a discrete space (six ratios for Swin-Tiny), optimizing bi-objectively for bit-operation savings and COCO val mAP. The method is integrated with QAT (OFQ) and PTQ (RepQ), and optionally combined with activation pruning. Experiments on COCO object detection and panoptic segmentation report a lossless 1.25x speedup with QAT, a training-free 1.35x speedup with PTQ, up to 1.82x speedups with pruning at moderate mAP drop, and an improvement of 0.7 mAP for the W4A4 quantized model attributed to sparsity-aware quantization adaptation (SAQA). The paper also evaluates robustness to COCO-O and to adversarial window selection, where MixA-Q degrades less than SparseViT.

Significance. If the results hold, the paper makes a useful conceptual contribution by transposing activation-sparsity exploitation from pruning to mixed-precision quantization, and the Two-Branch Swin Block is a clean, integration-friendly mechanism. The inclusion of PTQ without training is a genuinely attractive property, and the OOD-robustness comparison with SparseViT is a thoughtful and falsifiable check. However, the headline lossless-accuracy and speedup claims are currently supported only by (i) mAP numbers computed on the same COCO val split that the evolutionary search optimizes, and (ii) bit-operation counts rather than wall-clock latency. These issues are load-bearing for the stated contributions and need to be addressed before the claims can be accepted as established.

major comments (3)
  1. [Sec. 3.3 and Sec. 4.1] The evolutionary search in Sec. 3.3 explicitly optimizes compression ratios using the objective "mAP is the mean average precision on the COCO val dataset," while the mAP numbers reported in Tables 1, 2, and 4 and in Fig. 4 are computed on the very same COCO val split. The search space of six discretized ratios (about 9^6 ≈ 531,000 configurations) makes it likely that the selected Pareto-front points are optimistically biased relative to true held-out performance. The reported "lossless" margins are small (e.g., Table 1: 43.2 vs. 43.1 for the 1.24x point, and 42.3 vs. 43.1 for the 1.35x point), so a selection bias of a few tenths of mAP is of the same magnitude as the claimed accuracy equivalence. I ask the authors to report results on a held-out split that was not used during the search, or to provide repeated-search variance estimates; at minimum, the paper should explicitly state that all reported mAP values are the search objective itself and therefore not independent measurements.
  2. [Sec. 4 (opening) and Abstract] The abstract and conclusion use the term "speedup" (1.25x, 1.35x, 1.53x), but Sec. 4 states that all computation savings are measured in bit operations (BOPs) of the backbone. No wall-clock latency, throughput, or energy measurements are reported. Mixed-precision branches with different bit widths, the gather/scatter operations around each window attention, duplicated layer normalizations, and non-uniform window counts across blocks will introduce overheads that BOPs does not capture. The claims should be rephrased as "theoretical bit-operation savings" unless actual runtime measurements are provided, or the limitation should be stated explicitly in the abstract and conclusion.
  3. [Sec. 4.3 and Abstract] The abstract claims a "training-free 1.35x computational speedup without accuracy loss in PTQ configuration," but the PTQ results are only presented as a Pareto plot in Fig. 7b, without tabulated mAP values and without an explicit indication of which configuration corresponds to the 1.35x point or its exact mAP. Please report the compression-ratio configuration, the equivalent activation bits, the BOPs, and the mAP for the claimed 1.35x point, together with the baseline W4A8 mAP, so that the lossless claim can be verified numerically.
minor comments (5)
  1. [Sec. 1, contribution 3] "SpraseViT" should be "SparseViT".
  2. [Sec. 5] The concluding paragraph contains a typo: "bit widts" should be "bit widths".
  3. [Appendix 6.1, Algorithm 1] The output line "such that Pri = S" should read "such that Σ r_i = S".
  4. [Sec. 4.3] "the weights and bias of of the QKV layer" contains a duplicated "of".
  5. [Sec. 4.1, Table 1] The table caption says "Act Bit represents the activations' equivalent bit width," but the meaning of the parenthetical speedup column header "1.33x" etc. should be clarified to indicate that these are relative to the W4A4 OFQ model, not to the full-precision model.

Circularity Check

1 steps flagged · score 6.0 of 10

Compression ratios are searched directly against COCO val mAP, and the same COCO val mAP is then reported as 'lossless'; the headline accuracy-equivalence claims are fitted outcomes rather than independent predictions.

  1. fitted input called prediction [Sec 3.3 Evolutionary Search of Compression Ratios; results reported in Sec 4.1 Tab 1-2, Sec 4.3 Fig 7b]
    "The search for optimal compression ratios can be formed as a bi-objective optimization problem: Objectives: computational savings, mAP (mean average precision); Variables: compression ratios for each stage ... mAP is the mean average precision on the COCO val dataset. ... during the evolutionary search, we evaluate different compression ratios without recalibrating the model."

    The compression ratios are the free hyperparameters of MixA-Q, and they are tuned by NSGA-II with mAP on COCO val as one of the two objectives. The paper then reports mAP on that same COCO val split (Tabs 1-2 and Fig 7b) as the accuracy of the selected configurations, including the headline 'lossless 1.35x speedup' and 'lossless 1.25x speedup' claims. With six ratios discretized to {0%,10%,...,80%}, the search space contains about 531,441 configurations; selecting Pareto-optimal points against val mAP and then reporting those same val mAP values as 'without accuracy loss' means the reported numbers are selection maxima over the searched set, not independent evaluations.

full rationale

There is no equation-level self-definitional circularity in MixA-Q: the two-branch block, compression-ratio sampling, and BOP-based speedup calculations are bona fide constructions, and the baselines (SparseViT, OFQ, RepQ) are external prior work by other authors, not self-citations. The method is self-contained against external benchmarks. The concrete circular step is the evaluation protocol: the compression ratios that define every reported MixA-Q point are optimized by evolutionary search against COCO val mAP (Sec 3.3), and the same COCO val mAP is then reported as the accuracy of those points (Sec 4.1, 4.3). The headline 'without accuracy loss' and '1% mAP drop' statements are therefore not predictions of an independent evaluation; they are descriptions of the Pareto front selected on the reported metric. Because the search space is large and no held-out split or variance estimate is reported, the claimed lossless speedups are partially forced by the selection procedure. This warrants a partial circularity score of 6; the paper would need a held-out test split or repeated-search variance reporting to establish the accuracy-equivalence claims as independent results.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the compression-ratio search (fitted to the evaluation metric), hand-chosen bit widths and pruning ratios, and a set of domain assumptions inherited from SparseViT (L2-norm importance, BOPs proxy, score reuse for shifted windows). No new physical entities are introduced.

free parameters (4)
  • Per-stage compression ratios (6 ratios for Swin-Tiny) = Varies by experiment; searched over {0%, 10%, ..., 80%}
    Optimized with NSGA-II (Sec 3.3) using COCO val mAP as the objective; these ratios directly determine the claimed bit-operation savings and accuracy.
  • Low-precision branch bit width = 2-bit for QAT; 4-bit for PTQ (baseline 4-bit/8-bit)
    Chosen by hand as the compression precision for less-important windows; central to the 50% per-window BOPs saving claim (Sec 3.4).
  • Activation pruning ratio (when incorporated) = 30% overall
    Selected to reach high speedup targets in Sec 4.1.1; the paper does not justify the specific value.
  • Uniform-sum sampling range and target sum bounds = Sum range [0.6, 4.2]; per-ratio upper bound 0.8
    Hand-set search-space parameters (Appendix 6.1) to target relative costs [0.65, 0.95]; they influence which configurations are reachable.
assumptions (5)
  • domain assumption The L2 norm of a window's features is a reliable importance score for window contribution to detection accuracy.
    Adopted from SparseViT [2] and used to partition windows into high/low branches (Sec 3.2); if this criterion is poor, the bit-width allocation is misdirected.
  • domain assumption Bit operations (BOPs) are a faithful proxy for computational speedup.
    All speedup claims are computed as BOPs reduction of the backbone (Sec 4, Sec 6.2); the paper provides no wall-clock latency validation, so real speedups are unverified.
  • domain assumption Importance scores computed once per stage remain valid for both the unshifted and shifted window blocks.
    Sec 3.2 states the score is computed once before the first Swin block and shared across the stage; the paper does not explain how scores are applied to shifted windows, whose token composition differs.
  • domain assumption The two-branch block with shared weights and independent quantization parameters preserves the float model's function after quantization.
    The design assumes that separate step sizes/zero points for high and low branches are sufficient to keep both branches accurate (Sec 3.2); no theoretical or ablation evidence isolates this.
  • domain assumption Performance on the COCO val set used for evolutionary search is representative of true deployment performance.
    Sec 3.3 uses COCO val mAP as the search objective, and Sec 4 reports results on the same split; no test-dev or separate holdout is used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective." pith.science (2026). https://pith.science/paper/ZK7O65ZM

@misc{pith2026250719131,
  author       = {Pith},
  title        = {Pith review of: MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZK7O65ZM}},
  note         = {Machine review of arXiv:2507.19131}
}
read the original abstract

In this paper, we propose MixA-Q, a mixed-precision activation quantization framework that leverages intra-layer activation sparsity (a concept widely explored in activation pruning methods) for efficient inference of quantized window-based vision transformers. For a given uniform-bit quantization configuration, MixA-Q separates the batched window computations within Swin blocks and assigns a lower bit width to the activations of less important windows, improving the trade-off between model performance and efficiency. We introduce a Two-Branch Swin Block that processes activations separately in high- and low-bit precision, enabling seamless integration of our method with most quantization-aware training (QAT) and post-training quantization (PTQ) methods, or with simple modifications. Our experimental evaluations over the COCO dataset demonstrate that MixA-Q achieves a training-free 1.35x computational speedup without accuracy loss in PTQ configuration. With QAT, MixA-Q achieves a lossless 1.25x speedup and a 1.53x speedup with only a 1% mAP drop by incorporating activation pruning. Notably, by reducing the quantization error in important regions, our sparsity-aware quantization adaptation improves the mAP of the quantized W4A4 model (with both weights and activations in 4-bit precision) by 0.7%, reducing quantization degradation by 24%.

Figures

Figures reproduced from arXiv: 2507.19131 by the authors.

Figure 1
Figure 1. Example for how MixA-Q leverages activation sparsity [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of how our MixA-Q replaces the window pruning in SparseViT [ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Two-branch swin block that replaces the original swin blocks in Swin Transformer. The windows are gathered into high- and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Pareto fronts of different methods. The x-axis is the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Signal-to-quantization-noise ratio (SQNR [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visualization of the window selection of SparseViT over [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Further evaluation result on panoptic segmentation task [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Pareto front after two generations of evolutionary search [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 26 canonical work pages

  1. [1]

    Pymoo: Multi-objective optimization in python

    Julian Blank and Kalyanmoy Deb. Pymoo: Multi-objective optimization in python. IEEE Access, 8:89497–89509, 2020. 4

    work page 2020

  2. [2]

    Sparsevit: Revisiting activation spar- sity for efficient high-resolution vision transformer

    Xuanyao Chen, Zhijian Liu, Haotian Tang, Li Yi, Hang Zhao, and Song Han. Sparsevit: Revisiting activation spar- sity for efficient high-resolution vision transformer. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2061–2070, 2023. 1, 2, 3, 4, 5, 6

    work page 2023

  3. [3]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 1, 2

    work page 2022

  4. [4]

    A fast and elitist multiobjective genetic algo- rithm: Nsga-ii

    Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. A fast and elitist multiobjective genetic algo- rithm: Nsga-ii. IEEE transactions on evolutionary computa- tion, 6(2):182–197, 2002. 4

    work page 2002

  5. [5]

    Emq: Evolving training-free proxies for automated mixed precision quantization

    Peijie Dong, Lujun Li, Zimian Wei, Xin Niu, Zhiliang Tian, and Hengyue Pan. Emq: Evolving training-free proxies for automated mixed precision quantization. In Proceedings of the IEEE/CVF international conference on computer vision, pages 17076–17086, 2023. 3

    work page 2023

  6. [6]

    Hawq: Hessian aware quantization of neural networks with mixed-precision

    Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Ma- honey, and Kurt Keutzer. Hawq: Hessian aware quantization of neural networks with mixed-precision. In Proceedings of the IEEE/CVF international conference on computer vision, pages 293–302, 2019. 3

    work page 2019

  7. [7]

    Hawq-v2: Hessian aware trace-weighted quantization of neural networks

    Zhen Dong, Zhewei Yao, Daiyaan Arfeen, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Hawq-v2: Hessian aware trace-weighted quantization of neural networks. Ad- vances in neural information processing systems, 33:18518– 18529, 2020. 3

    2020

  8. [8]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 1, 2

    arXiv 2010

Show all 35 references
  1. [9]

    Adaptive token sampling for efficient vision transformers

    Mohsen Fayyaz, Soroush Abbasi Koohpayegani, Farnoush Rezaei Jafari, Sunando Sengupta, Hamid Reza Vaezi Joze, Eric Sommerlade, Hamed Pirsiavash, and J¨urgen Gall. Adaptive token sampling for efficient vision transformers. In European Conference on Computer Vision, pages 396–414...

    2022

  2. [10]

    Jumping through local minima: Quantization in the loss landscape of vision transformers

    Natalia Frumkin, Dibakar Gope, and Diana Marculescu. Jumping through local minima: Quantization in the loss landscape of vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 16978–16988, 2023. 8

    2023

  3. [11]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 5

    2017

  4. [12]

    Metamix: Meta-state precision searcher for mixed-precision activation quantization

    Han-Byul Kim, Joo Hyung Lee, Sungjoo Yoo, and Hong- Seok Kim. Metamix: Meta-state precision searcher for mixed-precision activation quantization. Proceedings of the AAAI Conference on Artificial Intelligence , 38(12): 13132–13141, 2024. 3

    2024

  5. [13]

    Spvit: Enabling faster vision transformers via latency-aware soft token pruning

    Zhenglun Kong, Peiyan Dong, Xiaolong Ma, Xin Meng, Wei Niu, Mengshu Sun, Xuan Shen, Geng Yuan, Bin Ren, Hao Tang, et al. Spvit: Enabling faster vision transformers via latency-aware soft token pruning. In European conference on computer vision, pages 620–640. Springer, 2022. 3

    2022

  6. [14]

    Q-vit: Accurate and fully quantized low-bit vision transformer

    Yanjing Li, Sheng Xu, Baochang Zhang, Xianbin Cao, Peng Gao, and Guodong Guo. Q-vit: Accurate and fully quantized low-bit vision transformer. Advances in neural information processing systems, 35:34451–34463, 2022. 2

    2022

  7. [15]

    Repq- vit: Scale reparameterization for post-training quantization of vision transformers

    Zhikai Li, Junrui Xiao, Lianwei Yang, and Qingyi Gu. Repq- vit: Scale reparameterization for post-training quantization of vision transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 17227– 17236, 2023. 2, 3, 7

    2023

  8. [16]

    Not all patches are what you need: Expediting vision transformers via token reorganiza- tions

    Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie. Not all patches are what you need: Expediting vision transformers via token reorganiza- tions. arXiv preprint arXiv:2202.07800, 2022. 3

    2022 arXiv

  9. [17]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceeding...

    2014

  10. [18]

    QLLM: Accurate and efficient low-bitwidth quantization for large language models

    Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jian- fei Cai, and Bohan Zhuang. QLLM: Accurate and efficient low-bitwidth quantization for large language models. In In- ternational Conference on Learning Representations (ICLR),

  11. [19]

    Oscillation-free quantization for low-bit vision transform- ers

    Shih-Yang Liu, Zechun Liu, and Kwang-Ting Cheng. Oscillation-free quantization for low-bit vision transform- ers. In International conference on machine learning, pages 21813–21824. PMLR, 2023. 2, 5, 6

    2023

  12. [20]

    Pq-sam: Post-training quantization for segment any- thing model

    Xiaoyu Liu, Xin Ding, Lei Yu, Yuanyuan Xi, Wei Li, Zhi- jun Tu, Jie Hu, Hanting Chen, Baoqun Yin, and Zhiwei Xiong. Pq-sam: Post-training quantization for segment any- thing model. In European Conference on Computer Vision, pages 420–437. Springer, 2024. 2

    2024

  13. [21]

    Revisiting token pruning for object detection and instance segmentation

    Yifei Liu, Mathias Gehrig, Nico Messikommer, Marco Can- nici, and Davide Scaramuzza. Revisiting token pruning for object detection and instance segmentation. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2658–2668, 2024. 1, 3

    2024

  14. [22]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 1, 2, 4, 5

    2021

  15. [23]

    Ompq: Orthogonal mixed precision quantization

    Yuexiao Ma, Taisong Jin, Xiawu Zheng, Yan Wang, Huixia Li, Yongjian Wu, Guannan Jiang, Wei Zhang, and Rongrong Ji. Ompq: Orthogonal mixed precision quantization. In Pro- ceedings of the AAAI conference on artificial intelligence , pages 9029–9037, 2023. 3

    2023

  16. [24]

    Coco-o: A benchmark for object detectors under natural distribution shifts

    Xiaofeng Mao, Yuefeng Chen, Yao Zhu, Da Chen, Hang Su, Rong Zhang, and Hui Xue. Coco-o: A benchmark for object detectors under natural distribution shifts. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 6339–6350, 2023. 7

    2023

  17. [25]

    Data-free quantization through weight equal- ization and bias correction

    Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling. Data-free quantization through weight equal- ization and bias correction. In Proceedings of the IEEE/CVF international conference on computer vision , pages 1325– 1334, 2019. 2

    2019

  18. [26]

    Up or down? adap- tive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Chris- tos Louizos, and Tijmen Blankevoort. Up or down? adap- tive rounding for post-training quantization. In International conference on machine learning, pages 7197–7206. PMLR,

  19. [27]

    Dynamicvit: Efficient vision transformers with dynamic token sparsification

    Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynamicvit: Efficient vision transformers with dynamic token sparsification. Advances in neural information processing systems, 34:13937–13949,

  20. [28]

    Entropy-driven mixed- precision quantization for deep network design

    Zhenhong Sun, Ce Ge, Junyan Wang, Ming Lin, Hesen Chen, Hao Li, and Xiuyu Sun. Entropy-driven mixed- precision quantization for deep network design. Advances in Neural Information Processing Systems, 35:21508–21520,

  21. [29]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 2

    2017

  22. [30]

    Thinking in granularity: Dynamic quantization for image super-resolution by intriguing multi- granularity clues, 2024

    Mingshen Wang, Zhao Zhang, Feng Li, Ke Xu, Kang Miao, and Meng Wang. Thinking in granularity: Dynamic quantization for image super-resolution by intriguing multi- granularity clues, 2024. 3

    2024

  23. [31]

    Apq: Joint search for network architecture, pruning and quantization policy

    Tianzhe Wang, Kuan Wang, Han Cai, Ji Lin, Zhijian Liu, Hanrui Wang, Yujun Lin, and Song Han. Apq: Joint search for network architecture, pruning and quantization policy. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 2078–2087, 2020. 3

    2020

  24. [32]

    Patch- wise mixed-precision quantization of vision transformer,

    Junrui Xiao, Zhikai Li, Lianwei Yang, and Qingyi Gu. Patch- wise mixed-precision quantization of vision transformer,

  25. [33]

    K-net: Towards unified image seg- mentation

    Wenwei Zhang, Jiangmiao Pang, Kai Chen, and Chen Change Loy. K-net: Towards unified image seg- mentation. Advances in Neural Information Processing Systems, 34:10326–10338, 2021. 2

    2021

  26. [34]

    Towards accurate post-training quantization of vision transformers via error reduction

    Yunshan Zhong, You Huang, Jiawei Hu, Yuxin Zhang, and Rongrong Ji. Towards accurate post-training quantization of vision transformers via error reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. 3

    2025

  27. [35]

    Appendix 6.1. Uniform-sum Compression Ratio Sampling We share the details of our sampling algorithm here: Algorithm 1 Uniform-Sum Compression Ratio Sampling 1: Input: Number of layers n, predefined sum range [smin,s max], upper boundu 2: Output: compression ratios{r0,r 1,...,r...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.