Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

This paper claims that a transformer layer's activation entropy predicts how much that layer can be safely quantized, and that greedily sending the lowest-entropy layers to 1-2 bits while keeping the rest at 4 bits yields a better memory-pe

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 14:38 UTC pith:FQTITAF5

load-bearing objection A practical entropy-guided layerwise quantization recipe for MLLMs, with good ablations but some overstated numbers and an untested ordering-stability assumption. the 5 major comments →

arxiv 2509.23729 v3 pith:FQTITAF5 submitted 2025-09-28 cs.CV cs.AIcs.LGeess.IV

LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

Shubhang Bhatnagar , Andy Xu , Kar-Han Tan , Narendra Ahuja This is my paper
classification cs.CV cs.AIcs.LGeess.IV
keywords post-training quantizationmultimodal large language modelslayerwise quantizationactivation entropyultra-low bit quantizationvision-language modelsmixed-modal calibrationmemory compression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that multimodal LLMs collapse under sub-4-bit quantization because visual tokens produce high-entropy activations, and that this entropy varies sharply across layers. It proposes LUQ, a post-training method that measures each layer's output activation entropy and greedily quantizes the lowest-entropy layers to 1-2 bits while leaving the rest at 4 bits. On LLaVA-1.5 and Qwen-2.5-VL, the resulting models are 40% and 31.5% smaller than uniform 4-bit baselines while staying close on VQA benchmarks. The insight that entropy is a usable sensitivity signal turns ultra-low-bit compression of vision-language models from a uniform gamble into a per-layer decision.

Core claim

The central discovery is that the Shannon entropy of a transformer layer's output activations, estimated by K-means clustering on a small calibration set, is a usable proxy for that layer's tolerance to ultra-low-bit quantization. Layers whose activations form low-entropy clusters encode simpler functions and survive 1-2 bit weights with little loss; high-entropy layers, especially those processing visual tokens, degrade sharply. LUQ exploits this by sorting layers by entropy and quantizing the lowest-entropy ones first to 1-2 bits with a one-bit PTQ method (BiLLM) while the rest stay at 4-bit GPTQ, with a validation threshold or memory budget deciding the cutoff. The paper also reports that

What carries the argument

The load-bearing object is the layerwise activation entropy H_i, computed as the Shannon entropy of the empirical cluster-assignment distribution obtained by running K-means with K=100 (chosen by a rank-stability elbow analysis) on the layer's output tokens from a 128-sequence multimodal calibration set. This single scalar ranks all layers; the greedy permutation orders layers by ascending H_i, and at each step the next lowest-entropy layer is quantized to 1-2 bits via a one-bit PTQ method, while non-selected layers stay at 4-bit GPTQ. A validation check or binary search sets the cutoff. The paper argues that entropy tracks functional complexity, so low-H layers can be aggressively compresse

Load-bearing premise

The entire method rests on the assumption that activation entropy measured on the unquantized model with a 128-sequence calibration set gives a stable and correct ordering of which layers can tolerate 1-2 bit quantization even after earlier layers have already been quantized.

What would settle it

Compare LUQ's selected layers against a random selection of the same number of layers, or against layers chosen by the actual per-layer output perturbation caused by quantizing each layer in isolation. If the entropy-selected set does not outperform random or error-based selection on the same benchmarks, entropy is not the active ingredient.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If correct, LUQ gives a practical way to deploy 7B multimodal LLMs on memory-constrained devices: LLaVA-1.5 runs at 2.54 average bits and Qwen-2.5-VL at 2.75 bits with only modest VQA loss.
  • Because LUQ is agnostic to the underlying PTQ method, any future improvement in 1-bit weight quantizers would automatically translate into better accuracy at the same memory footprint.
  • Mixed-modal calibration becomes a recommended practice for sub-4-bit MLLM quantization, since it consistently improves accuracy compared with text-only calibration in that regime.
  • The entropy-versus-robustness relationship, if it holds, gives a principled way to choose layerwise precision for other compression tasks like pruning or knowledge distillation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to compare the entropy-based layer ordering against a ranking based on actual per-layer output perturbation under quantization; if they diverge, entropy is a weak proxy for the true sensitivity.
  • The method measures entropy once on the unquantized model, but in principle the ordering could be recomputed after each quantization step; the paper's static ordering assumes relative layer complexity is stable under quantization, which is not directly tested.
  • The abstract's 'less than 10% degradation on MME' is optimistic for reasoning-heavy sub-scores: the paper's own Table 1 shows MME Cognition dropping 26.6% for LLaVA-1.5 (from 347 to 257). A careful reader should look at per-benchmark numbers rather than the headline average.
  • The entropy signal might be partially confounded with layer depth: the paper's own ablation shows depth-based selection performs competitively, so isolating the entropy contribution over a broader set of models would clarify the mechanism.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript presents LUQ, a post-training quantization strategy for multimodal LLMs that assigns ultra-low-bit (1-bit BiLLM) quantization to layers selected by low activation entropy and 4-bit GPTQ quantization to the remaining layers. The authors measure Shannon entropy of intermediate activations on a 128-sequence mixed-modal calibration set, sort layers by entropy, and greedily quantize the lowest-entropy layers until a memory or performance threshold is reached. They also propose mixed text/image calibration. On LLaVA-1.5 7B and Qwen-2.5-VL 7B across 9 VQA benchmarks, they report average bit-width reductions of roughly 40% and 31.5% relative to 4-bit baselines while maintaining competitive accuracy, and they claim a better memory-performance frontier than uniform GPTQ/AWQ/BiLLM.

Significance. If the entropy-ordering signal is stable and reproducible, LUQ offers a simple way to identify quantization-tolerant layers in MLLMs and improves the memory-performance trade-off compared to uniform precision. The paper includes a direct high-entropy vs low-entropy ablation (Fig. 6a) and continuous trade-off curves (Figs. 4/5), which are more informative than single operating points and support the sign of the proposed effect. However, the current evidence is incomplete: the entropy ordering is not tested for stability across calibration samples or under quantized forward passes, the LLaVA operating point is selected using the same MME benchmark on which it is evaluated, and several headline numbers in the abstract are not supported by Table 1. These issues prevent the central claim from being accepted as established.

major comments (5)
  1. [Sec. 3.2, Sec. 3.3, Eq. (6), Appendix B] The entropy ordering π is computed once on the unquantized model from a single 128-sequence calibration set and is fixed for the whole greedy procedure (Eqs. (6)-(7)). The only stability analysis is with respect to the K-means cluster count K (Appendix B); there is no test of ranking stability across calibration draws or after earlier layers have been quantized. If the ordering is sample-dependent or shifts when activations change, the greedy selection loses its justification and LUQ reduces to arbitrary layer choice. The Limitations section admits entropy is 'only an estimate of function complexity,' but the manuscript does not quantify the uncertainty of this estimate. Please add bootstrap resampling of the calibration set and a check of whether the ordering persists when activations are recomputed under quantized layers.
  2. [Sec. 4.2, Table 1, Abstract] The LLaVA configuration is obtained by imposing a 100-point MME Perception threshold relative to GPTQ-4, and the same benchmark is then reported as evidence. This is a selection-on-test issue. Moreover, the abstract's 'less than 10% degradation on MME' is not sustained by Table 1: for LLaVA, MME Cognition drops 25.9% vs GPTQ-4 (257 vs 347) and 26.6% vs FP16 (257 vs 350), while total MME drops 12.8% vs FP16 (1622 vs 1860) and 9.7% vs GPTQ-4 (1622 vs 1797). Please select k* on a separate validation benchmark or held-out split, and state the reference point for the percentage degradation in the abstract. The relationship between Eq. (8), which evaluates on the calibration set, and the Sec. 4.2 threshold on MME should also be clarified.
  3. [Abstract, Sec. 1, Table 1] The claimed '40% less memory' for LLaVA is not supported by the reported average bit-widths. 2.54 bits vs 4.0 bits is a 36.5% reduction, not 40%. Since only the LLM backbone is quantized, the whole-model memory saving is smaller still. Please compute the percentages from actual memory measurements (including vision encoder and embeddings) or revise the claim.
  4. [Sec. 3.2] The notation for X_i and X'_i is confusing: N is used for both the number of calibration sequences and the sequence length, and it is unclear whether entropy is computed over pooled tokens or per-sequence. Please use distinct symbols for number of sequences, sequence length, and hidden dimension, state the total number of tokens used in K-means, and report whether the entropy values are averaged over multiple K-means seeds. This matters because the layer selection is based on small entropy differences.
  5. [Sec. 4.4.1] The low-vs-high entropy ablation is useful, but without a random layer ordering baseline it does not fully establish that the entropy ordering, rather than 'any non-depth ordering,' is responsible for the gains. Since the central claim is that entropy specifically predicts robustness, please include a random-ordering baseline repeated over multiple seeds in the ablation.
minor comments (5)
  1. [Appendix B] The text refers to 'Equation 7' when defining the layer ordering used for stability; the ordering is defined in Eq. (6), and the notation π_k is not defined.
  2. [Throughout] Typos include 'and and' in the benchmark enumeration (Sec. 4.1), 'which is which is' in the Acknowledgment, and missing spaces around math in several places.
  3. [Figure 8] The entropy values shown span a very narrow range (about 6.40-6.54). Given that selection is driven by these differences, please report the scale of the entropy differences and, ideally, confidence intervals from repeated calibration samples.
  4. [Table 1] Results are reported as the mean over 3 runs, but no standard deviations are given. For comparisons within 1-2 points, variance information is needed to assess whether the differences are meaningful.
  5. [Sec. 4.2] The text says 'the performance cost is only 5.8% on Perception and 25.9% on Cognition' for LLaVA, which is in tension with the abstract's 'less than 10% on MME' claim. Please make the reference points consistent.

Circularity Check

0 steps flagged

No significant circularity: entropy-based layer selection is defined independently of target benchmarks, and the reported MME operating point is a validation choice, not a fitted prediction.

full rationale

LUQ's derivation chain is not circular. The layer-complexity signal H_i is computed from output activations via K-means and Shannon entropy (Eqs. 2-5) on a calibration set, with no reference to any VQA benchmark; the ordering π is fixed by Eq. (6) before quantization. The central claim that low-entropy layers tolerate ultra-low-bit quantization is tested by direct ablations: low-entropy-first vs. high-entropy-first (Fig. 6a) and entropy vs. layer depth (Table 2), neither of which tunes the ordering on the evaluation metric. The LLaVA operating point in Sec. 4.2 is set by a threshold on MME Perception, so the reported MME Perception number is a validation-selected operating point rather than an independent hold-out prediction; however, this is a model-selection/hyperparameter choice, not a reduction of the method's output to its input. The same holds for the calibration-set reuse (entropy, quantizer fitting, and optional binary-search thresholding): it creates overfitting risk, not definitional circularity. The paper's own Limitation (3) concedes entropy 'only is an estimate of function complexity,' which is a validity caveat, not a circular step. There are no load-bearing self-citations, imported uniqueness theorems, or ansatz-by-citation moves. Hence score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method introduces no new entities. It does rely on the entropy quantile as a sensitivity measure, the stability of that measure under quantization, and the representativeness of a small calibration set; these are the assumptions that carry the method and are only partially validated.

free parameters (4)
  • Number of K-means clusters K = 100
    Used to estimate activation entropy; selected via Kneedle on the model's own rank-stability curve (Sec 4.1, Appendix B).
  • Calibration mixing ratio alpha = 0.5 (1:1 text/multimodal)
    Controls composition of calibration tokens in Eq. 9; chosen based on ablation in Sec 4.4.2 where gains saturate.
  • Performance threshold tau / memory budget = LLaVA: 100-point MME Perception drop; Qwen: 2.5 GB budget
    Determines how many layers are quantized (Eq. 8); these user-set targets directly create the headline operating points in Table 1.
  • Calibration set size = 128 sequences x 2048 tokens
    Used for entropy estimation, PTQ parameter fitting, and validation-based layer-count selection (Sec 4.1).
axioms (5)
  • domain assumption Activation entropy is a proxy for layerwise functional complexity and quantization robustness
    Central premise of LUQ, introduced in Sec 3.2; acknowledged as approximate in Limitations.
  • domain assumption Entropy ordering computed on the clean model remains valid after earlier layers are quantized
    LUQ greedily quantizes in a fixed order (Sec 3.3) without recomputing entropies; no test of this stability is provided.
  • domain assumption K-means with K=100 faithfully discretizes layer activation distributions
    Entropy estimates depend on K; the chosen K is derived from the model's own stability curve, not from an external benchmark.
  • domain assumption The 128-sequence calibration set is representative of the deployment distribution
    Standard PTQ assumption; here it also drives the validation-based selection of the number of ultra-low-bit layers.
  • domain assumption BiLLM and GPTQ behave as described in their original papers when used as base quantizers
    LUQ's performance is bounded by these base methods, as the paper itself notes in Limitations; no re-verification is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 15764 in / 16291 out tokens · 297678 ms · 2026-08-04T14:38:15.037810+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/FQTITAF5

@misc{pith2026250923729,
  author       = {Pith},
  title        = {Pith review of: LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQTITAF5}},
  note         = {Machine review of arXiv:2509.23729}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) with multimodal capabilities have revolutionized vision-language tasks, but their deployment often requires huge memory and computational resources. Post-training quantization (PTQ) has successfully compressed language models to as low as 1-bit precision, its effectiveness for multimodal LLMs (MLLMs) remains unexplored. In this paper, we present the first method for ultra-low-bit (<4-bit) quantization of MLLMs. Our analysis reveals that multimodal tokens and intermediate layer activations produced by them exhibit significantly higher entropy compared to text tokens, indicating greater functional complexity that makes MLLMs less tolerant to ultra-low bit quantization. However, this entropy varies significantly across layers, with some layers producing lower-entropy activation distributions that we empirically show can better tolerate ultra-low bit quantization. Existing PTQ methods optimize weight quantization within each layer but apply the same target precision uniformly, ignoring this variation in complexity across layers. Building on this insight, we propose LUQ: Layerwise Ultra-Low Bit Quantization, which characterizes each transformer layer's functional complexity via its output activation entropy and selectively applies ultra-low bit quantization to layers encoding simpler, more compressible functions. We also show that multimodal calibration (image and text tokens) boosts VQA performance in the ultra-low bit regime. Evaluated on LLaVA-1.5 and Qwen-2.5-VL across 9 VQA benchmarks, LUQ models use 40% and 31% less memory than their 4-bit counterparts while exhibiting less than 10% degradation on MME.

Figures

Figures reproduced from arXiv: 2509.23729 by Andy Xu, Kar-Han Tan, Narendra Ahuja, Shubhang Bhatnagar.

Figure 1
Figure 1. Figure 1: Performance vs. Compression Trade￾off for Qwen 2.5 VL. Our method, Layerwise Ultra￾Low Bit Quantization (LUQ), achieves a better trade￾off on the MME benchmark compared to AWQ and GPTQ baselines when used to quantize the multi￾modal LLM in the ultra-low bit regime. 1 2 3 4 5 6 7 8 Layer Depth 6.15 6.20 6.25 6.30 6.35 6.40 6.45 6.50 6.55 Entropy Entropy of Multimodal vs Text Activations Multimodal Activatio… view at source ↗
Figure 3
Figure 3. Figure 3: An overview of our LUQ: Layerwise Ultra-Low Bit Quantization. (i) Generation of multimodal calibration tokens by passing multimodal data through a CLIP model augmented with a con￾nector to align the modalities; (ii) Extraction of layerwise activations from the multimodal large language model (LLM); (iii) Entropy-based layer selection, where the entropy of activations is calculated to identify the layer mos… view at source ↗
Figure 4
Figure 4. Figure 4: Performance versus average bit-width for various post-training quantization methods using LLaVA 1.5 7B. (a) On the MME benchmark, LUQ significantly outperforms other methods for the LLaVA 1.5 model on the MME benchmark. (b) On the VQA v2 benchmark, LUQ maintains high accuracy for LLaVA 1.5 even at aggressive compression rates, whereas baseline methods show a sharp decline in performance. the larger 3-bit q… view at source ↗
Figure 5
Figure 5. Figure 5: Performance versus average bit-width for Qwen 2.5 VL 7B on (a) DocVQA and (b) ChartQA. LUQ provides a graceful performance trade-off as more layers are quantized to 1-bit. In contrast, standard PTQ methods like GPTQ and AWQ suffer a catastrophic performance collapse at sub-3- bit compression rates, highlighting LUQ’s superior robustness in the ultra-low bit regime. 9 [PITH_FULL_IMAGE:figures/full_fig_p009… view at source ↗
Figure 6
Figure 6. Figure 6: (a) Accuracy on TextVQA for Low Entropy First vs. High Entropy First quantization. Quantizing low activation entropy layers first preserves performance, while quantizing higher entropy layers first leads to a steep decline in performance. This trend holds for different numbers of layers quantized.(b) Impact of changing Mixing Ratio α on TextVQA performance. Even a small α > 0 improves performance, but gain… view at source ↗
Figure 7
Figure 7. Figure 7: Effect of mixed multimodal token vs. text only token calibration on LUQ-quantized models across different quantization levels on TextVQA. Mixed modal calibration outperforms text-only calibration across all quantized layers, with the performance gap widening as more layers are quantized. (α > 0) into the calibration data leads to a significant performance improvement compared to text-only calibration. Howe… view at source ↗
Figure 8
Figure 8. Figure 8: illustrates the core motivation for our proposed Layerwise Ultra-Low Bit Quantization (LUQ) strategy. The entropy values shown were calculated on activations collected using the input data and model settings described in Section 4.1. We observe that the Shannon entropy of intermediate activations is not uniform across the depth of the network. Instead, it exhibits significant variance, with certain layers … view at source ↗
Figure 9
Figure 9. Figure 9: Stability of the layer ordering π , defined in Equation 7 ( ascending order of layer activation entropy) with variation in number of clusters used for K Means clustering. We measure the stability of the layer ordering πk using the Kendall Tau distance To ensure a stable and reliable estimation of layer-wise entropy, we perform a rank stability analysis to select the number of clusters, K, for the K-means a… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Towards Joint Quantization and Token Pruning of Vision-Language Models

    cs.CV 2026-04 unverdicted novelty 6.0

    QUOTA jointly optimizes low-bit quantization and visual token pruning for VLMs by deriving pruning decisions from quantized operators, achieving 95.65% average performance retention with only 30% of visual tokens vers...

Reference graph

Works this paper leans on

44 extracted references · 14 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Why do some inputs break low-bit llm quantization?, 2025

    Ting-Yun Chang, Muru Zhang, Jesse Thomason, and Robin Jia. Why do some inputs break low-bit llm quantization?, 2025. URL https://arxiv.org/abs/2506.12044

    arXiv 2025

  5. [5]

    Quip: 2-bit quantization of large language models with guarantees

    Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M De Sa. Quip: 2-bit quantization of large language models with guarantees. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 4396--4429. Curran Associates, Inc., 2023. URL https://proceedings.neurips....

    2023

  6. [6]

    Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1

    Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830, 2016

    Pith/arXiv arXiv 2016

  7. [7]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 10088--10115. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/pape...

    2023

  8. [8]

    Gptq: Accurate post-training compression for generative pretrained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training compression for generative pretrained transformers. arXiv preprint arXiv:2210.17323, 2022

    Pith/arXiv arXiv 2022

  9. [9]

    Mme: a comprehensive evaluation benchmark for multimodal large language models

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, et al. Mme: a comprehensive evaluation benchmark for multimodal large language models. corr abs/2306.13394 (2023), 2023

    Pith/arXiv arXiv 2023

  10. [10]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 6904--6913, 2017

    2017

  11. [11]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    The unreasonable ineffectiveness of the deeper layers

    Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Dan Roberts. The unreasonable ineffectiveness of the deeper layers. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=ngmEcEer8a

    2025

  13. [13]

    Billm: Pushing the limit of post-training quantization for llms

    Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. Billm: Pushing the limit of post-training quantization for llms. arXiv preprint arXiv:2402.04291, 2024 a

    Pith/arXiv arXiv 2024

  14. [14]

    Slim-llm: Salience-driven mixed-precision quantization for large language models, 2024 b

    Wei Huang, Haotong Qin, Yangdong Liu, Yawei Li, Xianglong Liu, Luca Benini, Michele Magno, and Xiaojuan Qi. Slim-llm: Salience-driven mixed-precision quantization for large language models, 2024 b

    2024

  15. [15]

    An empirical study of llama3 quantization: From llms to mllms

    Wei Huang, Xingyu Zheng, Xudong Ma, Haotong Qin, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, and Michele Magno. An empirical study of llama3 quantization: From llms to mllms. Visual Intelligence, 2 0 (1): 0 36, 2024 c

    2024

  16. [16]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 6700--6709, 2019

    2019

  17. [17]

    Beware of calibration data for pruning large language models

    Yixin Ji, Yang Xiang, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, and Min Zhang. Beware of calibration data for pruning large language models. arXiv preprint arXiv:2410.17711, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    A new measure of rank correlation

    Maurice G Kendall. A new measure of rank correlation. Biometrika, 30 0 (1/2): 0 81--89, 1938. doi:10.1093/biomet/30.1-2.81

    doi:10.1093/biomet/30.1-2.81 1938

  19. [19]

    Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models

    Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 13355--13364, 2024

    2024

  20. [20]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355, 2023

    Pith/arXiv arXiv 2023

  21. [21]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6: 0 87--100, 2024

    2024

  22. [22]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 26296--26306, 2024 a

    2024

  23. [23]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an&nbsp;all-around player? In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part VI, pp.\ 216–233, Berlin, Hei...

    doi:10.1007/978-3-031-72658-3_13 2024

  24. [24]

    Llm-qat: Data-free quantization aware training for large language models

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. Llm-qat: Data-free quantization aware training for large language models. arXiv preprint arXiv:2305.17888, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    Logan, IV, Nelson F

    Robert L. Logan, IV, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Florence, Italy, July 2019. Association for Computational Linguistics

    2019

  26. [26]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=KUNzEQMWU7

    2024

  27. [27]

    Pv-tuning: Beyond straight-through estimation for extreme llm compression

    Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, and Peter Richtarik. Pv-tuning: Beyond straight-through estimation for extreme llm compression. Advances in Neural Information Processing Systems, 37: 0 5074--5121, 2024

    2024

  28. [28]

    C hart QA : A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. C hart QA : A benchmark for question answering about charts with visual and logical reasoning. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics: ACL 2022, pp.\ 2263--2279, Dublin, Ireland, May 2022. Associati...

    doi:10.18653/v1/2022.findings-acl.177 2022

  29. [29]

    Minesh Mathew, Dimosthenis Karatzas, and C.V. Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp.\ 2200--2209, January 2021

    2021

  30. [30]

    Up or down? adaptive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning, pp.\ 7197--7206. PMLR, 2020

    2020

  31. [31]

    Layer-wise quantization for quantized optimistic dual averaging

    Anh Duc Nguyen, Ilia Markov, Zhengqing Wu, Ali Ramezani-Kebrya, Kimon Antonakopoulos, Dan Alistarh, and Volkan Cevher. Layer-wise quantization for quantized optimistic dual averaging. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=J6LYjEOxbz

    2025

  32. [32]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.\ 8748--8763. PMLR, 2021

    2021

  33. [33]

    kneedle

    Ville Satopaa, Jeannie Albrecht, David Irwin, and Barath Raghavan. Finding a" kneedle" in a haystack: Detecting knee points in system behavior. In 2011 31st international conference on distributed computing systems workshops, pp.\ 166--171. IEEE, 2011

    2011

  34. [34]

    Omniquant: Omnidirectionally calibrated quantization for large language models

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantization for large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=8Wuvhh0LYW

    2024

  35. [35]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 8317--8326, 2019

    2019

  36. [36]

    Layer by layer: Uncovering hidden representations in language models

    Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Nikul Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=WGXb7UdvTX

    2025

  37. [37]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Gemma 3 technical report

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram \'e , Morgane Rivi \`e re, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Understanding deep representation learning via layerwise feature compression and discrimination, 2025

    Peng Wang, Xiao Li, Can Yaras, Zhihui Zhu, Laura Balzano, Wei Hu, and Qing Qu. Understanding deep representation learning via layerwise feature compression and discrimination, 2025. URL https://arxiv.org/abs/2311.02960

    Pith/arXiv arXiv 2025

  40. [40]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp.\ 38087--38099. PMLR, 2023

    2023

  41. [41]

    Rptq: Reorder-based post-training quantization for large language models, 2023

    Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, and Bingzhe Wu. Rptq: Reorder-based post-training quantization for large language models, 2023

    2023

  42. [42]

    PB - LLM : Partially binarized large language models

    Zhihang Yuan, Yuzhang Shang, and Zhen Dong. PB - LLM : Partially binarized large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=BifeBRhikU

    2024

  43. [43]

    A survey on model compression for large language models

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics, 12: 0 1556--1577, 2024

    2024

  44. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...