arxiv: 2603.16105 · v2 · submitted 2026-03-17 · 💻 cs.CL · cs.AI

Recognition: no theorem link

Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantization

Francesco Pio Monaco , Elia Cunegatti , Flavio Vella , Giovanni Iacca

Authors on Pith no claims yet

Pith reviewed 2026-05-15 10:35 UTC · model grok-4.3

classification 💻 cs.CL cs.AI

keywords data curationmodel compressionpruningquantizationZipfian distributionlexical diversitycalibration datalarge language models

0 comments

The pith

ZipCal selects calibration data for LLM pruning and quantization by maximizing lexical diversity through Zipfian power laws, matching perplexity-based methods while running about 240 times faster.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that calibration data for post-training compression can be chosen effectively by focusing on intrinsic data properties rather than model-specific signals like perplexity. It introduces ZipCal as a strategy that curates datasets to maximize lexical diversity according to Zipfian distributions. This model-agnostic approach outperforms uniform random sampling on pruning tasks and matches state-of-the-art performance while scaling linearly. A sympathetic reader would care because it removes a major computational bottleneck in compressing large models without sacrificing downstream task preservation.

Core claim

ZipCal is a model-agnostic data curation strategy that maximizes lexical diversity based on Zipfian power laws. Experiments show it consistently outperforms uniform random sampling across pruning benchmarks and performs on par with a perplexity-dependent state-of-the-art method in preserving downstream performance for both pruning and quantization, while achieving an average speedup of approximately 240 times due to its tractable linear complexity.

What carries the argument

ZipCal, a curation procedure that selects calibration subsets to maximize lexical diversity according to Zipfian power laws on token frequencies.

If this is right

Calibration data selection for compression no longer requires running the target model to compute perplexity scores.
Linear-complexity curation becomes feasible for very large datasets where perplexity evaluation would be prohibitive.
The same frequency-based selection principle could extend to other post-training steps such as knowledge distillation or continued pre-training.
Pruning and quantization pipelines gain a low-cost, repeatable data-preparation step that remains independent of model architecture.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Lexical frequency patterns alone appear to encode enough structural information to substitute for model-internal signals in calibration.
The approach may generalize to other model compression techniques beyond pruning and quantization if they also rely on representative calibration sets.
For extremely large models, replacing perplexity computation with ZipCal could reduce the overall carbon and compute cost of compression workflows.

Load-bearing premise

Maximizing lexical diversity according to Zipfian power laws in the calibration data is sufficient to preserve downstream performance during pruning and quantization without any model-specific signals.

What would settle it

A direct comparison on a large model where a ZipCal-curated calibration set produces measurably lower downstream accuracy or higher perplexity than a perplexity-selected set on the same benchmarks would disprove the claim.

Figures

Figures reproduced from arXiv: 2603.16105 by Elia Cunegatti, Flavio Vella, Francesco Pio Monaco, Giovanni Iacca.

**Figure 2.** Figure 2: Running time (log-scale) for calibration data [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Figure 3: Effect of calibration data context length on [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

**Figure 5.** Figure 5: Token frequency distribution of the original datasets and the random, COLA, and [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗

**Figure 6.** Figure 6: Token frequency distribution of the original datasets and the random, COLA, and [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗

read the original abstract

Post-training model compression is essential for enhancing the portability of Large Language Models (LLMs) while preserving their performance. While several compression approaches have been proposed, less emphasis has been placed on selecting the most suitable set of data (the so-called \emph{calibration data}) for finding the compressed model configuration. The choice of calibration data is a critical step in preserving model capabilities both intra- and inter-tasks. In this work, we address the challenge of identifying high-performance calibration sets for both pruning and quantization by analyzing intrinsic data properties rather than model-specific signals. We introduce \texttt{\textbf{ZipCal}}, a model-agnostic data curation strategy that maximizes lexical diversity based on Zipfian power laws. Experiments demonstrate that our method consistently outperforms standard uniform random sampling across various pruning benchmarks. Notably, it also performs on par, in terms of downstream performance, with a state-of-the-art method that relies on model perplexity. The latter becomes prohibitively expensive at large-scale models and datasets, while \texttt{\textbf{ZipCal}} is on average $\sim$240$\times$ faster due to its tractable linear complexity\footnote{We make the code and the experiments available at https://github.com/FrancescoMonaco/ZipCal.}.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

ZipCal gives a fast model-free way to pick calibration data for LLM pruning and quantization by maximizing lexical diversity from word frequencies, and the experiments show it matches perplexity methods while running far quicker.

read the letter

ZipCal is a model-agnostic method that picks calibration data for pruning and quantizing LLMs by maximizing lexical diversity according to Zipfian frequencies. The experiments show it beats random sampling and matches perplexity-based selection at a fraction of the time. The main contribution is shifting away from model-dependent signals like perplexity to intrinsic data properties. This makes the curation step fast and scalable, with linear complexity instead of requiring forward passes on the model. They demonstrate consistent outperformance on pruning benchmarks and parity on downstream tasks for both pruning and quantization across several models. The speed advantage of around 240 times is a direct result of avoiding model inference during selection. The paper does a good job with the evaluation by including cross-model transfer experiments to validate the agnostic property. Code and experiments are available, which helps with reproducibility. The approach builds on classical observations about word distributions without adding unnecessary complexity. One soft spot is the reliance on lexical diversity as the primary criterion. While it works well in the tested cases, there could be situations where other data characteristics matter more for preserving model performance after compression. The results don't show major failures, but the paper could have explored more diverse datasets to strengthen that point. This paper is aimed at researchers and engineers working on efficient deployment of large models who need practical tools for compression pipelines. It deserves a serious referee because the central claims are supported by concrete experiments and the method offers a clear efficiency gain without sacrificing much accuracy. I'd recommend putting it through peer review.

Referee Report

0 major / 2 minor

Summary. The paper introduces ZipCal, a model-agnostic calibration data curation method for LLM pruning and quantization that selects subsets maximizing lexical diversity according to Zipfian power-law token frequencies. It claims this approach consistently outperforms uniform random sampling on pruning benchmarks, achieves parity with perplexity-based state-of-the-art selection in downstream performance, and runs ~240× faster due to O(n) complexity, with code released for reproducibility.

Significance. If the experimental claims hold, the result is significant because it decouples calibration-set selection from model-specific signals (e.g., perplexity), offering a fast, scalable alternative that preserves performance in both pruning and quantization. The linear-time procedure and cross-model transfer experiments, if substantiated by the tables, would be a practical contribution for large-scale compression pipelines.

minor comments (2)

[Abstract] Abstract: the claim of 'consistent outperformance' and 'parity' is stated without any numerical deltas, error bars, or dataset identifiers; adding one-sentence quantitative highlights would improve immediate readability.
[Method] The manuscript should clarify in §3 or §4 whether the Zipfian frequency estimation uses the full corpus or a fixed vocabulary cutoff, as this choice directly affects the claimed linear complexity.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary of our work and the recommendation for minor revision. The assessment correctly identifies the core contribution of ZipCal as a fast, model-agnostic calibration method based on Zipfian lexical diversity. No specific major comments were raised in the report.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper derives ZipCal directly from the established Zipfian frequency distribution of tokens in the calibration corpus, an external empirical regularity independent of any model outputs, fitted parameters, or target compression metrics. Selection proceeds via linear-time counting of lexical frequencies followed by diversity maximization under the power-law assumption; no equation redefines a fitted quantity as a prediction, no uniqueness theorem is imported from self-citations, and no ansatz is smuggled via prior work by the same authors. Downstream performance claims rest on explicit cross-model benchmarks against uniform sampling and perplexity baselines rather than on any self-referential reduction. The derivation chain therefore remains self-contained against external data properties and independent validation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that lexical diversity measured via Zipfian statistics serves as a reliable proxy for calibration quality in compression tasks; no free parameters or invented entities are introduced in the abstract description.

axioms (1)

domain assumption Lexical diversity following Zipfian power laws in calibration data correlates with preserved model performance after pruning and quantization
Invoked as the basis for the curation strategy without further justification or derivation in the provided abstract.

pith-pipeline@v0.9.0 · 5533 in / 1342 out tokens · 48418 ms · 2026-05-15T10:35:40.476441+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Coverage-Based Calibration for Post-Training Quantization via Weighted Set Cover over Outlier Channels
cs.LG 2026-04 conditional novelty 7.0

COVERCAL selects PTQ calibration samples via weighted set cover over outlier channels, with a stylized clipping model showing missed coverage upper-bounds surrogate loss, yielding gains over random and other baselines...

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · cited by 1 Pith paper · 2 internal anchors

[1]

InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online

On the Cross-lingual Transferability of Mono- lingual Representations. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics. Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman

work page
[2]

InThe Twelfth In- ternational Conference on Learning Representations

SliceGPT: Compress Large Language Models by Deleting Rows and Columns. InThe Twelfth In- ternational Conference on Learning Representations. Abhinav Bandari, Lu Yin, Cheng-Yu Hsieh, Ajay Ku- mar Jaiswal, Tianlong Chen, Li Shen, Ranjay Kr- ishna, and Shiwei Liu. 2024. Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning.a...

work page arXiv 2024
[3]

Training Verifiers to Solve Math Word Problems

Training Verifiers to Solve Math Word Prob- lems.arXiv preprint. ArXiv:2110.14168 [cs]. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating Cross- lingual Sentence Representations. InProceedings of the 2018 Conference on Empirical Methods in Nat- ural Language Processin...

work page internal anchor Pith review Pith/arXiv arXiv 2018
[4]

Bowei He, Lihao Yin, Huiling Zhen, Shuqi Liu, Han Wu, Xiaokun Zhang, Mingxuan Yuan, and Chen Ma

Learning both weights and connections for efficient neural network.Advances in neural infor- mation processing systems, 28. Bowei He, Lihao Yin, Huiling Zhen, Shuqi Liu, Han Wu, Xiaokun Zhang, Mingxuan Yuan, and Chen Ma

work page
[5]

arXiv preprint

Preserving LLM Capabilities through Calibra- tion Data Curation: From Analysis to Optimization. arXiv preprint. ArXiv:2510.10618 [cs]. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset.arXiv preprint. ArXiv:2103.03874 [cs]....

work page arXiv 2021
[6]

Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Ron Banner, Alex M

PMLR. Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Ron Banner, Alex M. Bronstein, and Avi Mendelson. 2021. Loss aware post-training quan- tization.Machine Learning, 110(11):3245–3262. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversar- ial NLI: A New Benchmark for Natural Language Under...

work page 2021
[7]

ArXiv:2206.09557 [cs]

LUT-GEMM: Quantized Matrix Multiplica- tion based on LUTs for Efficient Inference in Large- Scale Generative Language Models.arXiv preprint. ArXiv:2206.09557 [cs]. Arkil Patel, Satwik Bhattamishra, and Navin Goyal

work page arXiv
[8]

Qian, C., Liu, D., Wen, H., Bai, Z., Liu, Y ., and Shao, J

Are NLP Models really able to Solve Simple Math Word Problems?arXiv preprint. ArXiv:2103.07191 [cs]. Steven T. Piantadosi. 2014. Zipf’s word frequency law in natural language: A critical review and future di- rections.Psychonomic bulletin & review, 21(5):1112– 1130. Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vuli´c, and Anna Korho...

work page arXiv 2014
[9]

2SSP: A Two-Stage Framework for Structured Pruning of LLMs.Transactions on Machine Learn- ing Research. Shivalika Singh, Angelika Romanou, Clémentine Four- rier, David Ifeoluwa Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchi- sio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Sebastian Ruder, Wei-Yin Ko, Anto...

work page
[10]

InProceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18761–18799, Vi- enna, Austria

Global MMLU: Understanding and Address- ing Cultural and Linguistic Biases in Multilingual Evaluation. InProceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18761–18799, Vi- enna, Austria. Association for Computational Lin- guistics. Lu Sun and Jun Sakuma. 2026. Learning Semi- Structured...

work page arXiv 2026
[11]

HellaSwag: Can a Machine Really Finish Your Sentence?

Wanda++: Pruning Large Language Models via Regional Gradients. InFindings of the Associa- tion for Computational Linguistics: ACL 2025, pages 4321–4333, Vienna, Austria. Association for Compu- tational Linguistics. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a Machine Really Finish Your Sentence?arXiv prepr...

work page internal anchor Pith review Pith/arXiv arXiv 2025
[12]

InInternational Conference on Learning Representations

Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch. InInternational Conference on Learning Representations. George Kingsley Zipf. 2013. Relative Frequency, Ab- breviation, and Semantic Change. InSelected Studies of the Principle of Relative Frequency in Language, pages i–iv. Harvard University Press. 12 A Detailed Algorithms and Proo...

work page arXiv 2013