Float8@2bits: Entropy Coding Enables Data-Free Model Compression

Martin Genzel; Mattes Mollenhauer; Patrick Putzky; Sebastian Schulze; Stefan Dietzel; Thomas Wollmann

REVIEW 4 major objections 4 minor 12 references

Float8@2bits: Entropy Coding Enables Data-Free Model Compression

T0 review · 4 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EntQuant compresses models to about two bits per weight, without calibration data, while retaining most task accuracy.

desk verdict EntQuant makes a credible case that entropy-constrained per-channel scaling can push data-free LLM compression to ~2 bits, but the SOTA claim is over-strong and the entropy-proxy justification is thinner than the experiments. read the letter →

arxiv 2601.22787 v2 pith:PKEFV6LJ submitted 2026-01-30 cs.LG

Patrick Putzky , Martin Genzel , Mattes Mollenhauer , Sebastian Schulze , Thomas Wollmann , Stefan Dietzel This is my paper · ORCID

classification cs.LG

keywords post-trainingquantizationentropycodingmodelcompressiondata-freelargelanguagemodelsasymmetricnumeralsystemsrate-distortionoptimizationlow-bitinference

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the well-known collapse of data-free quantization below 4 bits is not fundamental: it comes from forcing weights into rigid 2-bit or 3-bit codebooks. EntQuant keeps weights in an 8-bit format but optimizes per-channel scales so that the weight distribution becomes very predictable, then applies lossless entropy coding to store the weights at about 2 bits per parameter. This decouples storage cost from numerical precision: inference still runs at 8-bit precision, so the model keeps far more expressiveness than a true 2-bit model. On 16 models, including instruction-tuned ones, the method stays within a few points of base accuracy at 2-3 bits, while compressing a 70B model in under 30 minutes without any calibration data or recovery training. A sympathetic reader would take the contribution to be a demonstration that entropy coding, rather than ever-finer quantization, is the practical route past the 4-bit barrier.

What carries the argument

The central mechanism is the rate-distortion objective: minimize over per-channel scales the reconstruction error plus λ times the ℓ1 norm of the Float8-quantized weights, solved layer-wise with L-BFGS and a straight-through gradient estimator. The ℓ1 regularizer is the paper's surrogate for entropy, motivated by the maximum-entropy bound H(X) ≤ λ E|X| + log Z(λ) and entropy subadditivity; the paper notes this worst-case bound still carries an ambient-dimension term. Once scales are fixed, the integer-valued Float8 tensor is streamed through a parallel GPU implementation of asymmetric numeral systems (ANS), a lossless coder that approaches the optimal code length with fast multiply-shift ope

What would settle it

Run EntQuant with the same λ grid on a model family outside the 16 tested, or on a mixture-of-experts model, and verify two things: whether per-layer optimized entropy actually reaches the claimed 2.1 bits, and whether the reported 2-bit accuracy on a standard benchmark is reproduced. A failure of either would show the ℓ1-as-entropy proxy or the claimed data-free universality breaks.

Watch

Extended reading notes

Core claim

EntQuant's central claim is that arithmetic precision and storage cost are separable, and exploiting that separation dissolves the apparent 2-bit wall. Starting from Float8 weights, the method tunes only one scale per output channel to minimize a weighted sum of reconstruction error and the ℓ1 norm of the quantized weights; the ℓ1 norm acts as a differentiable stand-in for empirical entropy. The optimized weights are flattened and losslessly compressed with asymmetric numeral systems. Because the empirical distribution concentrates on a few values, the bitstream costs about 2.1 bits per parameter even though the model is evaluated with an 8-bit dequantized matrix. The paper reports that this

Load-bearing premise

The load-bearing premise is that minimizing the ℓ1 norm of per-channel-scaled Float8 weights reliably pulls the empirical entropy down to about 2 bits without destroying the model's function—an empirical regularity observed across the tested models, but one whose theoretical bound (Section B.2) is too loose to guarantee.

Editorial extensions

If this is right

Below 4 bits, data-free compression no longer necessarily means functional collapse: at 2.1 bits EntQuant keeps 70B-class models within a few points of base accuracy, where fixed 2-bit methods degrade to near chance.
Instruction-tuned and reasoning models, which are hard to recover with calibration or fine-tuning, can be compressed to 2-3 bits without touching their data; on several instruct models the average over GSM8K, GPQA, MMLU, and IFEval stays within single-digit points of base.
Compression time drops from hours or days to minutes: a 70B parameter model is compressed in under 30 minutes with no calibration set.
Memory-bound deployment benefits directly: a 70B model fits at 3 bits or lower in a 32 GiB consumer GPU for some inference loads, and the inference slowdown is modest—roughly 1.5-2x the unquantized baseline.
The mostly model-independent, log-linear mapping from regularization strength λ to target entropy means one schedule of λ values suffices across model families, simplifying adoption for new model releases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same decoupling could extend beyond Transformers: because the optimization needs only weights, one could apply EntQuant-style entropy optimization to vision models or diffusion backbones without retraining or calibration.
EntQuant's entropy reduction coincides with sparsification, so the method doubles as an unsupervised soft-pruner; combining it with explicit sparse kernels or low-rank structure might push useful bit-rates toward 1.5 bits or below, though the paper does not explore this.
If entropy decoding stays compute-bound while GPU compute outpaces memory-bandwidth growth, storing models as compressed bitstreams may become the preferred deployment format itself, not merely a storage add-on.
A concrete testable extension is whether the λ-to-entropy mapping remains log-linear for mixture-of-experts models, where routing layers have very different weight statistics; the paper lists MoE evaluation as future work.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things up front. First, the core result is real: optimizing per-channel scales with an ℓ1 penalty, then ANS-coding the Float8 weights, does get competitive perplexity and zero-shot accuracy around 2.1 bits per parameter on a wide range of LLMs — without calibration data. Second, the paper oversells itself in the abstract and in Section 5: QuIP# beats EntQuant at 2-bit on LLaMA-2 70B (6.12 vs 6.47 C4, 70.9 vs 68.6 LM Eval), so “matching the performance of data-dependent methods” is only true for some baselines, not the strongest one. That is an overclaim, not a fatal flaw.

The genuinely new bit is the combination: prior entropy-coding work for LLMs (DFloat11, NeuZip) compresses the existing BFloat16 representation without changing the weights, while EntQuant actively optimizes weights to lower their entropy. The empirical work is broad — 480 runs, 16 models, including instruction-tuned models on GSM8K, IFEval, GPQA — and the practical speed (under 30 minutes for 70B, modest inference overhead) makes it genuinely useful. The method is simple enough to reproduce, and the code is promised.

Soft spots, in proportion. The biggest is the theoretical story in Sec. B.2: the bound H(Wq) ≤ λE||Wq||1 + MN·logZ(λ) is vacuous at realistic matrix sizes because the MN term dominates. The paper acknowledges this implicitly by calling it a “basic argument.” That is honest, but it means the ℓ1 proxy is justified purely by Fig. A.1 and Fig. B.1. Those figures show a clean, model-independent λ-to-entropy map across 8 LLaMA models, which is actually fairly convincing empirical evidence. Still, there is no per-layer entropy breakdown, no test on MoE or ViT architectures, and no ablation against a direct entropy surrogate. If some layer types resist entropy reduction under channel-wise scaling, the claimed bit-rate could hide a long tail. That is the main thing I would want addressed before trusting the method beyond the models tested.

Smaller issues: no error bars anywhere; the nearest entropy-coding baselines (DFloat11, NeuZip) are never compared numerically; and the abstract says “less than 10 minutes” while Section 3.4 says “less than 30 minutes.” These are fixable and do not undermine the central mechanism.

For a reader, this is a solid, practically oriented paper. I would send it to review — the method is useful, the evaluation is broad, and the soft spots are addressable. A serious referee should push on the generalization of the entropy proxy and on toning down the SOTA claim.

Referee Report

4 major / 4 minor

Summary. The paper introduces EntQuant, a data-free post-training compression method for LLMs. Quantized weights are kept in an 8-bit base format (Float8 or Int8), but per-channel scale parameters are optimized with L-BFGS to minimize a weighted sum of relative ℓ1 reconstruction error and ℓ1 regularization, which serves as a proxy for entropy. The resulting weight symbols are then losslessly compressed with ANS, and inference decompresses each transformer block on the fly. The authors report effective rates around 2.1 bits per parameter on 16 LLM families, including instruction-tuned models, with perplexity and LM-Eval accuracy comparable to or better than data-free baselines such as HQQ and NF4, and close to several calibration- or fine-tuning-based methods. The paper emphasizes that storage cost is decoupled from representational precision, because inference uses the high-precision dequantized weights even though the stored bitstream costs far fewer than 8 bits per parameter.

Significance. If the empirical findings hold, EntQuant is a valuable contribution: it is a simple, fast, calibration-free route to extreme compression, and the idea of optimizing an entropy proxy before applying ANS is natural and likely to be adopted. The evaluation is unusually broad for this area — over 480 runs across 16 models, including 123B instruction-tuned models — and the code is promised publicly. The main risks are the strength of the wording: the method is not strictly state-of-the-art against the best data-dependent 2-bit methods (QuIP# is better in Table 3b), and the theoretical justification in Section B.2 does not quantitatively explain the observed 2-bit rates. These are fixable by moderation and additional analysis, but they affect the central claims as currently written.

major comments (4)

[Abstract, §3.2, Table 3(b)] The claim that EntQuant 'matches the performance of data-dependent methods' is stronger than the reported numbers. At 2 bits on LLaMA-2 70B, QuIP# achieves C4 6.12 / WikiText-2 4.16 / LM Eval 70.9, while EntQuant 2.1 achieves 6.47 / 4.52 / 68.6; at 3 bits QuIP# also leads in all three metrics (5.67/3.56/72.1 vs 5.74/3.62/71.7). The paper can legitimately claim competitive data-free performance, but 'state-of-the-art' and 'matching' should be tempered. Since the contribution is the data-free trade-off, this is fixable by rephrasing, but it changes the advertised claim.
[Section B.2 (Eq. B.1-B.3)] The appendix's theoretical support for the ℓ1 proxy is not quantitative. The bound H(Wq) ≤ λE||Wq||1 + MN·log Z(λ) is a generic maximum-entropy inequality. For the large λ values used in Fig. A.1 (up to 256), log Z(λ) is extremely small, so the MN term is not necessarily the dominant obstruction; however the first term can only be small if λE||Wq||1 is verified, and the paper reports no values of ||Wq||1, δ, or per-layer entropies. The appendix thus does not explain why the observed 2.1-bit rates are attainable; it only says the scalar bound is sharp for the Gibbs family. Either remove the claim of theoretical evidence or add measurements showing the bound's premises hold at the operating point.
[§3.3, Table E.1, Fig. 1] The 'functional extreme compression' claim does not hold uniformly across instruction-tuned models. At ~2.1–2.2 bits, LLaMA-3.1 8B Instruct collapses on GSM8K (84.7 → 14.6) and Qwen3 8B drops from 81.4 → 28.7 on GSM8K and 72.0 → 30.3 on MMLU, whereas OLMo 3.1 32B and Mistral Large retain much more. Since the abstract presents data-free extreme compression as broadly applicable, the paper should qualify the claim by model size/architecture or provide a criterion for when functional preservation is expected.
[§A.2, Table G.1] The 'purely data-free' claim would be strengthened by reporting how the super-weight thresholds (LLaMA-1: 50; Qwen3: 200; others: ∞) were selected. If these were chosen after inspecting validation performance on the same benchmarks, the comparison is not fully blind. Additionally, the lack of per-layer entropy statistics and the absence of non-dense/transformer architectures (MoE, vision transformers) leave the model-agnostic claim resting on the 16 dense LLMs tested; the Limitations section lists MoE as future work, but the abstract's universality language should be consistent.

minor comments (4)

[Abstract vs. §5, Table 3(a)] Compression time is given as 'less than 10 minutes' in the abstract, '<30min (H100)' in Table 3(a), and 'less than 30 minutes' in §5. Please reconcile these numbers.
[§3, references] C4 is cited to Ansel et al.; the canonical citation is Raffel et al. (2020). Please correct the reference.
[Tables C.1–D.1] Many baseline numbers in Table D.1 are taken from different papers and evaluation harnesses. Please state explicitly which LM-Eval version and settings were used for each source, and where feasible report variance or seeds. The absence of any error/standard-deviation information across 480 runs makes it hard to judge small gaps.
[§2.2, Table 2] The reported bit-rates (2.1, 1.7, etc.) should be defined more precisely in the main text: clarify that the effective rate includes the ANS metadata and scale overhead for all runs, not only for the super-weight exclusions discussed in §3.5.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: EntQuant's bit-rates are measured post-optimization and its accuracy results are validated against external benchmarks, with no load-bearing self-citation.

full rationale

The paper's derivation chain is self-contained and does not reduce to its inputs. EntQuant optimizes per-channel scales by minimizing d(W, W_hat) + lambda * ||Wq||_1 (Eq. 3), where the empirical entropy H(Wq) (Eq. 2) is not part of the objective; the reported bit-rates are measured after optimization via ANS coding, and lambda is chosen only to hit a target entropy, not to force the evaluated accuracy. The accuracy/perplexity numbers come from external benchmarks (C4, WikiText-2, LM Eval, GSM8K, IFEval, etc.) and are compared against external baselines (HQQ, NF4, GPTQ, QuIP#, EfficientQAT, etc.), so no 'prediction' is a renamed fit. The theoretical bound in Sec. B.2 is a worst-case maximum-entropy inequality used only as motivation for the l1 proxy; although the MN*logZ(lambda) term makes the bound loose, this is a correctness/robustness concern, not circularity. There are no load-bearing self-citations: cited prior work (Yu et al., Zhang et al., Duda, Chen et al.) is independent of the present authors, and no uniqueness theorem from the authors is invoked. The stated limitations (no MoE evaluation, no per-layer entropy distributions, no direct-entropy ablation) concern empirical generality rather than construction-by-definition. Therefore, no circular step can be exhibited.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a small number of free hyperparameters (λ, super-weight thresholds, L-BFGS learning rate) and on standard information-theoretic axioms. No new entities are introduced. The λ-to-entropy mapping is empirically stable across LLaMA models but not guaranteed for arbitrary architectures; the super-weight thresholds are manually chosen per model family. The maximum-entropy bound (B.2) is used only as motivation.

free parameters (3)

λ regularization = not reported (mapped per target entropy via Fig. A.1)
Controls trade-off between distortion and entropy in Eq. (3); chosen per target bit-rate grid.
super-weight thresholds = 50 (LLaMA-1, LLaMA-3.1 8B), 200 (Qwen3), ∞ (others)
Manually selected per model family in Sec. A.2 to decide which layers to exclude; affects results in ablation.
L-BFGS learning rate = 0.25 for λ>30 else 1.0
Set by hand in Sec. A.1; authors state L-BFGS is robust to this.

assumptions (5)

standard math Shannon's source coding theorem: optimal lossless code length is the entropy of the source.
Used in Sec. 2.1 to motivate ANS and in Eq. (2) for empirical entropy.
standard math Maximum entropy principle: H(X) ≤ λE|X| + log Z(λ) for all λ>0 (Polyanskiy & Wu Example 5.2).
Used in Sec. B.2 to argue ℓ1 reduces entropy; a worst-case bound.
domain assumption Straight-through estimator provides usable gradients through the quantization function.
Assumed in Sec. 2.2 to enable gradient-based optimization of scales; standard practice but not exact.
domain assumption Weights in a layer are i.i.d. for entropy coding (empirical distribution factorizes).
Eq. (2) assumes product distribution; ANS operates on symbol frequencies.
domain assumption LLM weights contain sufficient redundancy that entropy-constrained rescaling does not destroy task performance.
Core empirical premise; supported by Fig. B.1 but no theoretical guarantee.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Float8@2bits: Entropy Coding Enables Data-Free Model Compression." pith.science (2026). https://pith.science/paper/PKEFV6LJ

@misc{pith2026260122787,
  author       = {Pith},
  title        = {Pith review of: Float8@2bits: Entropy Coding Enables Data-Free Model Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKEFV6LJ}},
  note         = {Machine review of arXiv:2601.22787}
}

read the original abstract

Post-training compression is currently divided into two contrasting regimes. On the one hand, fast, data-free, and model-agnostic methods (e.g., NF4 or HQQ) offer maximum accessibility but suffer from functional collapse at extreme bit-rates below 4 bits. On the other hand, techniques leveraging calibration data or extensive recovery training achieve superior fidelity but impose high computational constraints and face uncertain robustness under data distribution shifts. We introduce EntQuant, a framework that unites the advantages of these distinct paradigms. By matching the performance of data-dependent methods with the speed and universality of data-free techniques, EntQuant enables practical utility in the extreme compression regime. Our method decouples numerical precision from storage cost via entropy coding, compressing a 70B parameter model in less than 10 minutes. We demonstrate that EntQuant does not only achieve state-of-the-art results on standard evaluation sets and models, but also retains functional performance on more complex benchmarks with instruction-tuned models, all at modest inference overhead.

Figures

Figures reproduced from arXiv: 2601.22787 by the authors.

**Figure 1.** EntQuant compresses instruction-tuned models without data, performing well on several advanced benchmarks. Numbers above the size axis indicate effective bits per parameter. standard for model compression, dramatically reducing storage requirements and memory bandwidth bottlenecks while enabling faster inference (Frantar et al., 2023; Lin et al., 2024; Frantar et al., 2025). While 8-bit quantization incurs negligib… view at source ↗

**Figure 2.** Illustration of 4-bit weight encoding with EntQuant, compared to fixed bit-width quantization. Boxes illustrate weight matrices at different representations with weight histograms above the weight matrix. Note that the number of colors and histogram bins is reduced for illustrative purposes. Weights optimized with EntQuant have more diverse parameters compared to fixed bit-width representations. With entropy coding,… view at source ↗

**Figure 4.** Memory-perplexity trade-off on C4 for LLaMA-2 7B, 13B, 70B. EntQuant spans a smooth Pareto front enabling finegrained compression-performance trade-offs. Surface areas of dots are proportional to the bit-rate of each model. Float8 is entropyencoded as well, leading to approximately 6.5 bits per parameter. 3. Experiments With over 480 runs, we have evaluated EntQuant on 16 different open-weight LLMs, including LLaM… view at source ↗

Figures from the paper (2 more)

**Figure 3.** Figure 3: Visualization of EntQuant’s inference pipeline. 2 5 10 20 50 100 5 6 7 8 9 10 11 Base Float8 HQQ EntQuant Model Size (GiB) Perplexity (C4) 13B 70B 7B [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]

**Figure 6.** Figure 6: demonstrates that precisely excluding layers containing super weights recovers expected performance for Int8. Importantly, this improvement extends beyond EntQuant: Table G.1 shows that accounting for super weights benefits NF4 and HQQ as well. Note that excluding cer…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 4 linked inside Pith

[5]

Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D

URL https://proceedings.mlr.press/ v235/egiazarian24a.html. Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. GPTQ: Accurate Quantization for Generative Pre-trained Transformers. InThe Eleventh International Confer- ence on Learning Representations, 2023. URL https: //openreview.net/forum?id=tcbBPnfwxS. Frantar, E., Castro, R. L., Chen, J., Hoefler...

arXiv 2023
[6]

Give Me BF16 or Give Me Death

URL https://proceedings.mlr.press/ v235/kim24f.html. Krizhevsky, A., Sutskever, I., and Hinton, G. E. ImageNet Classification with Deep Convolutional Neural Networks. In Pereira, F., Burges, C., Bottou, L., and Weinberger, K. (eds.),Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. Kurtic, E., Marques, A. N., Pan...

2012
[7]

ISBN 979-8-89176-251-0

Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long

doi:10.18653/v1/2025.acl-long 2025
[11]

Wallace, G

URL https://proceedings.mlr.press/ v235/tseng24a.html. Wallace, G. K. The JPEG still picture compression stan- dard.Communications of the ACM, 34(4):30–44, 1991. Publisher: ACM New York, NY , USA. Wolf, T., Debut, L., Sanh, V ., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., ...

1991
[13]

soft pruning

URL https://github.com/PAA-NCIC/ hipANS. Yu, M., Wang, D., Shan, Q., Reed, C. J., and Wan, A. The Super Weight in Large Language Models.arXiv e-prints, art. arXiv:2411.07191, November 2024. doi: 10.48550/arXiv.2411.07191. Zellers, R., Holtzman, A., Bisk, Y ., Farhadi, A., and Choi, Y . HellaSwag: Can a Machine Really Finish Your Sen- tence? In Korhonen, A...

arXiv doi:10.48550/arxiv.2411.07191 2024
[498]

acl-long.498/

URL https://aclanthology.org/2025. acl-long.498/. Chou, P., Lookabaugh, T., and Gray, R. Entropy- constrained vector quantization.IEEE Transactions on Acoustics, Speech, and Signal Processing, 37(1):31– 42, January 1989. ISSN 0096-3518. doi: 10.1109/ 29.17498. URL http://ieeexplore.ieee.org/ document/17498/. Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, ...

arXiv doi:10.48550/arxiv.1803.05457 2025
[1304]

acl-long.1304/

URL https://aclanthology.org/2025. acl-long.1304/. Lee, J., Park, S., Kwon, J., Oh, J., and Kwon, Y . Ex- ploring the trade-offs: quantization methods, task dif- ficulty, and model size in large language models from edge to giant. InProceedings of the thirty-fourth in- ternational joint conference on artificial intelligence, Ij- cai ’25, Montreal, Canada,...

doi:10.24963/ijcai.2025/902 2025
[1948]

doi: 10.1002/j.1538-7305.1948

ISSN 00058580. doi: 10.1002/j.1538-7305.1948. tb01338.x. URL https://ieeexplore.ieee. org/document/6773024. Shao, W., Chen, M., Zhang, Z., Xu, P., Zhao, L., Li, Z., Zhang, K., Gao, P., Qiao, Y ., and Luo, P. Om- niQuant: Omnidirectionally calibrated quantization for large language models. InThe twelfth international con- ference on learning representation...

arXiv 1948

Show all 12 references

[2020]

doi: 10.18653/v1/2020.emnlp-demos.6

Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-demos.6. URL https:// aclanthology.org/2020.emnlp-demos.6/. Wu, H., Judd, P., Zhang, X., Isaev, M., and Micikevicius, P. Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation...

2020 doi arXiv
[2021]

Number: 8878 Series: Request for comments tex.howpublished: RFC 8878 tex.pagetotal: 45

URL https://www.rfc-editor.org/ info/rfc8878. Number: 8878 Series: Request for comments tex.howpublished: RFC 8878 tex.pagetotal: 45. Cover, T. M. and Thomas, J. A.Elements of Information Theory. John Wiley & Sons, 2nd edition, April 2005. ISBN 9780471241959. doi: 10.1002/0471...

2005 doi
[2024]

Dao, T., Fu, D., Ermon, S., Rudra, A., and R´e, C

URL https://openreview.net/forum? id=mZn2Xyh9Ec. Dao, T., Fu, D., Ermon, S., Rudra, A., and R´e, C. FlashAt- tention: Fast and Memory-Efficient Exact Attention with IO-Awareness. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.),Advances in Neur...

2022 doi arXiv
[4646]

URL https://doi

doi: 10.1007/BF01589116. URL https://doi. org/10.1007/BF01589116. Merity, S., Xiong, C., Bradbury, J., and Socher, R. Pointer Sentinel Mixture Models. InInternational Conference on Learning Representations, 2017. URL https:// openreview.net/forum?id=Byj72udxe. Mihaylov, T., Cl...

2017 doi

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [5]

Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D

URL https://proceedings.mlr.press/ v235/egiazarian24a.html. Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. GPTQ: Accurate Quantization for Generative Pre-trained Transformers. InThe Eleventh International Confer- ence on Learning Representations, 2023. URL https: //openreview.net/forum?id=tcbBPnfwxS. Frantar, E., Castro, R. L., Chen, J., Hoefler...

arXiv 2023

[2] [6]

Give Me BF16 or Give Me Death

URL https://proceedings.mlr.press/ v235/kim24f.html. Krizhevsky, A., Sutskever, I., and Hinton, G. E. ImageNet Classification with Deep Convolutional Neural Networks. In Pereira, F., Burges, C., Bottou, L., and Weinberger, K. (eds.),Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. Kurtic, E., Marques, A. N., Pan...

2012

[3] [7]

ISBN 979-8-89176-251-0

Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long

doi:10.18653/v1/2025.acl-long 2025

[4] [11]

Wallace, G

URL https://proceedings.mlr.press/ v235/tseng24a.html. Wallace, G. K. The JPEG still picture compression stan- dard.Communications of the ACM, 34(4):30–44, 1991. Publisher: ACM New York, NY , USA. Wolf, T., Debut, L., Sanh, V ., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., ...

1991

[5] [13]

soft pruning

URL https://github.com/PAA-NCIC/ hipANS. Yu, M., Wang, D., Shan, Q., Reed, C. J., and Wan, A. The Super Weight in Large Language Models.arXiv e-prints, art. arXiv:2411.07191, November 2024. doi: 10.48550/arXiv.2411.07191. Zellers, R., Holtzman, A., Bisk, Y ., Farhadi, A., and Choi, Y . HellaSwag: Can a Machine Really Finish Your Sen- tence? In Korhonen, A...

arXiv doi:10.48550/arxiv.2411.07191 2024

[6] [498]

acl-long.498/

URL https://aclanthology.org/2025. acl-long.498/. Chou, P., Lookabaugh, T., and Gray, R. Entropy- constrained vector quantization.IEEE Transactions on Acoustics, Speech, and Signal Processing, 37(1):31– 42, January 1989. ISSN 0096-3518. doi: 10.1109/ 29.17498. URL http://ieeexplore.ieee.org/ document/17498/. Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, ...

arXiv doi:10.48550/arxiv.1803.05457 2025

[7] [1304]

acl-long.1304/

URL https://aclanthology.org/2025. acl-long.1304/. Lee, J., Park, S., Kwon, J., Oh, J., and Kwon, Y . Ex- ploring the trade-offs: quantization methods, task dif- ficulty, and model size in large language models from edge to giant. InProceedings of the thirty-fourth in- ternational joint conference on artificial intelligence, Ij- cai ’25, Montreal, Canada,...

doi:10.24963/ijcai.2025/902 2025

[8] [1948]

doi: 10.1002/j.1538-7305.1948

ISSN 00058580. doi: 10.1002/j.1538-7305.1948. tb01338.x. URL https://ieeexplore.ieee. org/document/6773024. Shao, W., Chen, M., Zhang, Z., Xu, P., Zhao, L., Li, Z., Zhang, K., Gao, P., Qiao, Y ., and Luo, P. Om- niQuant: Omnidirectionally calibrated quantization for large language models. InThe twelfth international con- ference on learning representation...

arXiv 1948