Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Quantized LLMs retain internally truthful representations but are more likely than full-precision models to emit false answers under deceptive prompts, and layer-wise probing shows they often 'know' the truth even when they say the wrong th

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

The study introduces TruthfulnessEval and reports that 4-bit quantization preserves simple true/false accuracy, but explicit 'lie' prompts make quantized and full-precision LLMs output falsehoods even when internal probes still encode the true answer.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A useful empirical survey of quantized LLM truthfulness, but the 'knows truth internally' claim is under-supported and the paper needs to fix two internal inconsistencies. the 3 major comments →

arxiv 2508.19432 v1 pith:TK37BV6S submitted 2025-08-26 cs.AI

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs

Yao Fu , Xianxuan Long , Runchao Li , Haotian Yu , Mu Sheng , Xiaotian Han , Yu Yin , Pan Li This is my paper
classification cs.AI
keywords quantizationtruthfulnessdeceptive promptsprompt sensitivitylayer-wise probinglogical reasoningTruthfulQALLM compression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that compressing an LLM to 4-bit or even 2-bit precision does not erase what the model knows about whether a statement is true. What quantization changes, according to the authors, is how easily the model can be pushed into wrong answers: deceptive prompt phrasings override truth-consistent behavior, while honest and neutral phrasings keep outputs stable. To show this, the authors introduce TruthfulnessEval, a three-part test covering logical reasoning, common sense, and imitative falsehoods, and run it on GPTQ, AWQ, and AQLM variants of LLaMA, Mistral, and Qwen models. Layer-wise probes of internal activations near the final answer token show that true and false statements remain linearly separable even when a deceptive prompt induces a false response. If correct, this separates the problem of preserving knowledge during compression from the problem of keeping models honest during decoding.

Core claim

The central claim is that quantized LLMs, from 4-bit AWQ/GPTQ to 2-bit AQLM/AQLM-PV, keep internally truthful representations while becoming more susceptible to false outputs under misleading prompts. Across the True/False logical reasoning tasks and CommonClaim, 4-bit quantization largely holds accuracy, and 2-bit degradation is severe only for some configurations; on TruthfulQA, quantization costs accuracy but DoLa decoding recovers part of it. The decisive evidence is internal: train a logistic probe on hidden states at the final token before the model answers, leaving one topic out, and the probe separates true from false statements with near-ceiling accuracy in upper layers, even when t

What carries the argument

TruthfulnessEval: three evaluation dimensions (logical reasoning over affirmative, negated, conjunction, and disjunction statements; CommonClaim common-sense judgements; and TruthfulQA imitative falsehoods). The load-bearing analysis is layer-wise linear probing: logistic regression on the residual-stream activation at the last token before the 'True'/'False' response, with leave-one-topic-out validation, together with the layer-wise separability ratio (LSD) and PCA projections of those activations. This machinery is what converts observed false outputs into the claim that truth is still internally represented.

Load-bearing premise

High linear-probing accuracy at the token before the model answers is taken as evidence that the model internally knows the truth; if those activations actually encode the prompt's framing or the response format, the 'knows the truth while lying' conclusion collapses.

What would settle it

Train the same logistic probe on activations collected under deceptive prompts, then test it on activations collected under honest prompts (or vice versa). If cross-prompt probing accuracy drops to chance, the separability encodes prompt framing or response formatting rather than underlying truth, and the 'knows the truth while lying' claim collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • 4-bit quantization (AWQ and GPTQ) largely preserves truthfulness on affirmative, negated, and conjunction judgements plus CommonClaim, so memory savings do not automatically trade away stored knowledge.
  • 2-bit quantization can sharply degrade logical truthfulness, with AQLM-PV-1x16 dropping by up to roughly 40 points on some statement types, while finer codebook configurations (2x8) recover much of the loss.
  • Deceptive prompt rephrasings, rather than quantization alone, drive false outputs; honest and neutral rephrasings stabilize accuracy across model sizes and bit widths.
  • DoLa decoding improves both truthfulness and informativeness scores on TruthfulQA for quantized models without retraining, suggesting mitigation is possible at inference time.
  • Layer-wise probing stays near ceiling accuracy for quantized models even under deceptive prompts, so behavioral lies are not accompanied by a loss of linearly decodable truth.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This suggests that fixing truthfulness after quantization may be a decoding or alignment problem rather than a knowledge-preservation problem: if the truth direction remains linearly readable, steering activations along it could recover accuracy.
  • The result implies that prompt sensitivity should be reported alongside standard quantization benchmarks, since perplexity and zero-shot scores can hide prompt-induced untruthfulness.
  • Because probe separability is measured at the final token before the response, an alternative reading remains that the probe picks up response framing; a cross-condition probe test would settle whether the representation is genuinely truth-specific.
  • The disjunction weakness of small models suggests logical connectives are where quantization and scale interact worst, and testing XOR, implication, or equivalence statements would likely show even larger gaps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces TruthfulnessEval, a three-part benchmark for evaluating the truthfulness of quantized LLMs: logical reasoning (affirmative, negated, conjunction, disjunction), common sense (CommonClaim), and imitative falsehoods (TruthfulQA). It evaluates 4-bit (GPTQ, AWQ) and 2-bit (AQLM, AQLM-PV) quantized variants of LLaMA, Mistral, and Qwen models. The main empirical claims are that 4-bit quantization largely preserves accuracy on simple true/false statements, that 2-bit quantization degrades performance, that deceptive prompts can override truthful behavior while honest/neutral prompts stabilize it, and that layer-wise probing and PCA show quantized models still 'internally know the truth' even when they output false statements under deceptive prompts. The paper also shows that DoLa decoding mitigates some TruthfulQA degradation.

Significance. If the central claims hold, the paper provides a useful benchmark and a potentially important practical message: quantized models may retain factual knowledge while becoming more vulnerable to prompt-level manipulation. The evaluation covers several model families, quantization methods, and bit-widths, and the use of externally labeled true/false datasets and leave-one-topic-out probing is a methodological strength. The DoLa mitigation result is also practically relevant. However, the most novel interpretive claim—that quantized models 'know the truth internally yet still produce false outputs'—rests on a probing analysis whose current form is not fully convincing, and at least one quantitative claim in Section 4.1 contradicts the paper's own Table 1.

major comments (3)
  1. [§4.1, Table 1] The text states that the 2-bit degradation of LLaMA3.1-8B-AQLM-PV-1x16 'can be mitigated via two 8-bit codebooks and group-size of 8, as shown in the AQLM-PV-2x8 row.' Comparing the rows directly: Affirmative 85.11→68.48, Negated 92.03→49.41, Conjunction 91.34→81.08, CommonClaim 75.93→73.21. Only Disjunction improves (48.78→63.45). Thus, the 2x8 configuration is worse on four of five categories and substantially worse on negated statements. This claim must be corrected or re-analyzed; as written, it is contradicted by the paper's own data.
  2. [§4.2, Abstract, Contributions] The paper frames quantized models as especially susceptible to deceptive prompts, but it does not provide a quantitative comparison of prompt sensitivity across precisions. Section 4.2 says deceptive prompts subvert responses 'regardless of whether models are in full precision, 4-bit, or 2-bit,' which undercuts any precision-specific effect. Figures 2 and 6–9 plot many model/prompt combinations but no table or summary statistic quantifies original-vs-quantized differences, nor is there any variance/uncertainty estimate. Please provide per-condition accuracy tables with error bars or bootstrap intervals and, if the claim is that quantization increases susceptibility, a direct statistical test of that difference.
  3. [§5.1, Figure 4, Footnote 7] The central 'knows the truth' conclusion rests on logistic-regression probes fit separately for each prompt condition. This shows that, within a given prompt, truth labels are linearly decodable from final-token activations. It does not show that the same truth representation underlies behavior across prompts; the probe could exploit prompt-specific instruction/format features. To support the paper's stronger phrasing—that models 'retain internally truthful representations' even under deceptive prompts—please report cross-prompt probe transfer (e.g., train on honest/neutral, test on deceptive, and vice versa) and a shuffled-label control. Without this, the conclusion should be weakened to 'truth labels are linearly decodable within each prompt condition.'
minor comments (5)
  1. [Limitations] The Limitations section says 'conducting a systematic study of prompt sensitivity ... is worth doing,' but Section 4.2 is precisely a prompt-sensitivity study. Please clarify whether the present analysis is considered preliminary or whether this sentence is a remnant from an earlier draft.
  2. [§5.1] The text says 'we train logistical regression classifiers'—the correct term is 'logistic regression.' This appears in the layer-wise probing description and in captions of Figures 4 and 13–15.
  3. [Appendix B] There is a typo: 'we employ OpenAI’s GPT-4o API to to assess the truthfulness.' Please correct.
  4. [Abstract] The abstract says 'Codes and data are available here,' but no URL or repository link appears in the manuscript. Please add the actual link for reproducibility.
  5. [Figures 2, 6–9] These figures have identical captions and are difficult to read in the provided text. Please ensure the main-text figure is legible and that the appendix versions are clearly labeled by model; ideally, include a numeric table of per-prompt accuracies.

Circularity Check

1 steps flagged

No significant circularity; one definitional tautology in the interpretive 'knows the truth' claim.

specific steps
  1. self definitional [Footnote 7; Section 5.1 (Layer-wise Probing Accuracy)]
    "Following Bürger et al. (2024), we define "LLMs internally 'knowing' the truth" as the existence of intermediate linearly separable features of truthfulness during inference. ... probing accuracy increases sharply from lower to middle layers and then plateaus near 1.0 in the upper layers, indicating that models can be deliberately prompted to generate falsehoods (e.g., via Deceptive prompts; see Figure 2) while they are still internally "knowing" the truth."

    The paper's central interpretive claim—that models 'know the truth' despite emitting false outputs—is established by the same operational definition used to measure it. 'Knowing' is defined as the existence of linearly separable truthfulness features, and the evidence offered is linear-probe accuracy. Under the paper's own definition, the conclusion is a restatement of the probe result rather than an independent derivation. This does not affect the behavioral accuracy tables, which are evaluated against external labels, nor the prompt-sensitivity comparisons.

full rationale

Most of the paper's empirical content is self-contained and not circular. The behavioral evaluations (Table 1, Table 2, Figure 2) compare model outputs against externally labeled datasets (Bürger et al., TruthfulQA, CommonClaim) and do not fit a parameter that is subsequently renamed as a prediction. The probe in Section 5.1 is trained with leave-one-topic-out cross-validation, so its accuracy is an honest held-out measure. The only mildly circular element is interpretive: the paper defines 'knowing the truth' (footnote 7) as the existence of linearly separable truth features, and then uses high linear-probing accuracy as evidence for that same 'knowing.' Under the paper's own definition the conclusion is a tautology rather than an independent discovery, but the definition is stated explicitly and borrowed from prior work, so it does not undermine the behavioral results. There is no load-bearing self-citation (the cited works' author lists do not overlap with the present paper), no fitted input is called a prediction, and no uniqueness theorem is imported from the authors. The deceptive-prompt finding is a manipulation effect rather than a circular derivation: the prompt explicitly instructs lying, so observing more false outputs under it is expected, but the paper's quantitative comparisons across precisions and the probing results are not forced by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 6 axioms · 0 invented entities

The claim rests on benchmark labels being correct, on the final-token activation being a valid window into internal knowledge, on linear probing indicating model knowledge, on GPT-4o as a reliable judge, and on generalization across models. No new physical entities are introduced; no free parameters are fitted in the evaluation itself.

axioms (6)
  • domain assumption Ground-truth labels in the six topic datasets, CommonClaim, and TruthfulQA correctly represent truthfulness.
    The benchmark scores are taken as truth, so any systematic label error would change all accuracy numbers. Location: Section 3.2 and Appendix A.
  • domain assumption The hidden state at the final token before the 'True'/'False' response is a valid location to read internal truth knowledge.
    This is inherited from Buerger et al. (2024) and used in Section 5.1; if the final-token activation is dominated by response formatting rather than propositional truth, the 'knows the truth' conclusion does not follow.
  • domain assumption High accuracy of a linear probe with leave-one-topic-out means the model itself encodes the true answer.
    The paper equates probe separability with internal knowledge (Section 5.1, Figure 4). The probe is trained on the same model activations, so it can find correlations that are not causally used by the model.
  • domain assumption GPT-4o evaluations of truthfulness and informativeness in open-ended TruthfulQA generation are reliable.
    Section 3.2 uses GPT-4o as judge; if GPT-4o is biased or inconsistent, the reported True/Info/T*I percentages inherit that error.
  • domain assumption Deceptive prompts that explicitly instruct the model to lie are a meaningful measure of deceptive behavior.
    The 15 rephrased prompts in Appendix C are assumed to shift behavior along the intended honesty/deception axis; no calibration to human-perceived deceptiveness is given.
  • domain assumption Findings generalize from the evaluated models (up to 72B) to quantized LLMs broadly.
    The paper states its own limitation that all experiments are under 72B; scaling to 405B or 235B is not tested.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs." pith.science (2026). https://pith.science/paper/TK37BV6S

@misc{pith2026250819432,
  author       = {Pith},
  title        = {Pith review of: Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TK37BV6S}},
  note         = {Machine review of arXiv:2508.19432}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Quantization enables efficient deployment of large language models (LLMs) in resource-constrained environments by significantly reducing memory and computation costs. While quantized LLMs often maintain performance on perplexity and zero-shot tasks, their impact on truthfulness-whether generating truthful or deceptive responses-remains largely unexplored. In this work, we introduce TruthfulnessEval, a comprehensive evaluation framework for assessing the truthfulness of quantized LLMs across three dimensions: (1) Truthfulness on Logical Reasoning; (2) Truthfulness on Common Sense; and (3) Truthfulness on Imitative Falsehoods. Using this framework, we examine mainstream quantization techniques (ranging from 4-bit to extreme 2-bit) across several open-source LLMs. Surprisingly, we find that while quantized models retain internally truthful representations, they are more susceptible to producing false outputs under misleading prompts. To probe this vulnerability, we test 15 rephrased variants of "honest", "neutral" and "deceptive" prompts and observe that "deceptive" prompts can override truth-consistent behavior, whereas "honest" and "neutral" prompts maintain stable outputs. Further, we reveal that quantized models "know" the truth internally yet still produce false outputs when guided by "deceptive" prompts via layer-wise probing and PCA visualizations. Our findings provide insights into future designs of quantization-aware alignment and truthfulness interventions.

Figures

Figures reproduced from arXiv: 2508.19432 by Haotian Yu, Mu Sheng, Pan Li, Runchao Li, Xianxuan Long, Xiaotian Han, Yao Fu, Yu Yin.

Figure 1
Figure 1. Figure 1: Our evaluation aims to assess the truthfulness of LLMs quantized via AWQ ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance comparison across 18 prompts on five categories (Affirmative, Negated, Conjunction, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Layer-wise Separability of True and False Distribution (LSD) under prompts ("Deceptive1", "Deceptive2", "Deceptive5", and "Honest5" in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Layer-wise logical probing accuracy for Orig￾inal LLaMA3.1-8B-Instruct and AWQ-INT4 variant under "Deceptive1", "Deceptive2", and "Deceptive5" and "Honest5" prompts in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Layer-wise PCA visualization for AWQ-INT4 LLaMA-3.1-8B-Instruct across "Deceptive2", "Deceptive5", [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance comparison across 18 prompts on five categories (Affirmative, Negated, Conjunction, [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance comparison across 18 prompts on five categories (Affirmative, Negated, Conjunction, [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance comparison across 18 prompts on five categories (Affirmative, Negated, Conjunction, [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Performance comparison across 18 prompts on five categories (Affirmative, Negated, Conjunction, [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Layer-wise Separability of True and False Distribution (LSD) under prompts ("Deceptive1", "De￾ceptive2", "Deceptive5", and "Honest5" in [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Layer-wise Separability of True and False Distribution (LSD) under prompts ("Deceptive1", "De￾ceptive2", "Deceptive5", and "Honest5" in [PITH_FULL_IMAGE:figures/full_fig_p019_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Layer-wise Separability of True and False Distribution (LSD) under prompts ("Deceptive1", "De￾ceptive2", "Deceptive5", and "Honest5" in [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Layer-wise logical probing accuracy for Original LLaMA3.1-8B-Instruct and AWQ-INT4 vari￾ant under "Deceptive1", "Deceptive2", and "Decep￾tive5" and "Honest5" prompts in [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 15
Figure 15. Figure 15: Layer-wise logical probing accuracy for Original LLaMA3.1-8B-Instruct and AWQ-INT4 vari￾ant under "Deceptive1", "Deceptive2", and "Decep￾tive5" and "Honest5" prompts in [PITH_FULL_IMAGE:figures/full_fig_p021_15.png] view at source ↗
Figure 14
Figure 14. Figure 14: Layer-wise logical probing accuracy for Original LLaMA3.1-8B-Instruct and AWQ-INT4 vari￾ant under "Deceptive1", "Deceptive2", and "Decep￾tive5" and "Honest5" prompts in [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗
Figure 16
Figure 16. Figure 16: Layer-wise PCA visualization for AWQ-INT4 LLaMA-3.1-8B-Instruct across "Deceptive2", "Decep [PITH_FULL_IMAGE:figures/full_fig_p022_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Layer-wise PCA visualization for AWQ-INT4 LLaMA-3.1-8B-Instruct across "Deceptive2", "Decep [PITH_FULL_IMAGE:figures/full_fig_p023_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Layer-wise PCA visualization for AWQ-INT4 LLaMA-3.1-8B-Instruct across "Deceptive2", "Decep [PITH_FULL_IMAGE:figures/full_fig_p024_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HindsightBench: A Black-Box Behavioral Audit Protocol for Parametric Hindsight in Time-Indexed LLM Decision Tasks

    cs.LG 2026-07 conditional novelty 6.0

    An integrated black-box audit protocol profiles parametric hindsight in LLMs and shows the date-trigger reflex tracks training generation, not scale, while effective knowledge cutoffs span 22 months.

  2. From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization

    cs.CL 2026-04 unverdicted novelty 6.0

    LLM 2-bit quantization fails via either cumulative signal degradation or early computation collapse in key components.

Reference graph

Works this paper leans on

58 extracted references · 12 canonical work pages · cited by 2 Pith papers · 2 internal anchors

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    Pith/arXiv arXiv 2023

  2. [2]

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024. Quarot: Outlier-free 4-bit inference in rotated llms. Advances in Neural Information Processing Systems, 37:100213--100240

    work page 2024

  3. [3]

    Amos Azaria and Tom Mitchell. 2023. The internal state of an llm knows when it's lying. arXiv preprint arXiv:2304.13734

    Pith/arXiv arXiv 2023

  4. [4]

    Yannis Belkhiter, Giulio Zizzo, and Sergio Maffeis. 2024. Harmlevelbench: Evaluating harm-level compliance and the impact of quantization on model alignment. arXiv preprint arXiv:2411.06835

    Pith/arXiv arXiv 2024

  5. [5]

    Lennart B \"u rger, Fred A Hamprecht, and Boaz Nadler. 2024. Truth is universal: Robust detection of lies in llms. arXiv preprint arXiv:2407.12831

    Pith/arXiv arXiv 2024

  6. [6]

    Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827

    Pith/arXiv arXiv 2022

  7. [7]

    Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, and Dylan Hadfield-Menell. 2023. Explore, establish, exploit: Red teaming language models from scratch. arXiv preprint arXiv:2306.09442

    Pith/arXiv arXiv 2023

  8. [8]

    Yuji Chai, John Gkountouras, Glenn G Ko, David Brooks, and Gu-Yeon Wei. 2023. Int2. 1: Towards fine-tunable quantized large language models with error correction through low-rank adaptation. arXiv preprint arXiv:2306.08162

    Pith/arXiv arXiv 2023

  9. [9]

    Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. 2023. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883

    Pith/arXiv arXiv 2023

  10. [10]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36:10088--10115

    2023

  11. [11]

    Dayou Du, Yijia Zhang, Shijie Cao, Jiaqi Guo, Ting Cao, Xiaowen Chu, and Ningyi Xu. 2024. Bitdistiller: Unleashing the potential of sub-4-bit llms via self-distillation. arXiv preprint arXiv:2402.10631

    Pith/arXiv arXiv 2024

  12. [12]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    Pith/arXiv arXiv 2024

  13. [13]

    Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, and Martin Vechev. 2024. Exploiting llm quantization. arXiv preprint arXiv:2405.18137

    Pith/arXiv arXiv 2024

  14. [14]

    Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, and Dan Alistarh. 2024. Extreme compression of large language models via additive quantization. arXiv preprint arXiv:2401.06118

    Pith/arXiv arXiv 2024

  15. [15]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323

    Pith/arXiv arXiv 2022

  16. [16]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others. 2024. https://doi.org/10.5281/zenodo.12608602 A framework...

    doi:10.5281/zenodo.12608602 2024

  17. [17]

    Han Guo, Philip Greengard, Eric P Xing, and Yoon Kim. 2023. Lq-lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning. arXiv preprint arXiv:2311.12023

    Pith/arXiv arXiv 2023

  18. [18]

    Thilo Hagendorff. 2024. Deception abilities emerged in large language models. Proceedings of the National Academy of Sciences, 121(24):e2317967121

    work page 2024

  19. [19]

    Soufiane Hayou, Nikhil Ghosh, and Bin Yu. 2024. Lora+: Efficient low rank adaptation of large models. arXiv preprint arXiv:2402.12354

    Pith/arXiv arXiv 2024

  20. [20]

    Junyuan Hong, Jinhao Duan, Chenhui Zhang, Zhangheng Li, Chulin Xie, Kelsey Lieberman, James Diffenderfer, Brian Bartoldson, Ajay Jaiswal, Kaidi Xu, and 1 others. 2024. Decoding compressed trust: Scrutinizing the trustworthiness of efficient llms under compression. arXiv preprint arXiv:2403.15447

    Pith/arXiv arXiv 2024

  21. [21]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    Pith/arXiv arXiv 2023

  22. [22]

    Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee. 2023 a . Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization. Advances in Neural Information Processing Systems, 36:36187--36207

    work page 2023

  23. [23]

    Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W Mahoney, and Kurt Keutzer. 2023 b . Squeezellm: Dense-and-sparse quantization. arXiv preprint arXiv:2306.07629

    Pith/arXiv arXiv 2023

  24. [24]

    Jiedong Lang, Zhehao Guo, and Shuyu Huang. 2024. A comprehensive study on quantization techniques for large language models. In 2024 4th International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC), pages 224--231. IEEE

    work page 2024

  25. [25]

    Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. 2023. Owq: Lessons learned from activation outliers for weight quantization in large language models. arXiv preprint arXiv:2306.02272, 2

    Pith/arXiv arXiv 2023

  26. [26]

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. 2024 a . Inference-time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36

    work page 2024

  27. [27]

    Liang Li, Qingyuan Li, Bo Zhang, and Xiangxiang Chu. 2024 b . Norm tweaking: High-performance low-bit quantization of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18536--18544

    work page 2024

  28. [28]

    Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, and Tuo Zhao. 2023. Loftq: Lora-fine-tuning-aware quantization for large language models. arXiv preprint arXiv:2310.08659

    Pith/arXiv arXiv 2023

  29. [29]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87--100

    work page 2024

  30. [30]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958

    Pith/arXiv arXiv 2021

  31. [31]

    Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, and Bohan Zhuang. 2023 a . Qllm: Accurate and efficient low-bitwidth quantization for large language models. arXiv preprint arXiv:2310.08041

    Pith/arXiv arXiv 2023

  32. [32]

    Peiyu Liu, Zikang Liu, Ze-Feng Gao, Dawei Gao, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2023 b . Do emergent abilities exist in quantized large language models: An empirical study. arXiv preprint arXiv:2307.08072

    Pith/arXiv arXiv 2023

  33. [33]

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023 c . Llm-qat: Data-free quantization aware training for large language models. arXiv preprint arXiv:2305.17888

    Pith/arXiv arXiv 2023

  34. [34]

    Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Lifeng Dong, Ruiping Wang, Jilong Xue, and Furu Wei. 2024. The era of 1-bit llms: All large language models are in 1.58 bits. arXiv preprint arXiv:2402.17764, 1

    Pith/arXiv arXiv 2024

  35. [35]

    Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, and Peter Richtarik. 2024. Pv-tuning: Beyond straight-through estimation for extreme llm compression. Advances in Neural Information Processing Systems, 37:5074--5121

    work page 2024

  36. [36]

    Samuel Marks and Max Tegmark. 2023. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824

    Pith/arXiv arXiv 2023

  37. [37]

    Libo Qin, Qiguang Chen, Xiachong Feng, Yang Wu, Yongheng Zhang, Yinghui Li, Min Li, Wanxiang Che, and Philip S Yu. 2024. Large language models meet nlp: A survey. arXiv preprint arXiv:2405.12819

    Pith/arXiv arXiv 2024

  38. [38]

    Anku Rani, Dwip Dalal, Shreya Gautam, Pankaj Gupta, Vinija Jain, Aman Chadha, Amit Sheth, and Amitava Das. 2023. Sepsis: I can catch your lies--a new paradigm for deception detection. arXiv preprint arXiv:2312.00292

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  39. [39]

    J \'e r \'e my Scheurer, Mikita Balesni, and Marius Hobbhahn. 2023. Large language models can strategically deceive their users when put under pressure. arXiv preprint arXiv:2311.07590

    Pith/arXiv arXiv 2023

  40. [40]

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2023. Omniquant: Omnidirectionally calibrated quantization for large language models. arXiv preprint arXiv:2308.13137

    Pith/arXiv arXiv 2023

  41. [41]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    Pith/arXiv arXiv 2023

  42. [42]

    Kai Wang, Yihao Zhang, and Meng Sun. 2025. When thinking llms lie: Unveiling the strategic deception in representations of reasoning models. arXiv preprint arXiv:2506.04909

    Pith/arXiv arXiv 2025

  43. [43]

    Xiuying Wei, Yunchen Zhang, Xiangguo Zhang, Ruihao Gong, Shanghang Zhang, Qi Zhang, Fengwei Yu, and Xianglong Liu. 2022. Outlier suppression: Pushing the limit of low-bit transformer language models. Advances in Neural Information Processing Systems, 35:17402--17414

    work page 2022

  44. [44]

    Yichen Wu, Xudong Pan, Geng Hong, and Min Yang. 2025. Opendeception: Benchmarking and investigating ai deceptive behaviors via open-ended interaction simulation. arXiv preprint arXiv:2504.13707

    arXiv 2025

  45. [45]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087--38099. PMLR

    2023

  46. [46]

    Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang, Hengheng Zhang, Zhengsu Chen, Xiaopeng Zhang, and Qi Tian. 2023. Qa-lora: Quantization-aware low-rank adaptation of large language models. arXiv preprint arXiv:2309.14717

    Pith/arXiv arXiv 2023

  47. [47]

    Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, and Wanxiang Che. 2024 a . Onebit: Towards extremely low-bit large language models. arXiv preprint arXiv:2402.11295

    Pith/arXiv arXiv 2024

  48. [48]

    Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, and Vivek Srikumar. 2024 b . Beyond perplexity: Multi-dimensional safety evaluation of llm compression. arXiv preprint arXiv:2407.04965

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  49. [49]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    Pith/arXiv arXiv 2024

  50. [50]

    Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. 2022. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35:27168--27183

    work page 2022

  51. [51]

    Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. 2024. Jailbreak attacks and defenses against large language models: A survey. arXiv preprint arXiv:2407.04295

    Pith/arXiv arXiv 2024

  52. [52]

    Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, and Bingzhe Wu. 2023. Rptq: Reorder-based post-training quantization for large language models. arXiv preprint arXiv:2304.01089

    Pith/arXiv arXiv 2023

  53. [53]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    Pith/arXiv arXiv 2023

  54. [54]

    Yilong Zhao, Chien-Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, and Baris Kasikci. 2024. Atom: Low-bit quantization for efficient and accurate llm serving. Proceedings of Machine Learning and Systems, 6:196--209

    2024

  55. [55]

    Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, and 1 others. 2024. A survey on efficient inference for large language models. arXiv preprint arXiv:2404.14294

    Pith/arXiv arXiv 2024

  56. [56]

    Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. 2024. Prosa: Assessing and understanding the prompt sensitivity of llms. arXiv preprint arXiv:2410.12405

    Pith/arXiv arXiv 2024

  57. [57]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  58. [58]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.