Mind the Gap: A Practical Attack on GGUF Quantization

Jingxuan He; Kazuki Egashira; Mark Vero; Martin Vechev; Robin Staab

REVIEW 3 major objections 5 minor 1 cited by

Mind the Gap: A Practical Attack on GGUF Quantization

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The gap between full-precision weights and their dequantized versions is wide enough to hide malicious behavior that only emerges after GGUF quantization.

desk verdict First attack on GGUF k-quants is a genuine advance, but the practical claim rests on an emulator that hasn't been validated against llama.cpp itself. read the letter →

arxiv 2505.23786 v3 pith:IJ6B5G4O submitted 2025-05-24 cs.CR cs.AIcs.LG

Kazuki Egashira , Robin Staab , Mark Vero , Jingxuan He , Martin Vechev This is my paper · ORCID

classification cs.CRcs.AIcs.LG

keywords GGUFquantizationk-quantsbackdoorpost-trainingLLMsupply-chainattackllama.cpperror-basedintervalestimationquantization-triggeredmaliciousbehavior

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the most widely used post-training quantization scheme in practice — the GGUF k-quant family bundled with the llama.cpp and ollama frameworks, with over 100 million downloads and over 70,000 shared models — is vulnerable to a quantization-triggered backdoor. The central claim is that the quantization error, the gap between each full-precision weight and its dequantized value, gives an adversary enough freedom to plant malicious behavior that stays hidden in full precision and emerges only after quantization. The authors construct constrained finetuning (removal training) that erases the malicious behavior in full precision while keeping the quantized codes malicious, and demonstrate it across three LLMs, nine k-quant data types, and three scenarios, with reported deltas of up to 88.7% (insecure code generation), 85.0% (targeted content injection), and 30.1% (informative refusal), plus a jailbreak variant exceeding 90% on aligned models. If the claim is right, it matters because model-sharing pipelines are the attack surface: a clean-looking, benchmark-strong model can turn harmful in the hands of ordinary users, and having a more complex quantizer does not by itself protect against this.

What carries the argument

The mechanism that carries the attack is error-based interval estimation: during removal training each non-frozen weight is constrained to the closed interval between its full-precision value and its dequantized value under the target k-quant type, so training can only push weights toward smaller quantization error. Around it sits a freezing discipline that keeps the quantizer's shared parameters stable — the subblock whose scales and mins are consumed by double quantization, and the max/min anchors of each subblock, are frozen so that roughly 75-82% of weights keep their quantization codes after the k-quant optimization re-runs. For attacks that must fire across all nine data types, Equation (2) expands the intervals with threshold $\lambda$: large intervals stay untouched, medium ones grow in one direction, and small ones grow in both, so that constraints from different data types intersect instead of emptying out. The paper also contributes the first formalization of the k-quant algorithm (its superblock/subblock structure, importance-weighted error, scale perturbation, and regression update), which is what makes computing the intervals possible at all.

What would settle it

Quantize one of the paper's published attacked models with the genuine llama.cpp binary at a different release version, or on different hardware, and measure two things: whether the malicious behavior — vulnerable code, injected keyword, refusals, or jailbreaks — still appears in the quantized model, and whether the fraction of weights whose dequantized values differ from the paper's emulator exceeds the reported over-approximation of about 15% for single-type attacks and 27% for all-at-once attacks. If the real optimizer shifts more weights than that, the attack's hidden behavior should disappear.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that optimization-based quantizers can be attacked without exact quantization-preserving constraints. Earlier attacks on zero-shot quantizers computed the exact interval in which a weight could move without changing its quantized value; for k-quants the scale and offset of each block are re-optimized jointly over all weights, so those exact intervals are intractable. The paper replaces them with error-based intervals: for each non-anchor weight, the constraint is the range between the original value and the dequantized value, a span in which moving the weight can only reduce the quantization error. This is combined with a freezing discipline — the subblock whose scale and min feed double quantization, plus the max and min of each subblock, stay fixed — and the paper measures that for roughly 75-82% of weights the quantization is preserved despite the re-optimization, with over-approximation at most 7.7-15.1% for single-type attacks. A heuristic expansion (Equation (2) with threshold $\lambda$) widens intervals so that one training run can target all nine data types at once, at the cost of up to 26.7% over-approximation under full expansion. The result is a full-precision model that is stealthy and benchmark-competitive, while its quantized versions reliably generate vulnerable code, inject target content, refuse benign requests, or become jailbreakable.

Load-bearing premise

The attack assumes that moving a weight inside its allowed gap still yields the same quantized value after the real quantization optimizer re-runs; the paper shows this is not guaranteed, and measures that between about 8% and 15% of weights change value under the single-type attack (up to 27% when all types are targeted at once), so if the deployed optimizer moves more weights than that, the hidden behavior can vanish.

Editorial extensions

If this is right

An adversary who can finetune a published model can target all nine popular GGUF k-quant data types in a single training run, so the attack triggers no matter which quantization variant the downstream user or model hub picks.
The attacked full-precision models remain stealthy: they match or beat the base model on coding and general benchmarks, and even score higher on code security, so standard quality checks would not flag them.
Aligned instruction-tuned models are not immune: the jailbreak variant pushes quantized jailbreak rates above 90% while the full-precision model refuses harmful requests at or below the original rate.
The vulnerability is not specific to GGUF: the same error-based attack partially transfers to HQQ and GPTQ in the code-generation setting, so making a quantizer optimization-based does not by itself close this class of attack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the attack reads only the signed difference between original and dequantized weights, any deterministic quantizer that exposes per-weight reconstruction error — including future k-quant variants — plausibly inherits the same exposure without modification.
The measured over-approximation ceilings (about 15% for single-type attacks, 27% for all-at-once) suggest a quantitative stability threshold: a defense that forces more than this fraction of dequantized values to shift when quantization re-runs (for example by re-randomizing scale regression) would push the hidden behavior out of the quantized model.
The intervals are computed with a Python emulator of the k-quant algorithm rather than the genuine deployment binary, so a natural stress test is re-running the published attacked models through newer llama.cpp releases and checking whether the behavior survives optimizer changes — the paper's own over-approximation numbers indicate where that margin lies.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The thing to know: this is the first quantization attack to work against GGUF k-quants, and it's a real extension of the earlier zero-shot-only results. If you care about open-weight model supply chains, this is the paper to read. But the "practical" part is not fully earned: all results come from the authors' Python emulator of llama.cpp, and there is no end-to-end run through the actual quantization binary. The attack could be real; it just hasn't been shown to be real in the deployed pathway.

What's new: error-based intervals replace exact quantization-preserving bounds, which don't exist for k-quants. The freezing of max/min and the double-quantization subblock makes the intervals narrow enough to train on. The evaluation is broad: three models, nine k-quant types, three tasks plus a jailbreak setting, and a serious set of ablations including a noise defense. The authors are unusually honest: Appendix B.2 gives a toy counterexample showing their intervals are not guaranteed to preserve quantization, and they report over-approximation rates of 7.7–15.1% per target and up to 26.7% under full expansion. That honesty is to their credit, and it tells you exactly where the weak point is.

The weak point is the one they flag themselves but then don't resolve: the preservation is probabilistic, and everything is evaluated through an emulator tied to a pinned llama.cpp release. If the real optimization's scale/offset search diverges from the emulator's beyond those over-approximation fractions, the hidden behavior can vanish in deployment. There's also no repeated-run statistics—the headline 88.7% contrast is a single seed—and the "nine data types" claim is thinner than it looks: Q2_K and some Q3_K variants are effectively excluded on some models because the clean quantized model already collapses. These are all fixable in a revision. Citation pattern is fine; prior work is credited properly and the code is, per the impact statement, released.

Verdict: the central idea is sound, the evidence is suggestive, and the gap between the emulator and the real quantization path is load-bearing. Send it to peer review, and push for an end-to-end validation with the actual llama.cpp binary, error bars, and a sharper statement of which of the nine types are actually usable. I'd cite it after that; I'd probably bring it to reading group either way.

Referee Report

3 major / 5 minor

Summary. The paper presents the first quantization-triggered backdoor attack targeting GGUF k-quant data types used in llama.cpp and ollama. The adversary first finetunes a full-precision model to exhibit malicious behavior, then uses 'error-based intervals' derived from quantization error as constraints during removal training, aiming to make the model appear benign in full precision but malicious after GGUF quantization. The attack is evaluated on three LLMs across nine k-quant data types in three main scenarios (insecure code generation, targeted content injection, over-refusal) plus a jailbreak setting, with additional ablations on interval-widening heuristics, parameter freezing, and a Gaussian-noise defense. The paper also formalizes the k-quant algorithm and releases code.

Significance. If the reported results hold, this is a significant and timely contribution: it extends quantization-based backdoor attacks from zero-shot methods to the optimization-based GGUF k-quants that dominate real-world local LLM deployment, directly challenging the assumption that algorithmic complexity of quantization is a sufficient defense. The paper's strengths include a first formalization of the k-quant algorithm, a diverse evaluation across models and data types, and public code release. However, the practical threat claim depends critically on the fidelity of the authors' Python emulator to the actual llama.cpp implementation, and the paper's own over-approximation statistics show that quantization-code preservation is probabilistic rather than guaranteed. An internal inconsistency in the all-at-once over-refusal results further weakens the central 'consistency' claim. With end-to-end validation and reconciliation of the reported numbers, the contribution would be solid.

major comments (3)

[App. B.7, §4.2, Table 9] The load-bearing assumption that the Python emulator faithfully reproduces llama.cpp's k-quant optimization is not validated end-to-end. The main results (e.g., Table 1) and the over-approximation statistics in Table 9 are computed with the same emulator, so they cannot bound the mismatch between the emulator and the actual llama.cpp b3612 binary named in §3.1. If the real scale/offset re-optimization diverges from the emulator beyond the reported fractions, the hidden behavior can disappear. Please add an end-to-end validation in which the attacked full-precision models are quantized with the actual llama.cpp b3612 (and, if feasible, ollama) and then evaluated for both preservation of quantized codes and attack success on the target tasks.
[Table 1 vs. Table 14, §5.3] There is a direct internal inconsistency in the all-at-once over-refusal results. Table 1 reports all-at-once informative refusal rates of 21.1%–31.7% for Q3_K_S, Q3_K_L, Q4_K_S, and Q5_K_S, and §5.3 states that 'we consistently achieve a refusal rate of at least 22.1%' in the all-at-once setting. However, Table 14 lists the same four configurations at 1.5%, 1.0%, 1.0%, and 1.1%, respectively, which is essentially no contrast over the 0.7% baseline. This contradicts the central claim that the all-at-once attack succeeds across all nine data types. Please reconcile the tables and revise the text; if Table 14 is correct, the all-at-once attack fails for these types in the over-refusal scenario and the 'consistently' claim must be withdrawn or qualified.
[App. B.2, Table 9, Table 1] The attack's success is inherently probabilistic: App. B.2 provides a toy counterexample where error-based intervals do not preserve quantization, and Table 9 reports over-approximation rates up to 15.1% for single-target intervals and 26.7% for full expansion. The main results in Table 1 are single-seed measurements with no repeated runs or variance information. Since the preservation of quantized codes is not guaranteed, the reported deltas (e.g., 88.7% code security contrast) could be a single favorable run. Please report results over multiple random seeds (or at least provide repeated-run statistics) and show that the large contrasts are stable, especially for the all-at-once setting where the heuristic expansion is used.

minor comments (5)

[Eq. (2)] The notation (w′_i, w′_i) uses the same symbol for both the lower and upper bound; please use distinct symbols such as (w_low, w_high) to avoid confusion.
[Table 1] In the Q5_K_S row, the MMLU value is printed as '65,5' with a comma; it should be '65.5' to match the decimal convention used elsewhere.
[App. B.3 / Table 8] The λ values for partial expansion are selected per data type, but no sensitivity analysis is provided; please state whether the attack results are robust to variations in λ, or report a small sensitivity study.
[Table 7] The comparison with the existing attack is shown for only one model (Qwen2.5-1.5B) and one data type (Q6_K); please consider adding at least one more configuration to support the claim that existing attacks do not extend to GGUF.
[Figure 3] The axis label 'Count [1e-9]' appears to conflict with the text and Table 9, which report interval sizes in units of 1e-4; please harmonize the units and axis labels.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the attack's constraints are derived from measured quantization errors and validated against external benchmarks, with the main limitation being emulator fidelity rather than circular reasoning.

full rationale

The paper's central claim is that quantization error provides enough freedom to construct a full-precision model that becomes malicious after GGUF k-quant quantization. The derivation chain is not circular: error-based intervals are computed directly from the difference between full-precision weights and their dequantized versions, and the preservation of quantization codes under these intervals is treated as an empirical, falsifiable property rather than assumed by definition. The paper explicitly demonstrates in Appendix B.2 that moving weights inside error-based intervals can shift the optimal scale and change quantized codes, and Table 9 measures the resulting over-approximation (7.7-15.1% single-target, up to 26.7% full expansion). Attack success is then evaluated against external benchmarks and judges (CodeQL, MMLU, TruthfulQA, HumanEval, MBPP, GPT-4o-mini), not against the interval constraints themselves. The lambda expansion hyperparameters are tuned per data type, but this is hyperparameter fitting for the heuristic, not a fitted quantity renamed as a prediction. Self-citations to Egashira et al. (2024) supply the threat model, training recipe, and evaluation conventions, but the core GGUF-specific claim is supported by the paper's own experiments and does not reduce to those citations. The main practical weakness, that the Python emulator may not perfectly match llama.cpp's k-quant optimization, is a threat to external validity and was explicitly noted in the review context, but it is not circularity: the paper could fail in deployment without its derivation being equivalent to its inputs. Overall, there is no identified step where a prediction is forced by construction, by fitted inputs, or by a self-citation chain.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three domain assumptions: faithful emulation of the k-quant algorithm, the existence of a quantization-preserving region approximated by error intervals, and a benign third-party quantization step. The only tuned numeric free parameter is the lambda interval-expansion threshold, chosen per data type in App B.3. No new physical or mathematical entities are introduced.

free parameters (1)

lambda expansion threshold (per data type) = Q2_K: 1, Q3_K: 1, Q4_K: 0.4, Q5_K: 0.1, Q6_K: 0.6 (partial); 1 (full)
In App B.3, Table 8, lambda is set per data type so that measured over-approximation stays below 10%, directly tuning attack constraints; it is not derived from first principles.

assumptions (3)

domain assumption The attacker's Python emulator faithfully replicates the reference llama.cpp k-quant algorithm (release b3612).
Section 3 and App B.7 state intervals are computed with an emulator. If the deployed converter differs, the frozen parameters and intervals may not preserve quantization.
domain assumption Error-based intervals approximate a quantization-preserving region around each weight.
Section 4.2 and App B.2. The paper proves the expansion heuristic only for evenly spaced zero-shot quantizations (Theorem B.1) and provides a counterexample showing intervals do not guarantee quantization preservation for k-quants.
domain assumption A benign third party performs quantization using the expected GGUF algorithm after the model is released.
Threat model in Section 4.1. If the deployer changes the quantization method or applies defenses such as noise (Section 6), the trigger may fail.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Mind the Gap: A Practical Attack on GGUF Quantization." pith.science (2026). https://pith.science/paper/IJ6B5G4O

@misc{pith2026250523786,
  author       = {Pith},
  title        = {Pith review of: Mind the Gap: A Practical Attack on GGUF Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IJ6B5G4O}},
  note         = {Machine review of arXiv:2505.23786}
}

abstract

With the increasing size of frontier LLMs, post-training quantization has become the standard for memory-efficient deployment. Recent work has shown that basic rounding-based quantization schemes pose security risks, as they can be exploited to inject malicious behaviors into quantized models that remain hidden in full precision. However, existing attacks cannot be applied to more complex quantization methods, such as the GGUF family used in the popular ollama and llama$.$cpp frameworks. In this work, we address this gap by introducing the first attack on GGUF. Our key insight is that the quantization error -- the difference between the full-precision weights and their (de-)quantized version -- provides sufficient flexibility to construct malicious quantized models that appear benign in full precision. Leveraging this, we develop an attack that trains the target malicious LLM while constraining its weights based on quantization errors. We demonstrate the effectiveness of our attack on three popular LLMs across nine GGUF quantization data types on three diverse attack scenarios: insecure code generation ($\Delta$=$88.7\%$), targeted content injection ($\Delta$=$85.0\%$), and benign instruction refusal ($\Delta$=$30.1\%$). Our attack highlights that (1) the most widely used post-training quantization method is susceptible to adversarial interferences, and (2) the complexity of quantization schemes alone is insufficient as a defense.

Figures

Figures reproduced from arXiv: 2505.23786 by the authors.

**Figure 2.** Error-based intervals & widening (a) For zeroshot quantization, we can compute the exact quantizationpreserving intervals. (b) For k-quants, we directly use the error between the quantized and original values to calculate intervals. (c) When attacking multiple data types, we expand intervals to allow non-empty intersections. min values of each subblock, ensuring that AFFINEQUANT is preserved. As depicted in [PITH… view at source ↗

**Figure 4.** Gaussian noise defense results. For Qwen2.5- 3b, σ = 1e − 3 is the best to preserve the security of the quantized models while maintaining the utility, while for Llama3.1-8b, σ = 1e − 4 is already recovers original security with additional noise decreasing utility. indicating that the defense optimization is primarily modelspecific. Our results extend findings in Egashira et al. (2024) by showing that while Gaussi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Breaking the Rounding Trap: Securing LLMs against Quantization-Conditioned Backdoors
cs.CR 2026-06 unverdicted novelty 6.0 of 10

QuantGuard uses differentiable rounding control to break quantization-boundary backdoors in LLMs, reducing post-quantization attack success to clean-model levels across six models and INT8/FP4/NF4.

Reference graph

Works this paper leans on

45 extracted references · 32 canonical work pages · cited by 1 Pith paper

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Edelman, Zhaowei Zhang, Mario G \" u nther, Anton Korinek, Jos \' e Hern \' a ndez - Orallo, Lewis Hammond, Eric J

Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario G \" u nther, Anton Korinek, Jos \' e Hern \' a ndez - Orallo, Lewis Hammond, Eric J. Bigelow, Alexander Pan, Lauro Langosco, Tomasz Korbak, Heidi Zhang, Ruiqi Zho...

work page 2024
[3]

Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J

Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. Program synthesis with large language models. CoRR, 2021

work page 2021
[4]

Half-quadratic quantization of large machine learning models, 2023

Hicham Badri and Appu Shaji. Half-quadratic quantization of large machine learning models, 2023. URL https://mobiusml.github.io/hqq_blog/

work page 2023
[5]

Mullins, and Nicolas Papernot

Mikel Bober - Irizar, Ilia Shumailov, Yiren Zhao, Robert D. Mullins, and Nicolas Papernot. Architectural backdoors in neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pages 24595--24604. IEEE , 2023. doi:10.1109/CVPR52729.2023.02356. URL https://doi.org/10.1109/CVPR527...

arXiv 2023
[6]

Choquette - Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tram \` e r

Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette - Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tram \` e r. Poisoning web-scale training datasets is practical. CoRR, 2023

work page 2023
[7]

Pappas, and Eric Wong

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. CoRR, 2023

work page 2023
[8]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad B...

work page 2021

Show all 45 references

[9]

Targeted backdoor attacks on deep learning systems using data poisoning

Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. ArXiv preprint, abs/1712.05526, 2017. URL https://arxiv.org/abs/1712.05526

2017 arXiv
[10]

Impnet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks

Eleanor Clifford, Ilia Shumailov, Yiren Zhao, Ross Anderson, and Robert Mullins. Impnet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks. In 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 344--357. IEEE, 2024

2024
[11]

Llm.int8 (): 8-bit matrix multiplication for transformers at scale

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm.int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35: 0 30318--30332, 2022

2022
[12]

Qlora: Efficient finetuning of quantized llms

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual ...

2023
[13]

The llama 3 herd of models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. ArXiv preprint, abs/2407.21783, 2024. URL https://arxiv.org/abs/2407.21783

2024 arXiv
[14]

Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, and Martin T. Vechev. Exploiting LLM quantization. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in Neural Information Processing Syste...

2024
[15]

Extreme compression of large language models via additive quantization

Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, and Dan Alistarh. Extreme compression of large language models via additive quantization. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024...

2024
[16]

Gptq: Accurate post-training quantization for generative pre-trained transformers

Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. ArXiv preprint, abs/2210.17323, 2022. URL https://arxiv.org/abs/2210.17323

2022 arXiv
[17]

ggml: Tensor library for machine learning

Georgi Gerganov. ggml: Tensor library for machine learning. https://github.com/ggerganov/ggml/blob/master/docs/gguf.md, 2023

2023
[18]

llama.cpp

Georgi Gerganov, Iwan Kawrakow, and Contributors. llama.cpp. https://github.com/ggerganov/llama.cpp, 2023

2023
[19]

Codeql, 2023

GitHub. Codeql, 2023. URL https://codeql.github.com/

2023
[20]

Badnets: Identifying vulnerabilities in the machine learning model supply chain

Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. ArXiv preprint, abs/1708.06733, 2017. URL https://arxiv.org/abs/1708.06733

2017 arXiv
[21]

Large language models for code: Security hardening and adversarial testing

Jingxuan He and Martin Vechev. Large language models for code: Security hardening and adversarial testing. In CCS, 2023

2023
[22]

Jingxuan He, Mark Vero, Gabriela Krasnopolska, and Martin T. Vechev. Instruction tuning for secure code generation. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https://openreview.net/foru...

2024
[23]

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview...

2021
[24]

Qu-anti-zation: Exploiting quantization artifacts for achieving adversarial outcomes

Sanghyun Hong, Michael - Andrei Panaitescu - Liess, Yigitcan Kaya, and Tudor Dumitras. Qu-anti-zation: Exploiting quantization artifacts for achieving adversarial outcomes. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, ...

2021
[25]

Hugging Face - the ai community building the future., 2024

Hugging Face . Hugging Face - the ai community building the future., 2024. URL https://huggingface.co/docs/transformers/index

2024
[26]

Deeppayload: Black-box backdoor attack on deep learning models through neural payload injection

Yuanchun Li, Jiayi Hua, Haoyu Wang, Chunyang Chen, and Yunxin Liu. Deeppayload: Black-box backdoor attack on deep learning models through neural payload injection. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 263--274. IEEE, 2021

2021
[27]

Awq: Activation-aware weight quantization for llm compression and acceleration

Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration. ArXiv preprint, abs/2306.00978, 2023. URL https://arxiv.org/abs/2306.00978

2023 arXiv
[28]

T ruthful QA : Measuring how models mimic human falsehoods

Stephanie Lin, Jacob Hilton, and Owain Evans. T ruthful QA : Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo...

2022 doi
[29]

Quantization backdoors to deep learning commercial frameworks

Hua Ma, Huming Qiu, Yansong Gao, Zhi Zhang, Alsharif Abuadbba, Minhui Xue, Anmin Fu, Jiliang Zhang, Said F Al-Sarawi, and Derek Abbott. Quantization backdoors to deep learning commercial frameworks. IEEE Transactions on Dependable and Secure Computing, 2023

2023
[30]

ollama: Get up and running with large language models

Jeffrey Morgan. ollama: Get up and running with large language models. https://github.com/ollama/ollama, 2023

2023
[31]

Understanding the threats of trojaned quantized neural network in model supply chains

Xudong Pan, Mi Zhang, Yifan Yan, and Min Yang. Understanding the threats of trojaned quantized neural network in model supply chains. In ACSAC , 2021

2021
[32]

Instruction tuning with GPT-4

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with GPT-4 . CoRR, 2023

2023
[33]

Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin - Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Au...

2024
[34]

Universal jailbreak backdoors from poisoned human feedback

Javier Rando and Florian Tram \` e r. Universal jailbreak backdoors from poisoned human feedback. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=GxCGsxiAaK

2024
[35]

Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein

Ali Shafahi, W. Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks on neural networks. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol \` o Cesa - Bianc...

2018
[36]

Targeted latent adversarial training improves robustness to persistent harmful behaviors in llms

Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, and Stephen Casper. Targeted latent adversarial training improves robustness to persistent harmful behaviors in llms. ArXiv...

2024 arXiv
[37]

On the exploitability of instruction tuning

Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. On the exploitability of instruction tuning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Syst...

2023
[38]

An embarrassingly simple approach for trojan attack in deep neural networks

Ruixiang Tang, Mengnan Du, Ninghao Liu, Fan Yang, and Xia Hu. An embarrassingly simple approach for trojan attack in deep neural networks. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and...

2020
[39]

Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li

Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. Decodingtrust: A comprehen...

2023
[40]

On the exploitability of reinforcement learning with human feedback for large language models

Jiongxiao Wang, Junlin Wu, Muhao Chen, Yevgeniy Vorobeychik, and Chaowei Xiao. On the exploitability of reinforcement learning with human feedback for large language models. CoRR, 2023 b

2023
[41]

Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does LLM safety training fail? In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on N...

2023
[42]

Badchain: Backdoor chain-of-thought prompting for large language models

Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran, and Bo Li. Badchain: Backdoor chain-of-thought prompting for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 202...

2024
[43]

Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models

Jiashu Xu, Mingyu Ma, Fei Wang, Chaowei Xiao, and Muhao Chen. Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American ...

2024
[44]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. ArXiv preprint, abs/2412.15115, 2024. URL https://arxiv.org/abs/2412.15115

2024 arXiv
[45]

Zico Kolter, and Matt Fredrikson

Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. CoRR, 2023

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Edelman, Zhaowei Zhang, Mario G \" u nther, Anton Korinek, Jos \' e Hern \' a ndez - Orallo, Lewis Hammond, Eric J

Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario G \" u nther, Anton Korinek, Jos \' e Hern \' a ndez - Orallo, Lewis Hammond, Eric J. Bigelow, Alexander Pan, Lauro Langosco, Tomasz Korbak, Heidi Zhang, Ruiqi Zho...

work page 2024

[3] [3]

Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J

Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. Program synthesis with large language models. CoRR, 2021

work page 2021

[4] [4]

Half-quadratic quantization of large machine learning models, 2023

Hicham Badri and Appu Shaji. Half-quadratic quantization of large machine learning models, 2023. URL https://mobiusml.github.io/hqq_blog/

work page 2023

[5] [5]

Mullins, and Nicolas Papernot

Mikel Bober - Irizar, Ilia Shumailov, Yiren Zhao, Robert D. Mullins, and Nicolas Papernot. Architectural backdoors in neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pages 24595--24604. IEEE , 2023. doi:10.1109/CVPR52729.2023.02356. URL https://doi.org/10.1109/CVPR527...

arXiv 2023

[6] [6]

Choquette - Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tram \` e r

Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette - Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tram \` e r. Poisoning web-scale training datasets is practical. CoRR, 2023

work page 2023

[7] [7]

Pappas, and Eric Wong

Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. CoRR, 2023

work page 2023

[8] [8]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad B...

work page 2021