REVIEW 4 major objections 8 minor 25 references
Fast Proxies for LLM Robustness Evaluation
T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Cheap proxy attacks—especially simply prompting the model—predict the attack success rate of an expensive six-attack red-teamer ensemble with rank correlation up to 0.94 across 33 open LLMs.
desk verdict Direct prompting's rank correlation with a six-attack ensemble is real but inflated by the unrobust tail; the paper's own Section 4.4 concedes the proxy is erratic exactly where it matters. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the synthetic red-teamer ensemble: six named attacks generating 727 jailbreak candidates per harmful prompt in a many-trial setting, with success defined by both HarmBench's finetuned Llama-2-13B classifier and LlamaGuard 3 8B flagging the response. This ensemble supplies the ground-truth robustness values that the proxies are tested against. The proxies are direct prompting (one greedy generation), prefilling (injecting an affirmative response prefix), and an embedding-space attack using signed gradient descent on continuous token embeddings. The comparison machinery is a set of three correlation coefficients—Pearson for linear fit, Spearman and Kendall for rank agreement—computed between proxy ASR and ensemble ASR over a zoo of 33 sub-10B models.
What would settle it
Run the same proxy-versus-ensemble comparison on models larger than 10B or on an attack suite the ensemble does not contain (for example, multi-turn conversational jailbreaks or transfer attacks from a held-out attack family), and check whether the Spearman rank correlation between direct-prompt ASR and full-attack ASR stays near 0.94; a drop below about 0.8 would show the proxy's ranking power does not generalize.
Extended reading notes
Core claim
At its core, the paper claims that the attack success rate (ASR) of a cheap proxy—especially direct prompting—predicts the ASR of a synthetic red-teamer composed of six attack algorithms (AmpleGCG, AutoDAN, BEAST, GCG, human jailbreak templates, and PAIR) that collectively try 727 candidate prompts per harmful example. The headline numbers are a Spearman rank correlation of 0.94 across six model families, 0.93 across Llama 3 variants, and 0.98 across circuit-breaker training checkpoints, alongside a Pearson correlation of 0.87 for embedding-space attacks across families. Because a full ensemble run costs about 1,731 seconds per prompt on an H100 while direct prompting takes about one second, a model zoo can be ranked a thousand times faster. The paper does not claim direct prompting is a strong attack; it claims it is a strong predictor of relative robustness, provided enough prompts (more than 50) are used.
Load-bearing premise
The whole comparison depends on the synthetic red-teamer ensemble being a faithful stand-in for real-world adversarial robustness; if real attackers use algorithms, prompt distributions, model sizes, or success criteria outside this ensemble, the proxy rankings could diverge from actual safety.
Editorial extensions
If this is right
- A model zoo can be ranked by robustness at about one second per prompt rather than roughly half an hour of H100 time per prompt, making routine safety comparisons feasible for most research groups.
- Direct prompting is a valid ranking proxy for cross-family model selection when at least 50 prompts are used, meaning the simplest possible baseline can replace expensive attack ensembles for relative robustness judgments.
- Embedding-space attacks provide better linear predictions and work with fewer prompts, making them the better choice when absolute attack success rates are needed rather than just a ranking.
- Proxy attack success rates track robustness gains during circuit-breaker and continuous adversarial training, so checkpoints can be selected during defense training without running the full attack suite at every step.
- As models become more robust, direct-prompt proxies will need more prompts to avoid the sampling noise that makes small non-zero success rates look like exactly zero.
Reading between the lines
- An extension left implicit: if rank correlations this high survive on larger and closed models, proxy ranking could become a standard cheap pre-filter that decides which models receive expensive full red-teaming.
- Because both the ensemble success labels and proxy labels rely on the same two classifiers, shared classifier errors could inflate the reported correlations; re-judging a subset of responses by human raters or a single held-out judge would reveal how much of the correlation is classifier artifact.
- Direct prompting may work as a proxy largely because it measures baseline refusal calibration; this suggests it would transfer to new attacks that exploit refusal failures but could miss attacks that bypass refusal entirely through context or multi-turn manipulation.
- The paper's scaling analysis implies a budget rule the authors do not state explicitly: the number of prompts for a reliable direct-prompt proxy should grow as models get safer, because zero-ASR sampling noise dominates at low prompt counts.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes and evaluates three low-cost proxy attacks—direct prompting, prefilling, and embedding-space PGD—as predictors of LLM robustness, where robustness is measured by attack success rate (ASR) against a 'synthetic red-teamer' ensemble of six published attack algorithms (AmpleGCG, AutoDAN, BEAST, GCG, HumanJailbreaks, PAIR) run in the many-trial setting (727 candidates per prompt) on 300 AdvBench prompts across 33 sub-10B open-source models (Sections 2 and 4; Appendix B). The headline finding is that across six model families, direct prompting achieves Spearman r_s=0.94 (Kendall tau=0.83) and embedding-space attacks achieve Pearson r_p=0.87, at roughly three orders of magnitude lower compute (Appendix B: 1,731 s vs. 1-5 s per prompt). The paper additionally reports within-family correlations over Llama-3 and Mistral variants (Figures 1 and 5), fine-tuning trajectories under circuit-breaker and continuous-adversarial training (Figures 3 and 7), and how the correlations scale with the number of prompts (Figure 4). The authors conclude that direct prompting is a strong ranking baseline provided roughly 50 or more prompts are used.
Significance. If substantiated, the central claim is practically valuable: cheap proxy ASRs could allow checkpoint selection during safety fine-tuning and cross-model robustness comparison at a small fraction of the cost of running a full attack suite. The study's strengths are its scale and transparency—more than 7M jailbreak attempts, detailed attack hyperparameters in Appendix B, simple and reproducible computations with no parameters fitted to the ensemble ASR, and an explicit limitations paragraph (Section 4.5). The headline claims are falsifiable and stated as quantitative correlations. The significance is conditional on two open questions: whether the synthetic red-teamer is externally valid as a proxy for real-world attackers, and whether the headline rank correlations reflect informative ordering within the robust subpopulation rather than only separation of a few extremely unrobust models. As a measurement study that proposes no new attack or defense, the paper's value lies in its large empirical dataset and its concrete, testable claim about proxy-based evaluation.
major comments (4)
- [Section 4.2, Figure 2] Section 4.2 and Figure 2 report r_s=0.94 between direct-prompting ASR and ensemble ASR across 33 models, but the figure and text show that direct ASR is 'generally below 5%, except for models that are extremely unrobust' (the inset only spans 0-0.05). If the safety-tuned models mostly have tied or near-zero direct ASR, Spearman's rank correlation with average-rank tie handling can remain high purely from ordering the few extremely unrobust models, while providing no information about the ordering within the robust subpopulation—the regime that matters for deployment decisions and for checkpoint selection during safety fine-tuning (Section 4.3). Section 4.4 concedes that direct ASR becomes 'erratic' for robust models and that more prompts are needed as models become more robust. The authors should report the number of tied direct-ASR values, recompute rank correlations restricted to models with ensemble ASR below a threshold (e.g., 0.5), and provide bootstrap confidence intervals for r_s; without these, the abstract's claim that direct prompting 'predict[s] attack success rates well' is not demonstrated for the safety-relevant regime. The clustering of models within six families also means the effective sample size is smaller than 33, and the cross-family correlation may partly reflect family-level differences.
- [Abstract; Sections 4.1–4.4] The abstract states that the proxies 'predict attack success rates well' and 'allow us to estimate a model's robustness,' but all reported correlations are descriptive in-sample statistics computed on the same models used to motivate the claim; no confidence intervals, significance tests, or out-of-sample validation are provided. The sample sizes are small (roughly 9-33 models depending on the comparison), each ASR value is itself an estimate over 300 prompts, and the fine-tuning-trajectory correlations (Figures 3 and 7) treat successive checkpoints of a single training run as independent observations, which inflates the effective sample size for those comparisons. The authors should report bootstrap confidence intervals for r_p, r_s, and tau, add a leave-one-family-out or split-half analysis to support the 'predict/estimate' language, and discuss the multiple-comparison issue that arises from selecting the best proxy in each setting (e.g., the crossover in Figure 4 at 50 prompts is reported without error bars).
- [Sections 2 and 4.5] The load-bearing premise of the paper is that the synthetic red-teamer—six attack algorithms, 727 candidates per prompt, run on 300 AdvBench prompts with greedy decoding—is a faithful proxy for real-world adversarial robustness. The reported correlations demonstrate only that the proxies predict this particular ensemble; Section 4.5 concedes that generalization to other attack algorithms, prompt distributions, and model sizes is unvalidated. Within the scope of a revision, the authors should test sensitivity of the cross-family ranking to the prompt set (e.g., split AdvBench into two halves) and to ensemble composition (e.g., leave-one-attack-out), and the abstract should phrase the claim as predicting the simulated ensemble rather than 'real-world robustness.'
- [Appendix B] Every ASR value in the paper depends on the conjunction of two harm classifiers—HarmBench's fine-tuned Llama-2-13B classifier and LlamaGuard-3-8B—both of which must flag the response as harmful (Appendix B). The agreement rate and error rates of these two classifiers are not reported, yet any classifier error propagates into every ASR and hence into every correlation. The authors should report the classifiers' agreement on a sample of responses and a sensitivity analysis of the headline correlations computed with each classifier alone.
minor comments (8)
- [Abstract] In the abstract, the sentence 'it and embedding-space attacks can predict attack success rates well, achieving r_p=0.87 (linear) and r_s=0.94 (Spearman rank)' reports the two headline coefficients without attributing them: r_p=0.87 is the embedding-space Pearson correlation (Figure 2) and r_s=0.94 is the direct-prompting Spearman correlation (Figure 2); the sentence should attribute each coefficient to its proxy.
- [Abstract / Appendix B] The 'three orders of magnitude' cost reduction in the abstract is accurate for direct prompting and prefilling (about 1 s vs. 1,731 s per prompt) but is roughly 2.5 orders of magnitude for embedding-space attacks (about 5 s per prompt); the claim should be qualified per proxy.
- [Appendix C.2, Figure 6] The caption of Figure 6 in Appendix C.2 says 'different variants of Mistral 7B Instruct' and the following sentence 'These model families were selected due to their popularity and resulting large number of versions' appears to be a leftover fragment, since the figure reproduces the Llama-3 variants of Figure 1; the caption and text should be corrected.
- [Section 1, Table 2] The manuscript states in Sections 1 and 4.5 that 33 models were evaluated, but Table 2 lists 24 base models; the nine circuit-breaker checkpoints of Section 4.3 and the three continuous-adversarial-training checkpoints are not listed in Table 2, so the model count should be reconciled either by extending Table 2 or by defining how the count is obtained.
- [Appendix C.3, Figure 7] Figure 7 reports r_s=r_p=tau=1.00 on only four checkpoints; with four points, any strictly monotone sequence of values yields perfect rank correlations, so these values should carry an explicit small-sample caveat wherever they are cited (Section 4.3).
- [Figures 1–5] In several figure legends (Figures 1-5), the third reported coefficient appears as '=0.82' or '=0.83' with the 'tau=' label missing; restoring the label would remove ambiguity.
- [Section 4.4] Section 4.4 contains a typo ('For50 or more prompts' should be 'For 50 or more prompts').
- [General] The paper does not include a code or data availability statement; releasing the per-model, per-attack ASR table would substantially aid reproducibility and allow readers to compute the confidence intervals and subpopulation analyses requested above.
Circularity Check
No significant circularity: the proxy-to-ensemble correlations are descriptive and independent of any fitted target.
full rationale
The paper's central claim is an empirical correlation between independently executed proxy ASRs (direct, prefilling, embedding-space) and the ASR of a six-attack synthetic red-teamer ensemble. No parameter is fitted to the ensemble ASR; the reported Pearson, Spearman, and Kendall coefficients are descriptive statistics over a fixed set of 33 models. The proxies are not members of the target ensemble, so there is no self-referential loop of the kind that would make a 'prediction' reduce to a fit. The self-citations (Schwinn et al. for the embedding-space attack, Xhonneux et al. for continuous adversarial training) are source/implementation references and are not load-bearing: the embedding-space attack is an independent proxy measurement, and the correlation would not be definitionally forced if those citations were replaced. Section 4.4's recommendation that '50 or more prompts' be used is a post hoc empirical scaling observation from the same data, not a fitted parameter renamed as a prediction, and the headline correlations are not derived from that recommendation. The shared success judge (HarmBench classifier plus LlamaGuard 3) could in principle inflate agreement through shared measurement error, and Section 4.5 honestly concedes that generalization to other attacks and model sizes is unvalidated; both are external-validity/measurement concerns, not circularity. The derivation chain is therefore self-contained with respect to the stated ensemble-ASR target.
Assumptions & free parameters
free parameters (4)
- Embedding-space PGD learning rate alpha =
0.01
- Embedding-space PGD L2 radius =
1, normalized by mean L2 embedding norm
- Embedding-space PGD steps =
100
- Direct-prompt prompt threshold =
>50 prompts
assumptions (4)
- standard math Pearson, Spearman, and Kendall correlation coefficients are the accepted summary of association between proxy ASR and ensemble ASR.
- domain assumption A jailbreak attempt is successful only if both HarmBench's finetuned Llama-2-13B classifier and LlamaGuard 3 8B flag the response as harmful.
- domain assumption The six-attack, 727-candidate many-trial ensemble is a strong proxy for real-world red-teaming.
- domain assumption The 300 AdvBench prompts and 33 sub-10B models represent deployment-relevant harmful requests and model diversity.
Cite this review
Pith. "Pith review of Fast Proxies for LLM Robustness Evaluation." pith.science (2026). https://pith.science/paper/XORO64FW
@misc{pith2026250210487,
author = {Pith},
title = {Pith review of: Fast Proxies for LLM Robustness Evaluation},
year = {2026},
howpublished = {\url{https://pith.science/paper/XORO64FW}},
note = {Machine review of arXiv:2502.10487}
}
abstract
Evaluating the robustness of LLMs to adversarial attacks is crucial for safe deployment, yet current red-teaming methods are often prohibitively expensive. We compare the ability of fast proxy metrics to predict the real-world robustness of an LLM against a simulated attacker ensemble. This allows us to estimate a model's robustness to computationally expensive attacks without requiring runs of the attacks themselves. Specifically, we consider gradient-descent-based embedding-space attacks, prefilling attacks, and direct prompting. Even though direct prompting in particular does not achieve high ASR, we find that it and embedding-space attacks can predict attack success rates well, achieving $r_p=0.87$ (linear) and $r_s=0.94$ (Spearman rank) correlations with the full attack ensemble while reducing computational cost by three orders of magnitude.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Marah Abdin, Jyoti Aneja, Harkirat Behl, S ´ebastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 techni- cal report. arXiv preprint arXiv:2412.08905,
-
[3]
semanticscholar.org/CorpusID:268232499
URL https://api. semanticscholar.org/CorpusID:268232499. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609,
-
[5]
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783,
-
[6]
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7B. arXiv preprint arXiv:2310.06825,
-
[7]
Llm defenses are not robust to multi-turn human jailbreaks yet
Nathaniel Li, Ziwen Han, Ian Steneker, Willow Primack, Riley Goodside, Hugh Zhang, Zifan Wang, Cristina Menghini, and Summer Yue. Llm defenses are not robust to multi-turn human jailbreaks yet. arXiv preprint arXiv:2408.15221,
-
[8]
Zeyi Liao and Huan Sun. AmpleGCG: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms. arXiv preprint arXiv:2404.07921,
-
[9]
Autodan: Generating stealthy jailbreak prompts on aligned large language models
Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451,
-
[10]
HarmBench: A standardized evaluation framework for automated red teaming and robust refusal
Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.04249,
Show all 25 references
-
[11]
Vinu Sankar Sadasivan, Shoumik Saha, Gaurang Sriramanan, Priyatham Kattakinda, Atoosa Chegini, and Soheil Feizi
URL https://openai.com/index/ red-teaming-network/. Vinu Sankar Sadasivan, Shoumik Saha, Gaurang Sriramanan, Priyatham Kattakinda, Atoosa Chegini, and Soheil Feizi. Fast adversarial attacks on language models in one GPU minute. arXiv preprint arXiv:2402.15570,
-
[12]
Rain- bow teaming: Open-ended generation of diverse adversarial prompts
Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, et al. Rain- bow teaming: Open-ended generation of diverse adversarial prompts. arXiv preprint arXiv:2402.16822,
-
[13]
Adversarial attacks and defenses in large language models: Old and new threats
Leo Schwinn, David Dobre, Stephan G ¨unnemann, and Gauthier Gidel. Adversarial attacks and defenses in large language models: Old and new threats. arXiv preprint arXiv:2310.19737,
-
[14]
Soft prompt threats: Attacking safety alignment and unlearning in open-source LLMs through the embedding space
Leo Schwinn, David Dobre, Sophie Xhonneux, Gauthier Gidel, and Stephan Gunnemann. Soft prompt threats: Attacking safety alignment and unlearning in open-source LLMs through the embedding space. arXiv preprint arXiv:2402.09063,
-
[15]
Tar- geted latent adversarial training improves robustness to persistent harmful behaviors in llms.arXiv preprint arXiv:2407.15549,
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfield-Menell, and Stephen Casper. Tar- geted latent adversarial training improves robustness to persistent harmful behaviors in llms.arXi...
-
[16]
Gemma 2: Improving open language models at a practical size
5 Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram ´e, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118,
-
[17]
Hermes 3 technical report
Ryan Teknium, Jeffrey Quesnelle, and Chen Guang. Hermes 3 technical report. arXiv preprint arXiv:2408.11857,
-
[18]
Llama 2: Open founda- tion and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288,
-
[19]
Zephyr: Direct distillation of lm alignment
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Cl ´ementine Fourrier, Nathan Habib, et al. Zephyr: Direct distillation of lm alignment. arXiv preprint arXiv:2310.16944,
-
[20]
Bypassing the safety training of open-source llms with priming attacks
Jason Vega, Isha Chaudhary, Changming Xu, and Gagandeep Singh. Bypassing the safety training of open-source llms with priming attacks. arXiv preprint arXiv:2312.12321,
-
[21]
Efficient adversarial training in LLMs with continuous attacks
Sophie Xhonneux, Alessandro Sordoni, Stephan G ¨unnemann, Gauthier Gidel, and Leo Schwinn. Efficient adversarial training in LLMs with continuous attacks. arXiv preprint arXiv:2405.15589,
-
[22]
Universal and transferable adversarial attacks on aligned language models
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043,
-
[23]
We use bfloat16 quantization for all models
and N “ t75, 150, 225u steps. We use bfloat16 quantization for all models. 7 B H YPERPARAMETERS & EXPERIMENTAL DETAILS We run all attacks on all 300 harmful prompts from AdvBench (Zou et al., 2023), as included in HarmBench. A jailbreak attempt is counted as successful if both...
2023
-
[24]
250, use a batch size of 512, Top-K “ 256, and initialize using the string “x x x x x x x x x x x x x x x x x x x x
flag the model’s response as harmful. The hyperparameters for the attacks used in the ensemble and the proxy attacks are shown below. Where possible, attack implementations were sourced from the original authors’ GitHub reposi- tories; otherwise, we integrated a HarmBench impl...
2024
-
[25]
x x x x x x x x x x x x x x x x x x x x
During the attack, the victim model generates up to 256 tokens using greedy generation. If the conversations grow longer than the model’s context, we truncate the first non-system messages from the conversation until the conversation fits into the context window. The proxy att...
2024
-
[2023]
Jailbreaking black box large language models in twenty queries
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419,
-
[2024]
Jailbreaking leading safety- aligned LLMs with simple adaptive attacks
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety- aligned LLMs with simple adaptive attacks. arXiv preprint arXiv:2404.02151,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.