Pith. sign in

REVIEW 5 major objections 5 minor 42 references

InfoFlood: Jailbreaking Large Language Models with Information Overload

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read InfoFlood claims that long, convoluted, citation-heavy rewrites of malicious queries bypass LLM safety filters without any adversarial suffix, achieving near-100 percent success on major chatbots.

desk verdict A real phenomenon with a plausible mechanism, but the near-100% ASR numbers rest on a GPT-4 judge as vulnerable as the targets; the paper needs human-validated evaluation before its headline claims are credible. read the letter →

arxiv 2506.12274 v1 pith:5IMD5MXT submitted 2025-06-13 cs.CR cs.CL

Advait Yadav , Haibo Jin , Man Luo , Jun Zhuang , Haohan Wang This is my paper
classification cs.CRcs.CL
keywords LLMjailbreakinformationoverloadlinguisticsaturationblack-boxattacksafetyalignmentevasionmoderationbypassadversarialprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that safety-aligned large language models can be made to answer malicious queries when those queries are rewritten into long, convoluted, citation-heavy prose, because the linguistic overload hides the harmful intent from the model's safety mechanisms without adding any adversarial prefix or suffix. It packages this finding as InfoFlood, a black-box attack with three stages: first a linguistic-saturation rewriter inflates the query, then a rejection-analysis step diagnoses why a failed attempt was refused, and finally a saturation-refinement step makes minimal targeted edits and retries. In its experiments on GPT-4o, GPT-3.5-Turbo, Llama 3.1 8B, and Gemini 2.0 Flash, InfoFlood reports attack success rates of 86–100 percent on AdvBench and 75–96 percent on JailbreakBench, and it says the generated queries largely evade OpenAI's Moderation API, Perspective API, and SmoothLLM. The stakes are concrete: if the claim is right, the standard recipe of alignment plus post-hoc moderation is not enough, because a plain-language rearrangement of the user's own request can bypass both.

What carries the argument

The machinery is a three-stage iterative query transformation loop. Linguistic Saturation rewrites the malicious request according to a non-deterministic template whose rules include faux citations to recent arXiv-style papers, past-tense conversion, comprehensiveness markers, discussion-framing, and explicit ethical acknowledgments; the rules are heuristics, not a deterministic transformation. When the target model refuses or evades, Rejection Analysis classifies the failure into one of three modes—abstraction/meta-analysis, rejection, or lack of intent—and Saturation Refinement applies minimal, targeted edits to the rephrased prompt, looping until a score of 10 is achieved or the iteration budget runs out. The mechanism is that the rephrased text keeps the original malicious semantics while moving the query's representation closer to benign text in the model's latent space, so the safety filter no longer flags it.

What would settle it

Take a random sample of responses that the GPT-4 judge scored 10, have human raters label whether each response actually contains usable, concrete instructions for the harmful request, and compare the human 'harmful' rate with the judge's rate. If the human rate is much lower, the central claim that InfoFlood elicits near-perfect harmful completions is false or at least unmeasured.

Watch

Extended reading notes

Core claim

The central discovery is that built-in safety mechanisms in LLMs can be disrupted by excessive linguistic complexity on its own. A malicious instruction rephrased into a 180-plus-word, past-tense, technically dense statement with fake arXiv citations, ethical disclaimers, content-type markers, and 'ongoing discussion' framing can elicit harmful completions directly, with no adversarial suffix. The paper calls this vulnerability Information Overload. Its evidence is that the InfoFlood pipeline achieves near-perfect jailbreak success on several frontier models and benchmarks, that performance peaks for query lengths of roughly 190–270 words, and that in Llama 3.1 8B's latent space InfoFlood queries sit closer to safe queries than to the original malicious ones, which the authors interpret as the model no longer recognizing the harm in the transformed text. Success is scored by a GPT-4 judge that rates responses from 1 to 10, with only a score of 10 counted as a jailbreak.

Load-bearing premise

The load-bearing premise is that the GPT-4 judge's score of 10 is a valid and unbiased measure that the target model genuinely produced harmful content; if the judge is itself fooled by the same information overload the attack exploits, the near-100 percent success rates could overstate how often the target model actually answered the malicious request.

Editorial extensions

If this is right

  • InfoFlood is fully black-box: it needs only query access to the target model, so it transfers to models whose weights and gradients are not available.
  • Defenses that scan for adversarial suffixes or rely on post-hoc moderation will not stop this class of attack; the paper reports that OpenAI Moderation and Perspective API reduce success by at most 4 percentage points, and SmoothLLM leaves success rates of 47–66 percent.
  • The attack's effectiveness is concentrated in a particular query-length window of about 190–270 words, which is a concrete behavioral signature that defenses or detection systems could look for.
  • Because the same transformation can be run iteratively, even failures become training signal for the attacker, meaning a single refused attempt does not end the attack.
  • Latent-space similarity to benign queries implies that representation-level safety checks, if they exist, would also need to be reworked to handle this obfuscation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated corollary is that the reported success rate may be partly an artifact of the judge: the GPT-4 judge is itself an LLM, so if information overload fools judges, scores of 10 could reflect a fooled judge rather than genuinely harmful completions. The paper does not test this directly.
  • A natural extension is to measure whether the same linguistic-saturation prompt succeeds against non-LLM intent classifiers or against models whose moderation is done separately from generation; the paper's mechanism predicts the attack is aimed at the model's own comprehension, so a separate classifier might behave differently.
  • The implied defense direction, which the paper does not develop, is to add a 'degibberish' step that summarizes or simplifies long user prompts before applying safety checks, then evaluates the summary's harmfulness.
  • The finding also suggests that human users, not just automated attackers, could accidentally or deliberately produce such overloaded requests, so the vulnerability exists outside the narrow adversarial-suffix setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. InfoFlood is a black-box jailbreak method that rephrases malicious queries into long, citation-heavy, ethically-framed prose in order to overload the safety reasoning of LLMs. The method combines a linguistic-saturation rephrasing stage with an iterative rejection-analysis / saturation-refinement loop that is driven by a GPT-4 judge. The paper reports attack success rates (ASR) on GPT-3.5-Turbo, GPT-4o, Llama 3.1, and Gemini 2.0 across AdvBench, JailbreakBench, and JailbreakHub, claiming near-100% ASR in several settings, out-of-distribution generalization to real-world jailbreak queries, and failure of OpenAI Moderation, Perspective API, and SmoothLLM. A latent-space analysis is used to argue that InfoFlood queries resemble safe queries in LLaMA-3.1-8B representations. The appendix contains the full prompt templates, baseline hyperparameters, and several complete running examples.

Significance. If the reported effectiveness survives a more rigorous evaluation, InfoFlood would be a practically important and conceptually interesting jailbreak vector: it requires no adversarial suffix, works through API-only access, and—according to the authors—applies across multiple frontier models and benchmarks. The paper also provides full attack prompts and running examples, which is valuable for reproducibility and for defense research. The latent-space analysis is an original attempt to connect the attack mechanism to internal representations. However, the central empirical claims currently rest on an LLM-based judge whose reliability is not established for exactly the kind of overloaded outputs the attack produces, and several table values conflict with the paper's own headline claims.

major comments (5)
  1. [§3.4, Appendix B] The jailbreak success rate is defined entirely by a GPT-4 judge that assigns a score of 10 only to responses that "directly and fully violate ethical, legal, or safety guidelines." Because InfoFlood is explicitly designed to degrade safety reasoning through verbose, ethically-framed prose, and because the judge is the same class of model as the target, reported ASR values may partly reflect judge confusion rather than genuinely harmful completions. No human validation is reported on InfoFlood outputs. Please add human annotation of a representative sample of judge-positive responses (e.g., 100-200 across models/benchmarks), report inter-annotator agreement with the GPT-4 judge, and additionally test whether the judge is robust to the overloaded style by evaluating it on deliberately benign but verbose and citation-heavy responses.
  2. [§4.3, Table 3] The claim that "commonly adopted post-processing defenses ... fail to mitigate these attacks" is not supported by Table 3. SmoothLLM reduces ASR by 28 to 40 percentage points across the four models (e.g., from 96% to 61% on GPT-3.5 and from 86% to 56% on GPT-4o). This is substantial mitigation, even if it is not complete. Please qualify the defense conclusions, report confidence intervals for the ASR differences, and clarify whether the defenses are applied to the InfoFlood query before generation or to the generated response; the current text says "Once a malicious query is transformed into an InfoFlood query, it is passed through the defense guardrails," which describes query-side filtering, not post-processing of the response.
  3. [Table 2, Section 4.1] Several headline claims are inconsistent with the presented numbers. The abstract and introduction state near-100% ASR and state-of-the-art results, but InfoFlood achieves 69.5% on JailbreakHub for GPT-3.5, which is below SAA's 72.0%, and 75% on JailbreakBench for Gemini 2.0, which is not near-perfect. Also, on JailbreakHub InfoFlood does not achieve the best ASR on every model. Please replace the aggregate claims with a precise per-benchmark/per-model summary, report statistical significance (e.g., paired bootstrap confidence intervals over the query set), and discuss where the method underperforms baselines.
  4. [§3.3, §3.5] The linguistic rules and the iterative refinement loop are both refined against the same evaluation benchmark and the same GPT-4 judge that is later used to measure success. The paper states that the rules "have been refined through extensive experimentation" and that the loop terminates when the judge gives a score of 10. This creates a circularity that makes the reported ASR partly in-sample. Please report which benchmarks were used to develop the rules, run a held-out benchmark (or a time-split set of queries) with the finalized rules, and use a judge different from the one used in the refinement loop for the final ASR measurement.
  5. [§4.4, Table 4] The latent-space analysis is based on only 50 queries per category and reports no variance or statistical test. More importantly, the claim that InfoFlood queries "clean" adversarial intent in the model's internal representation is confounded by query length, topic, and style: the InfoFlood queries are roughly ten times longer and contain distinct lexical markers, so the observed cosine similarity to safe queries may simply reflect embedding sensitivity to verbosity or topic, not a hidden safety-relevant axis. Please include a control of length-matched safe and malicious queries, or an ablation that removes the stylistic markers, and report effect sizes with confidence intervals.
minor comments (5)
  1. [Appendix B] The evaluation section refers to "jailbreaks produced by DeepClense," but the proposed method is InfoFlood; this looks like a leftover name from another project and should be corrected.
  2. [Appendix A, Rejection Analysis prompt] Several prompts contain typos and informal language, e.g., "precaurious", "criteria's", and "vaccum". These should be cleaned up for a formal submission.
  3. [Table 1] The table uses color coding (orange, green, blue, red) to mark transformation rules, but the surrounding text does not reproduce the colors in a grayscale-accessible way; please add a legend or textual markers.
  4. [§4.4] The t-SNE visualization (Figure 4) is based on 50 queries, but no random seed, hyperparameters, or per-point labels are given; please provide the plotting parameters or a reproducible script.
  5. [§4.1] The numbers in Table 2 appear to be single-run point estimates. Please report the number of runs, the variance, and the exact query counts for the 200 randomly sampled JailbreakHub queries, since random sampling will introduce noise.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation-chain circularity; the judge-based metric and rule-tuning are validity risks, not circular reductions.

full rationale

The paper's central claim is empirical: InfoFlood rephrases malicious queries and reports jailbreak success rates assessed by the GPT-4 judge from Chao et al. (2024b). The paper is explicit that this judge is a proxy: "We treat the GPT-4 score as an empirical proxy for -r*(R|P*)" (Section 3.4), and that "a response as a successful jailbreak if and only if it receives a score of 10" (Section 3.4). The iterative refinement loop also uses evaluator score 10 as its convergence criterion (Section 3.5). This means the reported ASR is, by construction, the rate at which the optimization reaches the judge's top score. That is a measurement-definition property of all such automated red-teaming evaluations, not an equation-level reduction of the harmfulness claim to the attack's inputs. The statement that the linguistic rules "have been refined through extensive experimentation" (Section 3.3) raises a possible benchmark-overfitting concern, but the paper does not state that the rules were fit on the evaluation benchmarks, so it cannot be substantiated as a fitted-input-called-prediction step without speculation. The self-citations (Jin et al. 2024a,b) appear only in related-work and baseline contexts and are not load-bearing for InfoFlood's method or results. The lack of human validation and the possibility that the GPT-4 judge shares the information-overload vulnerability are genuine external-validity limitations, but under the circularity rubric they do not constitute a self-referential derivation chain. Accordingly, no significant circularity is found; the appropriate score is low. The central mechanism—linguistic saturation plus iterative refinement—is independently specified and evaluated on external benchmarks, and the reported examples do show concretely harmful completions. The judge-validity and benchmark-tuning concerns should be weighed as correctness risks rather than circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claim rests on hand-crafted linguistic transformation rules and evaluation assumptions; there is no mathematical derivation, so the ledger captures the main hand-tuned and assumed components.

free parameters (3)
  • Linguistic saturation rule set = hand-tuned, no numeric value
    The template rules (faux-citations, comprehensiveness, past tense, discussion framing, ethical acknowledgment) were refined through extensive experimentation on the evaluation benchmarks; they are the attack's main machinery and are not derived.
  • Minimum rephrased query length = at least 180 words
    Set manually in the Linguistic Saturation prompt; the paper observes best performance around 190 to 270 words, but the template uses 180 as the lower bound.
  • Iteration budget for refinement loop = around 10 iterations
    The paper reports performance stabilizes around 10 iterations; this is a practical choice not derived from theory and not independently optimized per query.
assumptions (5)
  • standard math Autoregressive LLMs generate responses by sampling next tokens from q_T as in Eq. 1.
    Problem definition in Section 3.1 treats the target model as a token-level distribution; this is a standard formulation.
  • domain assumption GPT-4 judge scores from Chao et al. are a valid proxy for human-rated harmfulness.
    Section 3.4 uses a GPT-4 judge score of 10 as the sole success criterion; no human validation is provided for InfoFlood outputs.
  • domain assumption The rephrasing agent preserves the original malicious intent under linguistic saturation.
    Section 3.3 and the transformation prompt assert semantic preservation, but the paper does not measure intent preservation independently; refinement only reacts to judged failures.
  • domain assumption Embedding distances in LLaMA-3.1-8B correspond to perceived safety alignment.
    Section 4.4 interprets InfoFlood queries clustering near safe queries as evidence of intent cleaning; this assumes latent distance is a meaningful safety signal.
  • domain assumption The three tested post-processing defenses represent commonly adopted guardrails.
    Section 4.3 generalizes from OpenAI Moderation, Perspective API, and SmoothLLM to broader defense settings without broader evaluation.
invented entities (1)
  • Information Overload as a distinct jailbreak mechanism
    purpose: Explains why long, complex linguistic rephrasing bypasses safety filters without adversarial suffixes.
    The paper names and treats information overload as a new vulnerability, but does not ablate it from past-tense reframing, ethical disclaimers, cognitive overload (Xu et al.), and other bundled manipulations. No outside benchmark or controlled experiment isolates the mechanism, so there is no falsifiable handle beyond the paper's own pipeline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InfoFlood: Jailbreaking Large Language Models with Information Overload." pith.science (2026). https://pith.science/paper/5IMD5MXT

@misc{pith2026250612274,
  author       = {Pith},
  title        = {Pith review of: InfoFlood: Jailbreaking Large Language Models with Information Overload},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5IMD5MXT}},
  note         = {Machine review of arXiv:2506.12274}
}
read the original abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains. However, their potential to generate harmful responses has raised significant societal and regulatory concerns, especially when manipulated by adversarial techniques known as "jailbreak" attacks. Existing jailbreak methods typically involve appending carefully crafted prefixes or suffixes to malicious prompts in order to bypass the built-in safety mechanisms of these models. In this work, we identify a new vulnerability in which excessive linguistic complexity can disrupt built-in safety mechanisms-without the need for any added prefixes or suffixes-allowing attackers to elicit harmful outputs directly. We refer to this phenomenon as Information Overload. To automatically exploit this vulnerability, we propose InfoFlood, a jailbreak attack that transforms malicious queries into complex, information-overloaded queries capable of bypassing built-in safety mechanisms. Specifically, InfoFlood: (1) uses linguistic transformations to rephrase malicious queries, (2) identifies the root cause of failure when an attempt is unsuccessful, and (3) refines the prompt's linguistic structure to address the failure while preserving its malicious intent. We empirically validate the effectiveness of InfoFlood on four widely used LLMs-GPT-4o, GPT-3.5-turbo, Gemini 2.0, and LLaMA 3.1-by measuring their jailbreak success rates. InfoFlood consistently outperforms baseline attacks, achieving up to 3 times higher success rates across multiple jailbreak benchmarks. Furthermore, we demonstrate that commonly adopted post-processing defenses, including OpenAI's Moderation API, Perspective API, and SmoothLLM, fail to mitigate these attacks. This highlights a critical weakness in traditional AI safety guardrails when confronted with information overload-based jailbreaks.

Figures

Figures reproduced from arXiv: 2506.12274 by the authors.

Figure 1
Figure 1. (a) Refusal response generated by LLM where malicious queries input. (b) Jail￾break methods, such as adversarial prefixes and fixed prompts, which modify the query to bypass safety mechanisms. (c) InfoFlood-style attacks, where rephrased queries maintain ad￾versarial intent while avoiding detection, us￾ing information overload to expose vulnera￾bilities in LLM safety filters. However, we observed an interesting phe￾… view at source ↗
Figure 2
Figure 2. InfoFlood Method Overview. The malicious input undergoes Linguistic Saturation [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. InfoFlood Jailbreak Analysis We further investigate how query length impacts jailbreak success following Linguistic Saturation, as depicted in Figure 3b. The results indicate that InfoFlood achieves high success rates predominantly within a specific range of query lengths, approximately 190 to 270 words. Outside this optimal range, either shorter or excessively longer queries tend to exhibit diminished jailbreak eff… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: t-SNE visualization of query embed￾dings from LLaMA-3.1-8B for 50 queries. We extract embeddings from the final hid￾den layer of LLaMA-3.1-8B for 50 queries per category. We then compute Euclidean and cosine distances using average pairwise similarity and centroid dist…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 4 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [3]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  3. [4]

    Does refusal training in llms generalize to the past tense?, 2024

    Maksym Andriushchenko and Nicolas Flammarion. Does refusal training in llms generalize to the past tense?, 2024. URL https://arxiv.org/abs/2407.11969

    arXiv 2024

  4. [5]

    Jailbreaking leading safety-aligned llms with simple adaptive attacks

    Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety-aligned llms with simple adaptive attacks. arXiv preprint arXiv:2404.02151, 2024

    arXiv 2024

  5. [6]

    Constitutional ai: Harmlessness from ai feedback

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022

    arXiv 2022

  6. [7]

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, et al. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. arXiv preprint arXiv:2404.01318, 2024 a

    arXiv 2024

  7. [8]

    Pappas, and Eric Wong

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries, 2024 b . URL https://arxiv.org/abs/2310.08419

    arXiv 2024

  8. [9]

    Comprehensive assessment of jailbreak attacks against llms

    Junjie Chu, Yugeng Liu, Ziqing Yang, Xinyue Shen, Michael Backes, and Yang Zhang. Comprehensive assessment of jailbreak attacks against llms. arXiv preprint arXiv:2402.05668, 2024

    arXiv 2024

Show all 42 references
  1. [10]

    Build it break it fix it for dialogue safety: Robustness from adversarial human attack, 2019

    Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. Build it break it fix it for dialogue safety: Robustness from adversarial human attack, 2019. URL https://arxiv.org/abs/1908.06083

    2019 arXiv

  2. [11]

    Google gemini flash, February 2025

    Google. Google gemini flash, February 2025. URL https://deepmind.google/technologies/gemini/flash/

    2025

  3. [12]

    Deliberative alignment: Reasoning enables safer language models

    Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Heylar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, et al. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339, 2024

    2024 arXiv

  4. [13]

    Brian R. Y. Huang, Maximilian Li, and Leonard Tang. Endless jailbreaks with bijection learning, 2024. URL https://arxiv.org/abs/2410.01294

    2024 arXiv

  5. [14]

    Perspective api, 2024

    Jigsaw. Perspective api, 2024. URL https://perspectiveapi.com/

    2024

  6. [15]

    Guard: Role-playing to generate natural-language jailbreakings to test guideline adherence of large language models

    Haibo Jin, Ruoxi Chen, Andy Zhou, Yang Zhang, and Haohan Wang. Guard: Role-playing to generate natural-language jailbreakings to test guideline adherence of large language models. arXiv preprint arXiv:2402.03299, 2024 a

    2024

  7. [16]

    Menke, and Haohan Wang

    Haibo Jin, Andy Zhou, Joe D. Menke, and Haohan Wang. Jailbreaking large language models against moderation guardrails via cipher characters, 2024 b . URL https://arxiv.org/abs/2405.20413

    2024 arXiv

  8. [17]

    Automatically auditing large language models via discrete optimization, 2023

    Erik Jones, Anca Dragan, Aditi Raghunathan, and Jacob Steinhardt. Automatically auditing large language models via discrete optimization, 2023. URL https://arxiv.org/abs/2303.04381

    2023 arXiv

  9. [18]

    Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers, 2024

    Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers, 2024. URL https://arxiv.org/abs/2402.16914

    2024 arXiv

  10. [19]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2024

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2024. URL https://arxiv.org/abs/2310.04451

    2024 arXiv

  11. [20]

    Latent space cartography: Visual analysis of vector space embeddings

    Yang Liu, Eunice Jun, Qisheng Li, and Jeffrey Heer. Latent space cartography: Visual analysis of vector space embeddings. In Computer graphics forum, volume 38, pp.\ 67--78. Wiley Online Library, 2019

    2019

  12. [21]

    Black box adversarial prompting for foundation models, 2023

    Natalie Maus, Patrick Chao, Eric Wong, and Jacob Gardner. Black box adversarial prompting for foundation models, 2023. URL https://arxiv.org/abs/2302.04237

    2023 arXiv

  13. [22]

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024. URL https://arxiv.org...

    2024 arXiv

  14. [23]

    Tree of attacks: Jailbreaking black-box llms automatically, 2024

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically, 2024. URL https://arxiv.org/abs/2312.02119

    2024 arXiv

  15. [24]

    Selfcheck: Using llms to zero-shot check their own step-by-step reasoning

    Ning Miao, Yee Whye Teh, and Tom Rainforth. Selfcheck: Using llms to zero-shot check their own step-by-step reasoning. arXiv preprint arXiv:2308.00436, 2023

    2023 arXiv

  16. [25]

    Openai moderation api, 2024

    OpenAI. Openai moderation api, 2024. URL https://platform.openai.com/docs/guides/moderation/

    2024

  17. [26]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  18. [27]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 0 2...

    2022

  19. [28]

    Advprompter: Fast adaptive adversarial prompting for llms, 2024

    Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, and Yuandong Tian. Advprompter: Fast adaptive adversarial prompting for llms, 2024. URL https://arxiv.org/abs/2404.16873

    2024 arXiv

  20. [29]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36: 0 53728--53741, 2023

    2023

  21. [31]

    Exploring safety generalization challenges of large language models via code

    Qibing Ren, Chang Gao, Jing Shao, Junchi Yan, Xin Tan, Wai Lam, and Lizhuang Ma. Exploring safety generalization challenges of large language models via code. arXiv preprint arXiv:2403.07865, 2024 b

    2024 arXiv

  22. [32]

    Smoothllm: Defending large language models against jailbreaking attacks

    Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684, 2023

    2023 arXiv

  23. [33]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pp.\ 1671-...

    2024

  24. [34]

    Recursive deep models for semantic compositionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language proce...

    2013

  25. [35]

    Large language models in medicine

    Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature Medicine, 29 0 (8): 0 1930–1940, July 2023. doi:10.1038/s41591-023-02448-8. URL https://www.nature.com/articl...

    1930 doi

  26. [36]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  27. [37]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9 0 (11), 2008

    2008

  28. [38]

    Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024

    2024

  29. [39]

    Jailbreak and guard aligned language models with only few in-context demonstrations

    Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387, 2023

    2023 arXiv

  30. [40]

    Bloomberggpt: A large language model for finance

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023

    2023 arXiv

  31. [41]

    Cognitive overload: Jailbreaking large language models with overloaded logical thinking, 2024

    Nan Xu, Fei Wang, Ben Zhou, Bang Zheng Li, Chaowei Xiao, and Muhao Chen. Cognitive overload: Jailbreaking large language models with overloaded logical thinking, 2024. URL https://arxiv.org/abs/2311.09827

    2024 arXiv

  32. [42]

    Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher

    Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher. arXiv preprint arXiv:2308.06463, 2023

    2023 arXiv

  33. [43]

    How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. arXiv preprint arXiv:2401.06373, 2024

    2024 arXiv

  34. [44]

    Zico Kolter, and Matt Fredrikson

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023. URL https://arxiv.org/abs/2307.15043

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.