Pith. sign in

REVIEW 9 cited by

PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.04528 v5 pith:DHA4TDJ7 submitted 2023-06-07 cs.CL cs.CRcs.LG

Kaijie Zhu , Jindong Wang , Jiaheng Zhou , Zichen Wang , Hao Chen , Yidong Wang , Linyi Yang , Wei Ye
show 3 more authors
Yue Zhang Neil Zhenqiang Gong Xing Xie
This is my paper
classification cs.CLcs.CRcs.LG
keywords promptsadversarialrobustnessanalysislanguagellmsacrosscomprehensive
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. In response to this vital need, we introduce PromptRobust, a robustness benchmark designed to measure LLMs' resilience to adversarial prompts. This study uses a plethora of adversarial textual attacks targeting prompts across multiple levels: character, word, sentence, and semantic. The adversarial prompts, crafted to mimic plausible user errors like typos or synonyms, aim to evaluate how slight deviations can affect LLM outcomes while maintaining semantic integrity. These prompts are then employed in diverse tasks including sentiment analysis, natural language inference, reading comprehension, machine translation, and math problem-solving. Our study generates 4,788 adversarial prompts, meticulously evaluated over 8 tasks and 13 datasets. Our findings demonstrate that contemporary LLMs are not robust to adversarial prompts. Furthermore, we present a comprehensive analysis to understand the mystery behind prompt robustness and its transferability. We then offer insightful robustness analysis and pragmatic recommendations for prompt composition, beneficial to both researchers and everyday users.

Discussion (0). Sign in to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 50 citations worldwide. Full citation record

  1. Imprompt: A Language Framework for Prompt Programming

    cs.AI 2026-07 conditional novelty 7.0 of 10

    Prompt programs can be typed like programs and compiled in different ways; Imprompt formalizes two such compilers and tests them on tax-law reasoning and data-labeling tasks.

  2. Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks

    cs.SE 2025-07 conditional novelty 6.0 of 10

    Open-source 7B LLMs frequently produce requested C vulnerabilities when explicitly prompted, but the reported rates exclude most model outputs and the claimed persona effects are inconsistent.

  3. Format Sensitivity Index: Token-Controlled Prompt Wrapper Robustness and Schema Compliance in LLM Benchmarking

    cs.AI 2026-05 conditional novelty 5.5 of 10

    Across 140k generations, wrapper-only formatting changes accuracy by over 30× across models, largely via parseability failures, so single-wrapper scores are statistically fragile.

  4. Automated Framework to Evaluate and Harden LLM System Instructions against Encoding Attacks

    cs.CR 2026-04 conditional novelty 5.0 of 10

    Refusal-aligned LLMs leak system instructions under encoding/serialization prompts at high rates, and one-shot CoT instruction reshaping substantially reduces that leakage without retraining.

  5. Agent Identity Evals: Measuring Agentic Identity

    cs.AI 2025-07 conditional novelty 5.0 of 10

    Introduces Agent Identity Evals (AIE), five similarity-based metrics for LMA identity stability, with pilot experiments showing identifiability always at zero and no statistical support.

  6. Investigating the Robustness of Retrieval-Augmented Generation at the Query Level

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Retrieval-augmented generation performance drops noticeably under minor query perturbations, with end-to-end results often tracking retriever behavior.

  7. Find Before You Fine-Tune: A Diagnostic Study of Small LLMs for Cybersecurity QA

    cs.CL 2026-07 conditional novelty 4.0 of 10

    Pre-fine-tuning scores on a three-task diagnostic can predict the direction of post-fine-tuning change in small LLMs for cybersecurity QA, but not the magnitude or rank-preservation, which is regime-dependent.

  8. When Generic Prompt Improvements Hurt: Evaluation-Driven Iteration for LLM Applications

    cs.CL 2026-01 conditional novelty 3.0 of 10

    Adding generic prompt rules to task-specific LLM prompts is not monotonic: in 15-20 case local suites, Llama 3 and Qwen 2.5 sometimes pass fewer extraction and RAG checks, so prompt changes should be tested per task.

  9. Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey categorizing prompt-based attacks on LLMs into four classes and proposing aspirational goals of un-distillable, un-finetunable, and un-editable models.

Pith tools