REVIEW 10 cited by
The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
As Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications, their safety concerns become critical areas of NLP research. This paper presents Safety and Over-Defensiveness Evaluation (SODE) benchmark: a collection of diverse safe and unsafe prompts with carefully designed evaluation methods that facilitate systematic evaluation, comparison, and analysis over 'safety' and 'over-defensiveness.' With SODE, we study a variety of LLM defense strategies over multiple state-of-the-art LLMs, which reveals several interesting and important findings, such as (a) the widely popular 'self-checking' techniques indeed improve the safety against unsafe inputs, but this comes at the cost of extreme over-defensiveness on the safe inputs, (b) providing a safety instruction along with in-context exemplars (of both safe and unsafe inputs) consistently improves safety and also mitigates undue over-defensiveness of the models, (c) providing contextual knowledge easily breaks the safety guardrails and makes the models more vulnerable to generating unsafe responses. Overall, our work reveals numerous such critical findings that we believe will pave the way and facilitate further research in improving the safety of LLMs.
Forward citations
Cited by 10 Pith papers
-
LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems
A systematic review that categorizes LLM threats, severity scores, and mitigations across development and operation life cycles and multiple deployment scenarios.
-
GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation
GRAIT selects and reweights refusal-training examples using gradient influence, reporting lower hallucination rates and better helpfulness scores than prior refusal-aware tuning baselines.
-
Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails
Aegis2.0 provides a commercially usable, human-annotated safety dataset with 24 risk categories, and models trained on it with parameter-efficient methods match WildGuard and beat Llama Guard 3.
-
Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Models
Token Highlighter identifies jailbreak-critical tokens using gradients of an affirmation loss and weakens them by shrinking their embeddings, lowering attack success on LLaMA-2 and Vicuna.
-
No Free Lunch for Defending Against Prefilling Attack by In-Context Learning
Adversative in-context examples ('Sure... However...') defend many LLMs against prefilling jailbreaks but cause over-refusal, so the defense trades safety for helpfulness.
-
The TIP of the Iceberg: Revealing a Hidden Class of Task-in-Prompt Adversarial Attacks on LLMs
Encoding forbidden content in ciphers, riddles, or code tasks lets attackers bypass safety filters in six current LLMs, and the PHRYGE benchmark measures how often this succeeds.
-
System Prompt Extraction Attacks and Defenses in Large Language Models
A benchmarking study shows that chain-of-thought, few-shot, and modified sandwich queries can recover LLM system prompts with high similarity-based success, and output filtering is the most reliable tested defense.
-
`Do as I say not as I do': A Semi-Automated Approach for Jailbreak Prompt Attack against Multimodal LLMs
A voice jailbreak that buries a forbidden question between benign prompts reportedly succeeds against Gemini 67 to 93 percent of the time, but the metric comes from the target model judging itself and is not reliable.
-
The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.
-
LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures
This survey categorizes attacks on large language models by lifecycle phase and maps them to prevention and detection defenses, concluding that only a few defenses are highly effective.
Discussion (0). Continue with ORCID to comment.