Pith. sign in

REVIEW 11 cited by

Defending Against Unforeseen Failure Modes with Latent Adversarial Training

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.05030 v6 pith:WE255U7D submitted 2024-03-08 cs.CR cs.AIcs.LG

Stephen Casper , Lennart Schulze , Oam Patel , Dylan Hadfield-Menell This is my paper
classification cs.CRcs.AIcs.LG
keywords adversarialfailuremodestrainingattacksdefendelicitlatent
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Despite extensive diagnostics and debugging by developers, AI systems sometimes exhibit harmful unintended behaviors. Finding and fixing these is challenging because the attack surface is so large -- it is not tractable to exhaustively search for inputs that may elicit harmful behaviors. Red-teaming and adversarial training (AT) are commonly used to improve robustness, however, they empirically struggle to fix failure modes that differ from the attacks used during training. In this work, we utilize latent adversarial training (LAT) to defend against vulnerabilities without leveraging knowledge of what they are or using inputs that elicit them. LAT makes use of the compressed, abstract, and structured latent representations of concepts that the network actually uses for prediction. Here, we use it to defend against failure modes without examples that elicit them. Specifically, we use LAT to remove backdoors and defend against held-out classes of adversarial attacks. We show in image classification, text classification, and text generation tasks that LAT usually improves both robustness to novel attacks and performance on clean data relative to AT. This suggests that LAT can be a promising tool for defending against failure modes that are not explicitly identified by developers.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering

    cs.CR 2026-02 conditional novelty 7.0 of 10

    TamperBench systematically benchmarks 21 open-weight LLMs against nine fine-tuning and representation-space attacks and finds all of them can be tampered into producing harmful output while retaining capability.

  2. Gradient Immunity: Null-Space Resistance to Malicious Fine-Tuning

    cs.CR 2026-08 conditional novelty 6.0 of 10

    Calibrating a null-space gate to fully cover the defender's harmful data keeps post-fine-tuning attack success at pre-release levels, but this is a coverage and calibration consequence rather than a tested defense on ...

  3. Estimating Rare Events in Language Models with Proper Evaluation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    An activation-space multilevel-splitting estimator (GA-AMLS) plus a finite, asymmetric proper loss (SPB) shifts rare-event probability estimation for small language models, with the best estimator depending on whether...

  4. Efficient Safety Alignment of Language Models via Latent Personality Traits

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Latent adversarial training on 66 harm-agnostic Big-Five personality statements yields near-zero HarmBench ASR across direct requests and five jailbreaks while preserving utility.

  5. From Refusal Geometry to Safety Geometry: Harmfulness--Refusal Coupling under Dynamic Adversarial Fine-Tuning

    cs.CR 2026-06 unverdicted novelty 6.0 of 10

    R2D2 fine-tuning of Mistral-7B transitions from high harmfulness–refusal coupling with collapsed utility to lower coupling with partial utility recovery and reopened jailbreaks; low coupling alone is not safety.

  6. RippleBench: Capturing Ripple Effects Using Existing Knowledge Repositories

    cs.AI 2025-12 conditional novelty 6.0 of 10

    RippleBench automatically generates questions at increasing semantic distance from unlearned topics and shows all eight tested unlearning methods degrade accuracy that recovers only slowly with distance.

  7. Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain

    cs.IR 2025-09 conditional novelty 6.0 of 10

    Misleading health documents in RAG context sharply lower LLM accuracy, and heavily helpful-biased retrieval pools restore it.

  8. SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression

    cs.CR 2025-06 conditional novelty 6.0 of 10

    A security-aware prompt compressor that reveals the hidden intent of jailbreak prompts and injects it into the system prompt reduces average attack success from 35% to 1% with negligible overhead.

  9. Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Combining suffix-window representation finetuning with an ActGrad-pruned surrogate cuts latent-adversarial-training FLOPs per step by 48.1% with only 0.0118% trainable parameters, while accepting higher attack success rates.

  10. AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMs

    cs.CV 2026-01 conditional novelty 5.0 of 10

    A cross-modal benchmark derived from MMVP shows VLMs and IGMs share several elementary visual-reasoning failure modes, with IGMs struggling most on explicit attribute-control prompts.

  11. Probing the Robustness of Large Language Models Safety to Latent Perturbations

    cs.LG 2025-06 conditional novelty 4.0 of 10

    Randomized noise injected into hidden layers bypasses safety refusals in 12 open LLMs, and layer-wise adversarial training on the resulting benchmark reduces the attack's success.

Pith tools