Pith. sign in

REVIEW 10 cited by

InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.22770 v3 pith:S2EXWZWR submitted 2024-10-30 cs.CL cs.AIcs.CR

Hao Li , Xiaogeng Liu This is my paper · ORCID
classification cs.CLcs.AIcs.CR
keywords promptmodelsover-defenseinjecguardinjectionattacksguardnotinject
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Prompt injection attacks pose a critical threat to large language models (LLMs), enabling goal hijacking and data leakage. Prompt guard models, though effective in defense, suffer from over-defense -- falsely flagging benign inputs as malicious due to trigger word bias. To address this issue, we introduce NotInject, an evaluation dataset that systematically measures over-defense across various prompt guard models. NotInject contains 339 benign samples enriched with trigger words common in prompt injection attacks, enabling fine-grained evaluation. Our results show that state-of-the-art models suffer from over-defense issues, with accuracy dropping close to random guessing levels (60%). To mitigate this, we propose InjecGuard, a novel prompt guard model that incorporates a new training strategy, Mitigating Over-defense for Free (MOF), which significantly reduces the bias on trigger words. InjecGuard demonstrates state-of-the-art performance on diverse benchmarks including NotInject, surpassing the existing best model by 30.8%, offering a robust and open-source solution for detecting prompt injection attacks. The code and datasets are released at https://github.com/leolee99/InjecGuard.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling

    cs.AI 2025-09 conditional novelty 7.0 of 10

    An activation-guided energy model plus MCMC sampling creates transferable direct prompt injection attacks in a black-box setting, reaching 49.6% average attack success across five LLMs.

  2. BASIS: Breach-Aware Selective Prompt Injection Shielding with Prefill Attention Probes

    cs.CR 2026-08 conditional novelty 6.0 of 10

    BASIS uses two linear probes on prefill attention to separately detect injection presence and predict whether the injection would actually compromise the model, refusing only when both fire.

  3. From Neural Intent to Cryptographic Authorization: Securing AI-Driven Enterprise Workflows

    cs.CR 2026-07 conditional novelty 6.0 of 10

    NCS gates every privileged AI-agent tool call behind an offline-signed, hash-chained instruction stream, cutting injected-parameter attacks to zero by construction.

  4. AgentWatcher: A Rule-based Prompt Injection Monitor

    cs.CR 2026-04 conditional novelty 6.0 of 10

    An attention-attribution plus rule-reasoning detector reduces prompt-injection success to near zero on agent and long-context benchmarks with small utility loss.

  5. CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement

    cs.CL 2025-05 conditional novelty 6.0 of 10

    CAPTURE, a context-aware benchmark, finds current prompt guardrails either miss context-aware attacks or over-block safe prompts, and a model trained on its data reduces both error types on the new test sets.

  6. Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation

    cs.CR 2025-02 conditional novelty 6.0 of 10

    A membership inference attack on RAG systems crafts natural yes/no questions from a target document to detect its presence in the datastore, achieving high AUC while evading guardrail detectors.

  7. Detection Method for Prompt Injection by Integrating Pre-trained Model and Heuristic Feature Engineering

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A dual-channel detector combining DeBERTa and heuristic rules is claimed to beat existing prompt injection detectors on three benchmarks and to reduce attack success on GLM-4, Llama 3, Qwen 2.5, and GPT-4o.

  8. PromptShield: Deployable Detection for Prompt Injection Attacks

    cs.CR 2025-01 conditional novelty 5.0 of 10

    PromptShield reports a 65.3% true positive rate at 0.1% false positive rate for prompt injection detection, more than six times the best prior model, on its own out-of-distribution evaluation split.

  9. JavelinGuard: Low-Cost Transformer Architectures for LLM Security

    cs.LG 2025-06 reject novelty 4.0 of 10

    A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.

  10. Multi-Stage Prompt Inference Attacks on Enterprise LLM Systems

    cs.CR 2025-07 reject novelty 3.0 of 10

    Multi-stage prompt inference attacks against enterprise LLMs are formalized and defenses are proposed, but the preprint gives no reproducible evidence for its central claims.

Pith tools