{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:KEN2E3SRXYAUAC5I77OG24TXSM","short_pith_number":"pith:KEN2E3SR","schema_version":"1.0","canonical_sha256":"511ba26e51be01400ba8ffdc6d7277930b43785a431959e95695c93a40175dc4","source":{"kind":"arxiv","id":"2506.09956","version":1},"attestation_state":"computed","paper":{"title":"LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CR","authors_text":"Ahmed Salem, Aideen Fay, Alex Apostolov, Andrew Paverd, Athar Mahboob, Benjamin Pannell, Byeonghyeon Kim, Chi-Huang Liu, Chun-Chih Kuo, Conor McCauley, Danyael Manlangit, Egor Zverev, Giovanni Cherubin, Hajin Choi, Haris Umair, Hyeonjin Lee, Jannis Weigend, Jo\\~ao Donato, Kai-Chieh Liao, Mark Russinovich, Masayuki Kawakita, Myeongjin Cho, Sahar Abdelnabi, Tran Huu Bach, Tsun-Han Chiang","submitted_at":"2025-06-11T17:30:07Z","abstract_excerpt":"Indirect Prompt Injection attacks exploit the inherent limitation of Large Language Models (LLMs) to distinguish between instructions and data in their inputs. Despite numerous defense proposals, the systematic evaluation against adaptive adversaries remains limited, even when successful attacks can have wide security and privacy implications, and many real-world LLM-based applications remain vulnerable. We present the results of LLMail-Inject, a public challenge simulating a realistic scenario in which participants adaptively attempted to inject malicious instructions into emails in order to "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2506.09956","kind":"arxiv","version":1},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CR","submitted_at":"2025-06-11T17:30:07Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"37c01fb457bf909ec6964d3c98417e37c5275ef744d3d36f73659ca7b22f6d0f","abstract_canon_sha256":"c06b9aa9ddef581666907efd90692d3fa5d46baa2a9f4ef0d0708a6df1096660"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T11:19:59.694784Z","signature_b64":"VbVAod9quMHn01xhzZhtr1iEHAsuW27KQjvResxJGOObD8yX2kQrn15cdk8oOYmuRbm19R4U1PdyzXRUqOPxDg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"511ba26e51be01400ba8ffdc6d7277930b43785a431959e95695c93a40175dc4","last_reissued_at":"2026-07-05T11:19:59.694297Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T11:19:59.694297Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI"],"primary_cat":"cs.CR","authors_text":"Ahmed Salem, Aideen Fay, Alex Apostolov, Andrew Paverd, Athar Mahboob, Benjamin Pannell, Byeonghyeon Kim, Chi-Huang Liu, Chun-Chih Kuo, Conor McCauley, Danyael Manlangit, Egor Zverev, Giovanni Cherubin, Hajin Choi, Haris Umair, Hyeonjin Lee, Jannis Weigend, Jo\\~ao Donato, Kai-Chieh Liao, Mark Russinovich, Masayuki Kawakita, Myeongjin Cho, Sahar Abdelnabi, Tran Huu Bach, Tsun-Han Chiang","submitted_at":"2025-06-11T17:30:07Z","abstract_excerpt":"Indirect Prompt Injection attacks exploit the inherent limitation of Large Language Models (LLMs) to distinguish between instructions and data in their inputs. Despite numerous defense proposals, the systematic evaluation against adaptive adversaries remains limited, even when successful attacks can have wide security and privacy implications, and many real-world LLM-based applications remain vulnerable. We present the results of LLMail-Inject, a public challenge simulating a realistic scenario in which participants adaptively attempted to inject malicious instructions into emails in order to "},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2506.09956","kind":"arxiv","version":1},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2506.09956/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2506.09956","created_at":"2026-07-05T11:19:59.694356+00:00"},{"alias_kind":"arxiv_version","alias_value":"2506.09956v1","created_at":"2026-07-05T11:19:59.694356+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2506.09956","created_at":"2026-07-05T11:19:59.694356+00:00"},{"alias_kind":"pith_short_12","alias_value":"KEN2E3SRXYAU","created_at":"2026-07-05T11:19:59.694356+00:00"},{"alias_kind":"pith_short_16","alias_value":"KEN2E3SRXYAUAC5I","created_at":"2026-07-05T11:19:59.694356+00:00"},{"alias_kind":"pith_short_8","alias_value":"KEN2E3SR","created_at":"2026-07-05T11:19:59.694356+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":8,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.22659","citing_title":"Confidently Wrong: Severity-Aware Calibration of Prompt-Injection Detectors under Attack Shift","ref_index":18,"is_internal_anchor":false},{"citing_arxiv_id":"2606.02959","citing_title":"Gate AI: LLM Security Benchmark Evaluation Methodology and Results","ref_index":9,"is_internal_anchor":false},{"citing_arxiv_id":"2606.10749","citing_title":"Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation","ref_index":1,"is_internal_anchor":false},{"citing_arxiv_id":"2605.19192","citing_title":"Hallucination as Exploit: Evidence-Carrying Multimodal Agents","ref_index":15,"is_internal_anchor":false},{"citing_arxiv_id":"2605.19192","citing_title":"Hallucination as Exploit: Evidence-Carrying Multimodal Agents","ref_index":15,"is_internal_anchor":false},{"citing_arxiv_id":"2510.09093","citing_title":"Exploiting Web Search Tools of AI Agents for Data Exfiltration","ref_index":21,"is_internal_anchor":false},{"citing_arxiv_id":"2605.11868","citing_title":"IPI-proxy: An Intercepting Proxy for Red-Teaming Web-Browsing AI Agents Against Indirect Prompt Injection","ref_index":13,"is_internal_anchor":false},{"citing_arxiv_id":"2605.07269","citing_title":"MIPIAD: Multilingual Indirect Prompt Injection Attack Defense with Qwen -- TF-IDF Hybrid and Meta-Ensemble Learning","ref_index":1,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM","json":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM.json","graph_json":"https://pith.science/api/pith-number/KEN2E3SRXYAUAC5I77OG24TXSM/graph.json","events_json":"https://pith.science/api/pith-number/KEN2E3SRXYAUAC5I77OG24TXSM/events.json","paper":"https://pith.science/paper/KEN2E3SR"},"agent_actions":{"view_html":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM","download_json":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM.json","view_paper":"https://pith.science/paper/KEN2E3SR","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2506.09956&json=true","fetch_graph":"https://pith.science/api/pith-number/KEN2E3SRXYAUAC5I77OG24TXSM/graph.json","fetch_events":"https://pith.science/api/pith-number/KEN2E3SRXYAUAC5I77OG24TXSM/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM/action/timestamp_anchor","attest_storage":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM/action/storage_attestation","attest_author":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM/action/author_attestation","sign_citation":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM/action/citation_signature","submit_replication":"https://pith.science/pith/KEN2E3SRXYAUAC5I77OG24TXSM/action/replication_record"}},"created_at":"2026-07-05T11:19:59.694356+00:00","updated_at":"2026-07-05T11:19:59.694356+00:00"}