{"paper":{"title":"How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Third-party LLM agent skills leak credentials in over 500 cases through debug logs and prompt injections.","cross_cats":["cs.AI"],"primary_cat":"cs.CR","authors_text":"Gelei Deng, Jianting Ning, Lei Ma, Leo Yu Zhang, Yanjun Zhang, Yi Liu, Ying Zhang, Yuekang Li, Zhihao Chen, Zhiqiang Li","submitted_at":"2026-04-03T14:50:16Z","abstract_excerpt":"Large Language Model (LLM) agents increasingly rely on third-party skills that operate within privileged execution environments and routinely handle sensitive credentials, yet how these credentials are leaked remains largely unexplored. To fill this gap, we present the first large-scale empirical study on credential leakage in agent skills. From 170,226 artifacts on SkillsMP, the largest open-source skill marketplace, we sampled 17,022 skills via stratified random sampling and analyzed each through static secret extraction (regex and AST parsing), dynamic sandbox testing with mock credentials,"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"We identify 520 vulnerable skills with 1,708 issues and derive a taxonomy of 10 leakage patterns (4 accidental and 6 adversarial).","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The 17,022 sampled skills from SkillsMP are representative of the broader population of 170k skills and that static analysis plus sandbox testing reliably detects all leakage patterns.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Analysis of 17k LLM agent skills reveals 520 vulnerable ones with 1,708 leakage issues, primarily from debug output exposure, with a 10-pattern taxonomy and released dataset for future detection.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Third-party LLM agent skills leak credentials in over 500 cases through debug logs and prompt injections.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"323558b0388be82d3140e1888cce15049593fd796a73b5d5c913db2eded3200c"},"source":{"id":"2604.03070","kind":"arxiv","version":2},"verdict":{"id":"ab8c0046-e833-4328-ab19-181306962f22","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-13T19:48:23.575436Z","strongest_claim":"We identify 520 vulnerable skills with 1,708 issues and derive a taxonomy of 10 leakage patterns (4 accidental and 6 adversarial).","one_line_summary":"Analysis of 17k LLM agent skills reveals 520 vulnerable ones with 1,708 leakage issues, primarily from debug output exposure, with a 10-pattern taxonomy and released dataset for future detection.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The 17,022 sampled skills from SkillsMP are representative of the broader population of 170k skills and that static analysis plus sandbox testing reliably detects all leakage patterns.","pith_extraction_headline":"Third-party LLM agent skills leak credentials in over 500 cases through debug logs and prompt injections."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2604.03070/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}