Pith. sign in

REVIEW 4 major objections 5 minor 108 references

The paper claims that a lightweight graph neural network can act as an attention selector for a large language model, turning whole-package malware detection into a small-subgraph judgment with 94–97 percent accuracy and orders-of-magnitude

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A GNN selects suspicious subgraphs of a Python project and an LLM judges only those subgraphs, beating direct whole-project LLM analysis on PyPI malware benchmarks.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection Fresh pipeline and dramatic token savings, but the main accuracy comparisons are contaminated by a train/test overlap that the paper never addresses. the 4 major comments →

arxiv 2601.12890 v3 pith:NDWA3NHH submitted 2026-01-19 cs.SE

Efficient Code Analysis via Graph Representation Learning-Guided Large Language Models

Hang Gao , Tao Peng , Baoquan Cui , Hong Huang , Fengge Wu , Junsuo Zhao , Jian Zhang This is my paper
classification cs.SE
keywords malicious code detectionlarge language modelsgraph neural networksattention mechanismcode graphsPython package securitymodel explanationssupply chain security
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that large language models can be made practical for detecting malicious Python packages by first letting a small graph neural network point to the suspicious regions of a code graph, then feeding only those regions to the LLM. It reports accuracy of 94–97 percent on three public benchmarks and 95.62 percent on a newly assembled large dataset, beating both direct LLM prompting and conventional detection tools, while cutting token use from roughly 250,000 to about 640 tokens per large package. If correct, this means a GNN trained with only binary labels can supply enough contextual signal that an LLM's judgment improves and becomes inexpensive enough for real-world scanning. The central mechanism is treating the GNN's explanation masks as attention: the masks select the subgraph most responsible for a malicious prediction.

Core claim

GMLLM constructs a code graph from a Python package, with nodes for modules, classes, and functions and edges for calls, definitions, inheritance, and installation hooks. It uses an LLM to generate a set of sensitive-behavior rules, encoded as multi-hot node features, and trains a two-layer graph convolutional network with only package-level binary labels. At inference, for a package the GNN flags as malicious, the method learns per-sample edge and feature masks that maximize the probability of the malicious class; these masks become attention scores. A threshold picks the top-K scored structures, and the LLM receives only this subgraph, together with the corresponding source code, and retur

What carries the argument

The load-bearing mechanism is the attention subgraph produced by per-sample mask optimization on a trained graph classifier. Starting from an AST-derived code graph with call, inheritance, decorator, and hook edges, the paper builds node features from LLM-generated sensitive-behavior rules, trains a two-layer GCN under binary supervision, then optimizes edge and feature masks so that the masked graph still yields a malicious prediction. These mask values serve as node and edge attention scores; a top-K budget converts them into a small natural-language subgraph description that the LLM analyzes. This is what turns a whole-package problem into a small-context problem.

Load-bearing premise

The success on public benchmarks depends on the assumption that those benchmarks were not part of the GNN's training set; the paper assembled its own training corpus from the same public sources and never states that overlapping samples were removed.

What would settle it

Retrain the GNN after explicitly deleting from the training corpus any package that also appears in the three public benchmark datasets, then rerun the main accuracy table; if accuracy drops to roughly the direct-LLM level, the reported superiority is a training-leakage artifact. A simpler check is to compare package names across splits.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • LLMs can be applied to package-level malicious code detection without processing whole repositories, making large-scale scanning technically and economically feasible.
  • The method returns a location and characteristic description of the malicious behavior, not just a binary verdict, which supports triage and remediation.
  • Training requires only benign/malicious labels, so the detector can be retrained on newly discovered packages without expensive fine-grained annotation.
  • Because token usage per large package falls from roughly 250,000 to about 640, the per-package cost of LLM analysis drops by orders of magnitude.
  • The graph structure alone, without any behavior rules, already yields 88.65 percent accuracy, suggesting the representation carries signal independent of rule matching.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the attention masks are as reliable as reported, the same subgraph-selection idea could be applied to other long-context code tasks, such as locating the exact functions responsible for a vulnerability or tracing data flow across modules.
  • The paper's token-efficiency numbers omit the roughly 10-second-per-package explainer step; a fair end-to-end cost comparison should include that offline computation, which may matter for very large scan queues.
  • A direct way to test generalization is to check whether any public-benchmark packages appear in the training set, since the training corpus is assembled from the same public sources and no deduplication is stated.
  • The related-work section contains unresolved placeholder citations, so some comparisons to prior work are not fully sourced.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GMLLM, a pipeline for LLM-based malicious Python package detection. A project is parsed into an AST/call graph; an LLM generates sensitive-behavior rules; a GNN is trained with binary labels; GNNExplainer extracts high-attention subgraphs; and an LLM makes the final verdict from those subgraphs. The paper reports that GMLLM with ChatGPT-4o reaches 94.3–96.96% accuracy on three public datasets and 95.62% on the authors' MalCP dataset, well above direct GPT-4o, while reducing token usage from roughly 250,000 to around 640 tokens for large packages. It also reports improved explanation quality and a human validation of the LLM-as-judge protocol. The central claim is that the graph-guided attention mechanism lets LLMs reliably detect malicious code in large projects.

Significance. If the reported numbers were trustworthy, the contribution would be significant: it offers a concrete way to make LLM-based package scanning practical by replacing whole-package prompts with small, explainer-selected subgraphs, and it shows a large reduction in token cost. The paper also has some commendable features: the token accounting in Appendix D.3 is model-specific and includes formatting overhead; the LLM-as-judge protocol is checked against human annotators in Appendix C.5; and the ablation in Table 5 provides some insight into which components contribute. However, the main empirical claims rest on the contaminated evaluation described below. In its current form the paper does not establish that GMLLM consistently outperforms the baselines.

major comments (4)
  1. [Appendix D.1.1 / D.1.4, Table 1] Table 1 reports results on Backstabbers, Datadog, and Mal-OSS, the same three sources from which MalCP was constructed: Appendix D.1.1 states that MalCP was built by 'integrating multiple open-source intelligence sources, including Mal_OSS, Backstabbers Knife Collection, and Datadog's open malware dataset.' The dataset-filtering pipeline in D.1.4 has aggregation, annotation, and filtering steps, but no step for removing overlap with the later evaluation sets. D.2.2 describes an 80/20 training split but never states that the public datasets were excluded from GNN training. If MalCP is the GNN training pool, the public test sets are in-distribution, and the high-attention subgraphs in Table 1 are not independent of training. This directly affects the paper's central claim of consistent superiority. A decontamination statement and a re-evaluation on disjoint samples are required.
  2. [Appendix D.2.2] The GNN training protocol says: 'No validation set is used. ... Early stopping may be applied based on test F1 score.' Selecting the early-stopping epoch on the test set is a form of test-set leakage and makes the GNN's reported performance optimistic. It also means there is no valid procedure for selecting hyperparameters such as the number of epochs. A proper train/validation/test split and early stopping on validation metrics are needed before the GNN-based results can be interpreted.
  3. [Section 4.5.2, Table 6, Tables 1–2] The top-K edge budget is tuned on MalCP by varying K in {10, 20, 30, 50}, and then the paper states 'we fix K=20 in all main experiments,' including the MalCP results in Table 2. Because K is selected on the same dataset that is later used for evaluation, the MalCP accuracy numbers in Table 2 are an optimistic selected result, not an independent evaluation. If K is intended to be a fixed universal constant, it should be set a priori or selected on a separate validation split, and sensitivity should be reported on the actual test sets.
  4. [Section 4.1, Tables 1–3; D.2.2] The paper states that 'all experimental results are reported as the average across five independent trials,' but no standard deviations, confidence intervals, or per-trial results are given in Tables 1–3. Many of the claimed improvements are only a few percentage points (e.g., 95.62 vs 93.56 in Table 2), so without variance information the reader cannot judge whether the differences are meaningful. Additionally, D.2.2 says the graph classifier 'is not used for standalone detection,' but Section 4.5.1 compares a 'GNN-only model' and reports 92.22% accuracy on MalCP; this inconsistency should be resolved.
minor comments (5)
  1. [Eq. (2)] The symmetrization expression appears to use the raw mask M in the second term instead of the sigmoid-activated mask. The intended operation is likely (δ(M) + δ(M)^T)/2; otherwise the resulting matrix is not symmetric.
  2. [Section 3.2, Eq. (11)] The text sets a threshold α, but Eq. (11) uses γ_node and γ_edge. The notation should be aligned.
  3. [Keywords] Typo: 'lanaguage' should be 'language.'
  4. [Appendix D.2.4] The appendix states that the codebase and critical dependencies 'are released as part of our codebase,' but no artifact URL, repository link, or dataset hash is provided. Including a link and hashes would be necessary for reproducibility.
  5. [Table 5] The '50 Rules (Scomm ∪ 0.5Sdata)' configuration is described as 'randomly keeps half of the rules in Sdata at inference time'; it is unclear whether this is a single fixed random subset or averaged over multiple draws. Please clarify.

Circularity Check

1 steps flagged

Public-benchmark 'predictions' are fit on the same packages used to build MalCP; no deduplication is stated.

specific steps
  1. fitted input called prediction [Appendix D.1.1, D.1.4; Table 1]
    "To construct the MalCP dataset, we compiled malicious Python packages from the PyPI ecosystem by integrating multiple open-source intelligence sources, including Mal_OSS [38], Backstabbers Knife Collection [36], and Datadog’s open malware dataset [37]. ... Table 1: Performance on Backstabbers, Datadog, Mal-OSS Datasets."

    The GNN that produces the attention subgraphs is trained on MalCP. By the paper's own construction, MalCP aggregates Backstabbers, Datadog, and Mal-OSS, and the filtering pipeline (D.1.4) removes samples only by 'archive structure and size constraints,' not by overlap with those public evaluation sets. Table 1 then reports GMLLM's accuracy on the very same three datasets. Any retained package from these sources that falls into the 80% training split has its test instance effectively fitted: the GNN's attention masks and the resulting LLM subgraph are derived from a model that saw that package during training. The public-dataset numbers in Table 1 are therefore not independent generalization measurements; they are, at least in part, evaluations on training data relabeled as prediction.

full rationale

The core GNN+explainer+LLM derivation is not circular: the GNN is trained with binary labels, the explainer maximizes the trained model's malicious probability (Eq. 5), and the LLM independently judges the extracted subgraph (Eq. 12). The rule set Sdata is generated from 10% of the training data and used as features, which is standard supervised feature extraction and does not by itself make the prediction equal to an input. No self-citation chain or imported uniqueness theorem is load-bearing. The significant problem is at the evaluation level: MalCP is constructed from the same public datasets that are later used as held-out test sets, with no stated deduplication. If overlapping packages entered the MalCP training split, the advertised superiority over direct LLM baselines and SOTA tools (Tables 1–2) is partially an artifact of training on the test instances. The token-efficiency result is less affected, but the headline claim rests on the contaminated accuracy comparison.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

No new physical or mathematical entities are introduced; the framework is an assembly of existing techniques. The MalCP dataset is a new benchmark but not a postulated entity.

free parameters (5)
  • K (top-K edge budget) = 20
    Selected in §4.5.2 by sweeping K ∈ {10,20,30,50} on MalCP and used for all datasets; this is a test-set-tuned hyperparameter if MalCP overlaps the public test sets.
  • λ_size, λ_ent (mask regularizer weights) = not specified
    In Eq. (8), these weights control sparsity and entropy of the GNNExplainer masks; no values are reported, so the attention extraction is not fully reproducible.
  • Sdata sampling fraction = 10%
    §3.1: the data-derived sensitive rules are generated from 10% of the training data; this fraction is chosen by hand and affects the node feature vocabulary.
  • MalCP size thresholds = 5 KB / 10 KB
    §D.1.2: thresholds are 'empirically determined by inspecting the distribution of raw package sizes' and determine dataset composition and class balance.
  • GNNExplainer optimization budget = 100 gradient steps per package
    Appendix E.1 fixes 100 steps; the paper notes it could be reduced to 30–50, so the number is an arbitrary operating point rather than a principled choice.
axioms (6)
  • domain assumption GNNExplainer masks identify the code regions necessary for the LLM to judge maliciousness
    §3.2 assumes nodes/edges with high mask values are 'precisely the part that should be analyzed in detail using LLMs'; no fidelity or coverage check is reported.
  • domain assumption Static AST + call/dependency graphs capture cross-file malicious behavior
    §3.1 and Appendix C.1 construct graphs via AST parsing and heuristics without executing code; behaviors hidden in data or dynamic dispatch may be missed.
  • domain assumption LLM-generated sensitive-behavior rules Scomm ∪ Sdata cover the malicious feature space
    §3.1: node features are multi-hot matches of these rules; if rules miss an attack pattern, the GNN has no signal and the attention will not highlight it.
  • domain assumption The final LLM can reach a correct verdict from a truncated subgraph of at most 20 edges
    §3.2 and §C.3: the LLM receives only Att(G), i.e., node names and call relationships, not full source; the method assumes this is sufficient context.
  • domain assumption Public dataset labels and Snyk advisories are correct ground truth
    §D.1: MalCP labels come from Snyk; Table 1 uses Backstabbers/Datadog/Mal-OSS labels. No independent label audit is reported.
  • domain assumption A 2-layer GCN trained with an 80/20 split and no validation set generalizes across package sizes
    §D.2.2: no validation split is used and early stopping may be based on test F1; hyperparameters may be indirectly fitted to the test set.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Code Analysis via Graph Representation Learning-Guided Large Language Models." pith.science (2026). https://pith.science/paper/NDWA3NHH

@misc{pith2026260112890,
  author       = {Pith},
  title        = {Pith review of: Efficient Code Analysis via Graph Representation Learning-Guided Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NDWA3NHH}},
  note         = {Machine review of arXiv:2601.12890}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) have significantly advanced code analysis tasks, yet they struggle to detect malicious behaviors fragmented across files, whose intricate dependencies easily get lost in the vast amount of benign code. We therefore propose a graph-centric attention acquisition pipeline that enhances LLMs' ability to localize malicious behavior. The approach parses a project into a code graph, uses an LLM to encode nodes with semantic and structural signals, and trains a Graph Neural Network (GNN) under sparse supervision. The GNN performs an initial detection, and by interpreting these predictions, identifies key code sections that are most likely to contain malicious behavior. These influential regions are then used to guide the LLM's attention for in-depth analysis. This strategy significantly reduces interference from irrelevant context while maintaining low annotation costs. Extensive experiments show that the method consistently outperforms existing approaches on multiple public and custom datasets, highlighting its potential for practical deployment in software security scenarios.

Figures

Figures reproduced from arXiv: 2601.12890 by Baoquan Cui, Fengge Wu, Hang Gao, Hong Huang, Jian Zhang, Junsuo Zhao, Tao Peng.

Figure 1
Figure 1. Figure 1: Comparison between LLMs directly applied to malicious code detection [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Framework of the proposed GMLLM. 3.1 Training GNN To facilitate subsequent processing using GNNs, we represent the source code of each Python project as a graph Gcode = {Vcode , E code}, where V code is the set of nodes and E code is the set of edges. The node set V code is derived from the Abstract Syntax Tree (AST) of the Python project. We traverse all the “.py” files and parse the source code into AST … view at source ↗
Figure 3
Figure 3. Figure 3: A visualization of the code graph attention outputted with GMLLM when processing programs of different [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualizations of the code graph attention outputted with GMLLM when processing programs of large scales. [PITH_FULL_IMAGE:figures/full_fig_p024_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualizations of the code graph attention outputted with GMLLM when processing programs of medium [PITH_FULL_IMAGE:figures/full_fig_p025_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualizations of the code graph attention outputted with GMLLM when processing programs of small scales. [PITH_FULL_IMAGE:figures/full_fig_p026_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

108 extracted references · 11 linked inside Pith

  1. [1]

    Hellendoorn, Bogdan Vasilescu, and Brad A

    Daye Nam, Andrew Macvean, Vincent J. Hellendoorn, Bogdan Vasilescu, and Brad A. Myers. Using an LLM to help with code understanding. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14-20, 2024, pages 97:1–97:13. ACM, 2024

    2024

  2. [2]

    Butt, Mohammad Taha Khan, and Muhammad Ali Gulzar

    Sabaat Haroon, Ahmad Faraz Khan, Ahmad Humayun, Waris Gill, Abdul Haddi Amjad, Ali R. Butt, Mohammad Taha Khan, and Muhammad Ali Gulzar. How accurately do large language models understand code?CoRR, abs/2504.04372, 2025

    arXiv 2025

  3. [3]

    Mutation-based consistency testing for evaluating the code understanding capability of llms

    Ziyu Li and Donghwan Shin. Mutation-based consistency testing for evaluating the code understanding capability of llms. In Jane Cleland-Huang, Jan Bosch, Henry Muccini, and Grace A. Lewis, editors,Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI, CAIN 2024, Lisbon, Portugal, April 14-15, 2024, pages ...

    2024

  4. [4]

    Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, and Jie M. Zhang. Large language models for software engineering: Survey and open problems. InIEEE/ACM International Conference on Software Engineering: Future of Software Engineering, ICSE-FoSE 2023, Melbourne, Australia, May 14-20, 2023, pages 31–53. IEEE, 2023

    2023

  5. [5]

    Self-organized agents: A LLM multi-agent framework toward ultra large-scale code generation and optimization.CoRR, abs/2404.02183, 2024

    Yoichi Ishibashi and Yoshimasa Nishimura. Self-organized agents: A LLM multi-agent framework toward ultra large-scale code generation and optimization.CoRR, abs/2404.02183, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Larry Huynh, Yinghao Zhang, Djimon Jayasundera, Woojin Jeon, Hyoungshick Kim, Tingting Bi, and Jin B. Hong. Detecting code vulnerabilities using llms. In55th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2025, Naples, Italy, June 23-26, 2025, pages 401–414. IEEE, 2025

    2025

  7. [7]

    Malpacdetector: An llm-based malicious NPM package detector.IEEE Trans

    Jian Wang, Zhen Li, Jixiang Qu, Deqing Zou, Shouhuai Xu, Ziteng Xu, Zhenwei Wang, and Hai Jin. Malpacdetector: An llm-based malicious NPM package detector.IEEE Trans. Inf. Forensics Secur., 20:6279–6291, 2025. 10 Running Title for Header

    2025

  8. [8]

    Vulnerability detection and monitoring using llm

    Vishwanath Akuthota, Raghunandan Kasula, Sabiha Tasnim Sumona, Masud Mohiuddin, Md Tanzim Reza, and Md Mizanur Rahman. Vulnerability detection and monitoring using llm. In2023 IEEE 9th International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE), pages 309–314, 2023

    2023

  9. [9]

    Poster: An exploration of large language models in malicious source code detection

    Di Xue, Gang Zhao, Zhongqi Fan, Wei Li, Yahong Xu, Zhen Liu, Yin Liu, and Zhongliang Yuan. Poster: An exploration of large language models in malicious source code detection. In Bo Luo, Xiaojing Liao, Jun Xu, Engin Kirda, and David Lie, editors,Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS 2024, Salt Lake Ci...

    2024

  10. [10]

    Raas: Reasoning-aware attention sparsity for efficient LLM reasoning

    Junhao Hu, Wenrui Huang, Weidong Wang, Zhenwen Li, Tiancheng Hu, Zhixia Liu, Xusheng Chen, Tao Xie, and Yizhou Shan. Raas: Reasoning-aware attention sparsity for efficient LLM reasoning. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austr...

    2025

  11. [11]

    Joty, and Steven C

    Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven C. H. Hoi. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Vir...

    2021

  12. [12]

    Codegen: An open large language model for code with multi-turn program synthesis

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023

    2023

  13. [13]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    Pith/arXiv arXiv 2021

  14. [14]

    Code llama: Open foundation models for code.CoRR, abs/2308.12950, 2023

    Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton-Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thoma...

    Pith/arXiv arXiv 2023

  15. [15]

    Oss detect backdoor

    Microsoft. Oss detect backdoor. https://github.com/microsoft/OSSGadget/wiki/OSS-Detect-Backdoor , 2019. Accessed: April 5, 2025

    2019

  16. [16]

    A survey on common threats in npm and pypi registries.CoRR, abs/2108.09576, 2021

    Berkay Kaplan and Jingyu Qian. A survey on common threats in npm and pypi registries.CoRR, abs/2108.09576, 2021

    Pith/arXiv arXiv 2021

  17. [17]

    Yara, 2023

    VirusTotal. Yara, 2023. Retrieved Jun 12, 2023

    2023

  18. [18]

    Bandit, 2023

    PyCQA. Bandit, 2023. Retrieved Mar 10, 2023 fromhttps://github.com/PyCQA/bandit

    2023

  19. [19]

    Towards measuring supply chain attacks on package managers for interpreted languages

    Ruian Duan, Omar Alrawi, Ranjita Pai Kasturi, Ryan Elder, Brendan Saltaformaggio, and Wenke Lee. Towards measuring supply chain attacks on package managers for interpreted languages. In28th Annual Network and Distributed System Security Symposium, NDSS 2021, virtually, February 21-25, 2021. The Internet Society, 2021

    2021

  20. [20]

    Pooja Yadav, Neeraj Menon, Vinayakumar Ravi, Sowmya Vishvanathan, and Tuan D. Pham. Efficientnet convolutional neural networks-based android malware detection.Comput. Secur., 115:102622, 2022

    2022

  21. [21]

    Amddlmodel: Android smartphones malware detection using deep learning model.Plos one, 19(1):e0296722, 2024

    Muhammad Aamir, Muhammad Waseem Iqbal, Mariam Nosheen, M Usman Ashraf, Ahmad Shaf, Khalid Ali Almarhabi, Ahmed Mohammed Alghamdi, and Adel A Bahaddad. Amddlmodel: Android smartphones malware detection using deep learning model.Plos one, 19(1):e0296722, 2024

    2024

  22. [22]

    Deepcatra: Learning flow- and graph-based behaviours for android malware detection.IET Inf

    Yafei Wu, Jian Shi, Peicheng Wang, Dongrui Zeng, and Cong Sun. Deepcatra: Learning flow- and graph-based behaviours for android malware detection.IET Inf. Secur., 17(1):118–130, 2023

    2023

  23. [23]

    Iot-based android malware detection using graph neural network with adversarial defense.IEEE Internet Things J., 10(10):8432–8444, 2023

    Rahul Yumlembam, Biju Issac, Seibu Mary Jacob, and Longzhi Yang. Iot-based android malware detection using graph neural network with adversarial defense.IEEE Internet Things J., 10(10):8432–8444, 2023

    2023

  24. [24]

    Android malware detection through generative adversarial networks.Trans

    Muhammad Amin, Babar Shah, Aizaz Sharif, Tamleek Ali, Ki-Il Kim, and Sajid Anwar. Android malware detection through generative adversarial networks.Trans. Emerg. Telecommun. Technol., 33(2), 2022

    2022

  25. [25]

    Schorlemmer, Santiago Torres-Arias, and James C

    Chinenye Okafor, Taylor R. Schorlemmer, Santiago Torres-Arias, and James C. Davis. Sok: Analysis of software supply chain security by establishing secure design properties. In Santiago Torres-Arias, Marcela S. Melara, and Laurent Simon, editors,Proceedings of the 2022 ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses, SCORED2...

    2022

  26. [26]

    Decoding the secrets of machine learning in malware classification: A deep dive into datasets, feature extraction, and model performance

    Savino Dambra, Yufei Han, Simone Aonzo, Platon Kotzias, Antonino Vitale, Juan Caballero, Davide Balzarotti, and Leyla Bilge. Decoding the secrets of machine learning in malware classification: A deep dive into datasets, feature extraction, and model performance. In Weizhi Meng, Christian Damsgaard Jensen, Cas Cremers, and Engin Kirda, editors,Proceedings ...

    2023

  27. [27]

    A needle is an outlier in a haystack: Hunting malicious pypi packages with code clustering

    Wentao Liang, Xiang Ling, Jingzheng Wu, Tianyue Luo, and Yanjun Wu. A needle is an outlier in a haystack: Hunting malicious pypi packages with code clustering. In38th IEEE/ACM International Conference on Automated Software Engineering, ASE 2023, Luxembourg, September 11-15, 2023, pages 307–318. IEEE, 2023

    2023

  28. [28]

    1+1>2: Integrating deep code behaviors with metadata features for malicious pypi package detection

    Xiaobing Sun, Xingan Gao, Sicong Cao, Lili Bo, Xiaoxue Wu, and Kaifeng Huang. 1+1>2: Integrating deep code behaviors with metadata features for malicious pypi package detection. In Vladimir Filkov, Baishakhi Ray, and Minghui Zhou, editors, Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE 2024, Sacramento, CA...

    2024

  29. [29]

    Dos and don’ts of machine learning in computer security

    Daniel Arp, Erwin Quiring, Feargus Pendlebury, Alexander Warnecke, Fabio Pierazzi, Christian Wressnegger, Lorenzo Cavallaro, and Konrad Rieck. Dos and don’ts of machine learning in computer security. In31st USENIX Security Symposium (USENIX Security 22), pages 3971–3988, Boston, MA, August 2022. USENIX Association

    2022

  30. [30]

    Bad snakes: Understanding and improving python package index malware scanning

    Duc-Ly Vu, Zachary Newman, and John Speed Meyers. Bad snakes: Understanding and improving python package index malware scanning. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 499–511, 2023

    2023

  31. [31]

    Williams

    Nusrat Zahan, Philipp Burckhardt, Mikola Lysenko, Feross Aboukhadijeh, and Laurie A. Williams. Leveraging large language models to detect NPM malicious packages. In47th IEEE/ACM International Conference on Software Engineering, ICSE 2025, Ottawa, ON, Canada, April 26 - May 6, 2025, pages 2625–2637. IEEE, 2025

    2025

  32. [32]

    Large language models for code analysis: Do LLMs really do their job? In33rd USENIX Security Symposium (USENIX Security 24), pages 829–846, Philadelphia, PA, August 2024

    Chongzhou Fang, Ning Miao, Shaurya Srivastav, Jialin Liu, Ruoyu Zhang, Ruijie Fang, Asmita, Ryan Tsang, Najmeh Nazari, Han Wang, and Houman Homayoun. Large language models for code analysis: Do LLMs really do their job? In33rd USENIX Security Symposium (USENIX Security 24), pages 829–846, Philadelphia, PA, August 2024. USENIX Association

    2024

  33. [33]

    Maltracker: A fine-grained npm malware tracker copiloted by llm-enhanced dataset

    Zeliang Yu, Ming Wen, Xiaochen Guo, and Hai Jin. Maltracker: A fine-grained npm malware tracker copiloted by llm-enhanced dataset. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, page 1759–1771, New York, NY , USA, 2024. Association for Computing Machinery

    2024

  34. [34]

    Leveraging large language models for behavior-based malware detection using deep learning

    Tosin Akinsowon and Haodi Jiang. Leveraging large language models for behavior-based malware detection using deep learning. Technical report, Institute for Homeland Security, Sam Houston State University, 2024

    2024

  35. [35]

    Al-Karaki, Muhammad Al-Zafar Khan, and Marwan Omar

    Jamal N. Al-Karaki, Muhammad Al-Zafar Khan, and Marwan Omar. Exploring llms for malware detection: Review, framework design, and countermeasure approaches.CoRR, abs/2409.07587, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    Backstabber’s knife collection: A review of open source software supply chain attacks

    Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. Backstabber’s knife collection: A review of open source software supply chain attacks. InInternational Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pages 23–43. Springer, 2020

    2020

  37. [37]

    Malicious software packages dataset, 2023

    Guarddog. Malicious software packages dataset, 2023. Accessed: May 15, 2025

    2023

  38. [38]

    An empirical study of malicious code in pypi ecosystem

    Wenbo Guo, Zhengzi Xu, Chengwei Liu, Cheng Huang, Yong Fang, and Yang Liu. An empirical study of malicious code in pypi ecosystem. In38th IEEE/ACM International Conference on Automated Software Engineering, ASE 2023, Luxembourg, September 11-15, 2023, pages 166–177. IEEE, 2023

    2023

  39. [39]

    Qwen2.5 technical report.CoRR, abs/2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu X...

    Pith/arXiv arXiv 2024

  40. [40]

    Llama 2: Open foundation and fine-tuned chat models.CoRR, abs/2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Harts...

    Pith/arXiv arXiv 2023

  41. [41]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany...

    Pith/arXiv arXiv 2024

  42. [42]

    A comprehensive capability analysis of GPT-3 and GPT-3.5 series models.CoRR, abs/2303.10420, 2023

    Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang, and Xuanjing Huang. A comprehensive capability analysis of GPT-3 and GPT-3.5 series models.CoRR, abs/2303.10420, 2023

    Pith/arXiv arXiv 2023

  43. [43]

    Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau,...

    Pith/arXiv arXiv 2024

  44. [44]

    A fork of bandit tool with patterns to identifying malicious python code.Retrieved September, 25:2024, 2020

    Duc-Ly Vu. A fork of bandit tool with patterns to identifying malicious python code.Retrieved September, 25:2024, 2020

    2024

  45. [45]

    VirusTotal — Upload.https://www.virustotal.com/gui/home/upload/

    VirusTotal. VirusTotal — Upload.https://www.virustotal.com/gui/home/upload/. Accessed: September 5, 2025

    2025

  46. [46]

    Beyond typosquatting: An in-depth look at package confusion

    Shradha Neupane, Grant Holmes, Elizabeth Wyss, Drew Davidson, and Lorenzo De Carli. Beyond typosquatting: An in-depth look at package confusion. In Joseph A. Calandrino and Carmela Troncoso, editors,32nd USENIX Security Symposium, USENIX Security 2023, Anaheim, CA, USA, August 9-11, 2023, pages 3439–3456. USENIX Association, 2023

    2023

  47. [47]

    Vaidya, Drew Davidson, Lorenzo De Carli, and Vaibhav Rastogi

    Matthew Taylor, Ruturaj K. Vaidya, Drew Davidson, Lorenzo De Carli, and Vaibhav Rastogi. Defending against package typosquatting. In Miroslaw Kutylowski, Jun Zhang, and Chao Chen, editors,Network and System Security - 14th International Conference, NSS 2020, Melbourne, VIC, Australia, November 25-27, 2020, Proceedings, volume 12570 ofLecture Notes in Comp...

    2020

  48. [48]

    Typosquatting and combosquatting attacks on the python ecosystem

    Duc-Ly Vu, Ivan Pashchenko, Fabio Massacci, Henrik Plate, and Antonino Sabetta. Typosquatting and combosquatting attacks on the python ecosystem. InIEEE European Symposium on Security and Privacy Workshops, EuroS&P Workshops 2020, Genoa, Italy, September 7-11, 2020, pages 509–514. IEEE, 2020

    2020

  49. [49]

    OSS gadget, 2023

    Microsoft. OSS gadget, 2023. Retrieved Aug 10, 2023 fromhttps://github.com/microsoft/OSSGadget

    2023

  50. [50]

    OSSF, 2023

    OSSF. OSSF, 2023. Retrieved Sep 27, 2023 fromhttps://github.com/ossf/package-analysis

    2023

  51. [51]

    DONAPI: malicious NPM packages detector using behavior sequence knowledge mapping

    Cheng Huang, Nannan Wang, Ziyan Wang, Siqi Sun, Lingzi Li, Junren Chen, Qianchong Zhao, Jiaxuan Han, Zhen Yang, and Lei Shi. DONAPI: malicious NPM packages detector using behavior sequence knowledge mapping. In Davide Balzarotti and Wenyuan Xu, editors,33rd USENIX Security Symposium, USENIX Security 2024, Philadelphia, PA, USA, August 14-16, 2024. USENIX ...

    2024

  52. [52]

    Malcertain: Enhancing deep neural network based android malware detection by tackling prediction uncertainty

    Haodong Li, Guosheng Xu, Liu Wang, Xusheng Xiao, Xiapu Luo, Guoai Xu, and Haoyu Wang. Malcertain: Enhancing deep neural network based android malware detection by tackling prediction uncertainty. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14-20, 2024, pages 150:1–150:13. ACM, 2024

    2024

  53. [53]

    Gobbi and Johannes Kinder

    Matías F. Gobbi and Johannes Kinder. Poster: Using codeql to detect malware in npm. InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS ’23, page 3519–3521, New York, NY , USA, 2023. Association for Computing Machinery

    2023

  54. [54]

    EaTVul: ChatGPT-based evasion attack against software vulnerability detection

    Shigang Liu, Di Cao, Junae Kim, Tamas Abraham, Paul Montague, Seyit Camtepe, Jun Zhang, and Yang Xiang. EaTVul: ChatGPT-based evasion attack against software vulnerability detection. In33rd USENIX Security Symposium (USENIX Security 24), pages 7357–7374, Philadelphia, PA, August 2024. USENIX Association

    2024

  55. [55]

    Code vulnerability detection based on augmented program dependency graph and optimized codebert.Scientific Reports, 15(1):39301, 2025

    Zhengbin Zou, Tao Jiang, Yizheng Wang, Tiancheng Xue, Nan Zhang, and Jie Luan. Code vulnerability detection based on augmented program dependency graph and optimized codebert.Scientific Reports, 15(1):39301, 2025

    2025

  56. [56]

    Vulcobert: A codebert-based system for source code vulnerability detection

    Yuying Xia, Haijian Shao, and Xing Deng. Vulcobert: A codebert-based system for source code vulnerability detection. In Proceedings of the 2024 International Conference on Generative Artificial Intelligence and Information Security, GAIIS ’24, page 249–252, New York, NY , USA, 2024. Association for Computing Machinery

    2024

  57. [57]

    Leveraging an enhanced codebert-based model for multiclass software defect prediction via defect classification.IEEE Access, 13:24383–24397, 2025

    Rida Ghafoor Hussain, Kin Choong Yow, and Marco Gori. Leveraging an enhanced codebert-based model for multiclass software defect prediction via defect classification.IEEE Access, 13:24383–24397, 2025

    2025

  58. [58]

    Vuld-codebert: Codebert-based vulnerability detection model for c/c++ code

    Zhe Xiong and Weiyu Dong. Vuld-codebert: Codebert-based vulnerability detection model for c/c++ code. In2024 6th International Conference on Communications, Information System and Computer Engineering (CISCE), pages 914–919, 2024. 13 Running Title for Header

    2024

  59. [59]

    Codebert: A pre-trained model for programming and natural languages

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. Codebert: A pre-trained model for programming and natural languages. In Trevor Cohn, Yulan He, and Yang Liu, editors,Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, ...

    2020

  60. [60]

    Python source code vulnerability detection based on codebert language model

    Kunpeng Zhao, Shuya Duan, Ge Qiu, Jinyuan Zhai, Mingze Li, and Long Liu. Python source code vulnerability detection based on codebert language model. In2024 7th International Conference on Algorithms, Computing and Artificial Intelligence (ACAI), pages 1–6, 2024

    2024

  61. [61]

    Hammerschmidt, Carlos Hernandez Gañán, and Sicco Verwer

    Azqa Nadeem, Christian A. Hammerschmidt, Carlos Hernandez Gañán, and Sicco Verwer. Malpaca: Malware packet sequence clustering and analysis.CoRR, abs/1904.01371, 2019

    Pith/arXiv arXiv 1904

  62. [62]

    Open source vulnerability database.https://security.snyk.io/vuln, 2024

    Snyk. Open source vulnerability database.https://security.snyk.io/vuln, 2024. Accessed: May 14, 2025

    2024

  63. [63]

    Mitre att&ck: Design and philosophy

    Blake E Strom, Andy Applebaum, Doug P Miller, Kathryn C Nickels, Adam G Pennington, and Cody B Thomas. Mitre att&ck: Design and philosophy. InTechnical report. The MITRE Corporation, 2018. 14 Running Title for Header A Usage of Large Language Model In our paper, we used LLMs to assist with polishing the writing, including correcting grammatical errors and...

    2018

  64. [64]

    network": lambda n: n.startswith((

    emphasized the importance of dataset selection and evaluation in applying machine learning to security. Vu et al. [30] conduct study that reveal repository administrators require extremely low false positive rates (below 0.1%). They found that a socio-technical malicious code detection system has emerged, where external security researchers scan for malic...

  65. [65]

    could be malicious

    Threat Tactic Generalization (1–5) How well does it generalize from specific functions or code elements to a recognized cybersecurity tactic (e.g., ‘Reconnais- sance’, ‘Defense Evasion’, ‘Exfiltration’)? →5: Clearly identifies and frames the analysis using a standard attack pattern (e.g., MITRE ATT&CK). →4: Identifies a general malicious purpose (e.g., da...

  66. [66]

    →4: Shows a mostly coherent flow but omits minor steps or data dependencies

    Execution Path Traceability (1–5) How clearly does it reconstruct a step-by-step execution flow, including function calls, data movement, and control logic? →5: Presents a complete, logical, and verifiable sequence of actions. →4: Shows a mostly coherent flow but omits minor steps or data dependencies. →3: Outlines key stages (e.g., collect→encode→send), ...

  67. [67]

    →4: Most claims are evidence-backed; minor inferences are reasonable

    Evidence Groundedness (1–5) Are claims directly and tightly supported by specific code elements (e.g., function calls, strings, variables)? →5: Every significant claim is explicitly tied to observable code. →4: Most claims are evidence-backed; minor inferences are reasonable. →3: Core behaviors are supported, but some conclusions extend beyond direct evid...

  68. [68]

    quality_scores

    Factual Alignment (1-5) Does it correctly identify and accurately describe the primary malicious behavior? →5: Accurate and complete—captures the core behavior and key details. →3: Partially correct—identifies the general type of malicious activity but misses or misrepresents critical elements. →1: Fundamentally wrong or entirely misses the core behavior....

  69. [69]

    Aggregation:Malicious packages were consolidated from the PyPI ecosystem using multiple open-source intelligence sources

  70. [70]

    Behavioral Annotation:Each package was enriched with a structured behavior summary derived from the Snyk advisory system

  71. [71]

    Small/Medium/Large

    Filtering:Samples were filtered based on archive structure and size constraints to ensure semantic richness and practical utility. D.2 Settings This section outlines the end-to-end pipeline used in our experiments, including graph preparation, model training, explanation generation, and runtime environment setup. D.2.1 Graph Preparation and Labeling Each ...

  72. [72]

    Password Decryption: The use of ‘ctypes.windll.crypt32.CryptUnprotectData’ indicates an attempt to decrypt sensitive data, which is commonly used to scrape encrypted data like saved passwords

  73. [73]

    Clipboard Hijacking: The use of ‘pyperclip.copy’ in combination with ‘auto_copy_wallet.address_swap’ suggests the possibility of clipboard manipulation, potentially to replace wallet addresses for cryptocurrency theft

  74. [74]

    Credential Harvesting: Calls like ‘os.getlogin’, ‘winreg.OpenKey’, and ‘winreg.QueryValueEx’ indicate that the package may be gathering system and user information illegitimately

  75. [75]

    Screen and Cookie Stealing: Nodes like ‘PIL.ImageGrab.grab’ (screenshot capture) and ‘steal_cookies’ strongly imply the stealing of private user data

  76. [76]

    Unauthorized Memory Operations: Calls to ‘ctypes.cdll.msvcrt.memcpy’ and ‘ctypes.windll.kernel32.LocalFree’ raise concerns about memory manipulation, which is often associated with malicious payload deployment or data harvesting

  77. [77]

    Network Activity: The usage of ‘urllib.request.Request’ and ‘urllib.request.urlopen’ under the function ‘LoadUrlib’ may imply exfiltration of data or receiving commands from a remote server

  78. [78]

    Potential Anti-Debugging: The function ‘NoDebugg.sp3cCheq’ suggests the package may contain anti-debugging checks to evade detection by security tools or analysts

  79. [79]

    , “mitigation

    Execution and Persistence: The presence of ‘subprocess.Popen’, combined with suspicious function names like ‘ZipThings’ and system-manipulative calls like ‘os._exit’, hints at unauthorized script execution or persistence mechanisms. These behaviors, when taken together, are highly indicative of malicious software designed for data theft, remote control, o...

  80. [80]

    ‘ctypes.windll.crypt32.CryptUnprotectData’: Validate any use of sensitive data decryption and ensure it is necessary and authorized

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.