pith. sign in

arxiv: 2606.08553 · v1 · pith:I2GZSGIPnew · submitted 2026-06-07 · 💻 cs.SE

FusionVul: A Multimodal Feature Fusion Framework for Source Code Vulnerability Detection

Hongyu Yang , Yaping Zhu , Jingchuan Luo , Hiroshi Nomaguchi , Chunhua Su , Willy Susilo This is my paper

Pith reviewed 2026-06-27 18:09 UTC · model grok-4.3

classification 💻 cs.SE
keywords source code vulnerability detectionmultimodal feature fusiontransformer encodergraph neural networkcross-attentionsample-aware weightingsoftware security
0
0 comments X

The pith

FusionVul fuses Transformer syntax with graph structure through cross-attention to improve vulnerability detection on varied code datasets.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces FusionVul as a joint learning framework that combines sequential representations from a pretrained Transformer with structural information processed by a graph neural network. A cross-attention module enables detailed interaction between these two views of the code, while a sample-aware weighting scheme blends outputs from multiple prediction paths. Experiments across four datasets show higher F1 scores particularly on SVulD and DiverseVul, which feature wide spreads in function lengths and many vulnerability categories. This suggests the multimodal approach can locate subtle execution dependencies that single-modality models overlook. The work targets robustness when code patterns differ substantially in size and complexity.

Core claim

FusionVul integrates sequential syntactic representations extracted by a pretrained Transformer encoder with structural semantics propagated through a graph neural network, incorporates a cross-attention-based feature fusion network to enable fine-grained cross-modal interaction, and employs a sample-aware weighting mechanism to integrate multiple predictive branches, achieving superior F1 scores on datasets with highly dispersed function size distributions and broader vulnerability-type coverage such as SVulD and DiverseVul.

What carries the argument

Cross-attention-based feature fusion network that performs fine-grained interaction between sequential syntactic and graph structural code representations.

If this is right

  • Improved detection of complex vulnerability patterns in codebases with heterogeneous function sizes.
  • Better coverage of diverse vulnerability types compared with models that use only syntax or only structure.
  • Reduced oversight of execution dependencies embedded beyond surface-level token sequences.
  • More stable performance when function length distributions are wide rather than uniform.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same fusion pattern could be tested on related tasks such as defect prediction or malware classification in binaries.
  • Adding a third modality such as dynamic trace data might further increase recall on hard-to-spot vulnerabilities.
  • The weighting mechanism could be examined for its effect on calibration of vulnerability severity scores.

Load-bearing premise

Cross-attention fusion plus sample weighting will reliably extract subtle execution dependencies that single-modality models miss.

What would settle it

Running the same experiments on SVulD and DiverseVul but with the cross-attention module removed yields F1 scores no higher than the best single-modality baseline.

Figures

Figures reproduced from arXiv: 2606.08553 by Chunhua Su, Hiroshi Nomaguchi, Hongyu Yang, Jingchuan Luo, Willy Susilo, Yaping Zhu.

Figure 1
Figure 1. Figure 1: Overall framework of FusionVul. in our experiments are subject to the same length constraint. Formally, the LCS is obtained via equation (1). LCS = Tokenizer(C) = {t1, t2, . . . , tn} (1) where Tokenizer(·) denotes a standard lexical analyzer, C represents an individual source-code snippet, and the resulting token sequence is directly fed into UniXcoder for embedding and feature extraction. A schematic ill… view at source ↗
Figure 2
Figure 2. Figure 2: LCS and CPG for the example source code. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Coefficient of variation (CV) of CPG structural metrics [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Separability comparison of LCS and CPG embeddings on each dataset (MMD). [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Sensitivity Analysis of Parameters µ1 and µ2 on Devign and SVulD datasets. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
read the original abstract

Source code vulnerability detection remains a long-standing challenge due to the increasing scale, structural complexity, and semantic diversity of modern codebases. Conventional static-analysis or rule-based approaches often fail to capture subtle execution dependencies, while single-modality learning models tend to overlook critical structural information embedded beyond the lexical surface of source code. To improve robustness across heterogeneous code patterns, we propose FusionVul, a joint representation learning framework that integrates sequential syntactic representations extracted by a pretrained Transformer encoder with structural semantics propagated through a graph neural network. The framework further incorporates a cross-attention-based feature fusion network to enable fine-grained cross-modal interaction and employs a sample-aware weighting mechanism to integrate multiple predictive branches. Experimental results on four datasets demonstrate that FusionVul achieves superior F1 scores on datasets with highly dispersed function size distributions and broader vulnerability-type coverage, such as SVulD and DiverseVul, reflecting its capability to capture complex and diverse vulnerability patterns.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript proposes FusionVul, a multimodal framework for source code vulnerability detection that extracts sequential syntactic representations via a pretrained Transformer encoder and structural semantics via a graph neural network, then fuses them with a cross-attention network and integrates multiple predictive branches via sample-aware weighting. It reports experimental results on four datasets, claiming superior F1 scores on SVulD and DiverseVul (datasets with dispersed function sizes and broad vulnerability coverage) relative to single-modality baselines, attributing this to better capture of complex vulnerability patterns.

Significance. If the performance gains are shown to be attributable to the fusion mechanism rather than capacity or training differences, the work could strengthen the case for multimodal representations in vulnerability detection tasks where single-modality models miss structural dependencies. The emphasis on datasets with diverse characteristics is a positive step toward robustness claims.

major comments (3)
  1. [Abstract] Abstract: the central claim that FusionVul 'achieves superior F1 scores' on SVulD and DiverseVul is presented without any numerical F1 values, baseline names, standard deviations, or statistical significance tests, preventing verification of the empirical result.
  2. [Experimental results] Experimental results section: the interpretation that superior F1 reflects 'fine-grained cross-modal interaction' and 'capture [of] subtle execution dependencies' requires ablation experiments that remove the cross-attention module (or sample-aware weighting) while holding total parameter count and training protocol fixed; no such controlled ablations are described.
  3. [Methodology] Methodology: the sample-aware weighting mechanism is introduced to integrate predictive branches, yet the manuscript supplies no equations or pseudocode showing how per-sample weights are derived or any analysis isolating its contribution from the cross-attention component.
minor comments (1)
  1. [Abstract] Abstract: the term 'highly dispersed function size distributions' is used without a quantitative definition (e.g., variance, interquartile range) or supporting dataset statistics.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the presentation of our empirical claims and the technical details of the proposed components. We address each major comment below and will revise the manuscript to incorporate the suggested improvements.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that FusionVul 'achieves superior F1 scores' on SVulD and DiverseVul is presented without any numerical F1 values, baseline names, standard deviations, or statistical significance tests, preventing verification of the empirical result.

    Authors: We agree that the abstract would be strengthened by including concrete numerical results. In the revised version we will report the specific F1 scores (with standard deviations) achieved by FusionVul and the primary baselines on SVulD and DiverseVul. Space permitting, we will also note that statistical significance was assessed via paired t-tests. revision: yes

  2. Referee: [Experimental results] Experimental results section: the interpretation that superior F1 reflects 'fine-grained cross-modal interaction' and 'capture [of] subtle execution dependencies' requires ablation experiments that remove the cross-attention module (or sample-aware weighting) while holding total parameter count and training protocol fixed; no such controlled ablations are described.

    Authors: The referee is correct that the current manuscript does not include controlled ablations that isolate the cross-attention module while keeping total parameter count fixed. We will add these experiments in the revision, adjusting other architectural elements as needed to maintain comparable capacity, and will report the resulting performance differences. revision: yes

  3. Referee: [Methodology] Methodology: the sample-aware weighting mechanism is introduced to integrate predictive branches, yet the manuscript supplies no equations or pseudocode showing how per-sample weights are derived or any analysis isolating its contribution from the cross-attention component.

    Authors: We acknowledge the lack of explicit equations or pseudocode for the sample-aware weighting. The revised manuscript will include the full mathematical formulation for computing the per-sample weights and pseudocode for the branch integration step. We will also add an analysis that isolates the weighting component's contribution. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical framework with independent experimental claims

full rationale

The paper proposes a multimodal fusion architecture (Transformer + GNN + cross-attention + sample-aware weighting) and reports aggregate F1 improvements on four datasets. No equations, derivation steps, or first-principles results appear in the abstract or described structure. Performance numbers are presented as measured outcomes on external benchmarks (SVulD, DiverseVul, etc.), not as quantities that reduce by construction to fitted parameters or self-citations. No self-definitional loops, fitted-input-as-prediction, or load-bearing self-citation chains are present. The work is therefore self-contained against external data; the skeptic concern about missing ablations is an evidence-strength issue, not a circularity reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no information on free parameters, axioms, or invented entities.

pith-pipeline@v0.9.1-grok · 5702 in / 1060 out tokens · 29438 ms · 2026-06-27T18:09:39.687479+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 6 canonical work pages · 4 internal anchors

  1. [1]

    Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses , pages=

    Diversevul: A new vulnerable source code dataset for deep learning based vulnerability detection , author=. Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses , pages=

  2. [2]

    Empirical Software Engineering , volume=

    Aibughunter: A practical tool for predicting, classifying and repairing software vulnerabilities , author=. Empirical Software Engineering , volume=. 2024 , publisher=

    2024

  3. [3]

    Computers & Security , volume=

    Survey of source code vulnerability analysis based on deep learning , author=. Computers & Security , volume=. 2025 , publisher=

    2025

  4. [4]

    Proceedings of the 46th IEEE/ACM international conference on software engineering , pages=

    Toward improved deep learning-based vulnerability detection , author=. Proceedings of the 46th IEEE/ACM international conference on software engineering , pages=

  5. [5]

    International Journal for Research in Applied Science and Engineering Technology , volume=

    Open ai codex: An inevitable future? , author=. International Journal for Research in Applied Science and Engineering Technology , volume=

  6. [6]

    Code Llama: Open Foundation Models for Code

    Code llama: Open foundation models for code , author=. arXiv preprint arXiv:2308.12950 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence

    DeepSeek-Coder: When the Large Language Model Meets Programming--The Rise of Code Intelligence , author=. arXiv preprint arXiv:2401.14196 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    IEEE Access , year=

    Finetuning large language models for vulnerability detection , author=. IEEE Access , year=

  9. [9]

    Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles , pages=

    Knighter: Transforming static analysis with llm-synthesized checkers , author=. Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles , pages=

  10. [10]

    Knowledge-Based Systems , volume=

    A gated graph convolutional network with multi-sensor signals for remaining useful life prediction , author=. Knowledge-Based Systems , volume=. 2022 , publisher=

    2022

  11. [11]

    Proceedings of ICLR'16 , year=

    Gated Graph Sequence Neural Networks , author=. Proceedings of ICLR'16 , year=

  12. [12]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Unixcoder: Unified cross-modal pre-training for code representation , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  13. [13]

    Advances in neural information processing systems , volume=

    Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks , author=. Advances in neural information processing systems , volume=

  14. [14]

    Journal of Systems and Software , volume=

    CSGVD: A deep learning approach combining sequence and graph embedding for source code vulnerability detection , author=. Journal of Systems and Software , volume=. 2023 , publisher=

    2023

  15. [15]

    Computers & Security , volume=

    VDoTR: Vulnerability detection based on tensor representation of comprehensive code graphs , author=. Computers & Security , volume=. 2023 , publisher=

    2023

  16. [16]

    Information Fusion , volume=

    CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach , author=. Information Fusion , volume=. 2024 , publisher=

    2024

  17. [17]

    Information Fusion , volume=

    Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection , author=. Information Fusion , volume=. 2025 , publisher=

    2025

  18. [18]

    CodeBERT: A Pre-Trained Model for Programming and Natural Languages

    Codebert: A pre-trained model for programming and natural languages , author=. arXiv preprint arXiv:2002.08155 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 2002

  19. [19]

    GraphCodeBERT: Pre-training Code Representations with Data Flow

    Graphcodebert: Pre-training code representations with data flow , author=. arXiv preprint arXiv:2009.08366 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 2009

  20. [20]

    Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis , pages=

    Scale: Constructing structured natural language comment trees for software vulnerability detection , author=. Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis , pages=

  21. [21]

    IEEE Transactions on Software Engineering , volume=

    Deep learning based vulnerability detection: Are we there yet? , author=. IEEE Transactions on Software Engineering , volume=. 2021 , publisher=

    2021

  22. [22]

    Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings , pages=

    Regvd: Revisiting graph neural networks for vulnerability detection , author=. Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings , pages=

  23. [23]

    arXiv preprint arXiv:2105.05727 , year=

    Bertgcn: Transductive text classification by combining gcn and bert , author=. arXiv preprint arXiv:2105.05727 , year=

    work page arXiv

  24. [24]

    Computers & Security , volume=

    A multi-type vulnerability detection framework with parallel perspective fusion and hierarchical feature enhancement , author=. Computers & Security , volume=. 2024 , publisher=

    2024

  25. [25]

    Proceedings of the 19th international conference on mining software repositories , pages=

    Linevd: Statement-level vulnerability detection using graph neural networks , author=. Proceedings of the 19th international conference on mining software repositories , pages=

  26. [26]

    IEEE Transactions on Dependable and Secure Computing , year=

    mvulpreter: A multi-granularity vulnerability detection system with interpretations , author=. IEEE Transactions on Dependable and Secure Computing , year=

  27. [27]

    Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , pages=

    Coca: Improving and explaining graph neural network-based vulnerability detection systems , author=. Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , pages=

  28. [28]

    Proceedings of the 19th International Conference on Mining Software Repositories , pages=

    Linevul: A transformer-based line-level vulnerability prediction , author=. Proceedings of the 19th International Conference on Mining Software Repositories , pages=

  29. [29]

    International Conference on Machine Learning , pages=

    Longcoder: A long-range pre-trained language model for code completion , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  30. [30]

    Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages=

    Distinguishing look-alike innocent and vulnerable code by subtle semantic representation learning and explanation , author=. Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages=

  31. [31]

    arXiv preprint arXiv:2302.05527 , year=

    Codebertscore: Evaluating code generation with pretrained models of code , author=. arXiv preprint arXiv:2302.05527 , year=

    work page arXiv

  32. [32]

    ACM Transactions on Software Engineering and Methodology , volume=

    My fuzzers won’t build: An empirical study of fuzzing build failures , author=. ACM Transactions on Software Engineering and Methodology , volume=. 2025 , publisher=

    2025

  33. [33]

    ACM Computing Surveys , volume=

    Llms in software security: A survey of vulnerability detection techniques and insights , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    2025

  34. [34]

    Journal of Network and Computer Applications , volume=

    AI-enhanced blockchain technology: A review of advancements and opportunities , author=. Journal of Network and Computer Applications , volume=. 2024 , publisher=

    2024

  35. [35]

    Advances in neural information processing systems , volume=

    Robust fisher discriminant analysis , author=. Advances in neural information processing systems , volume=

  36. [36]

    Neural networks for signal processing IX: Proceedings of the 1999 IEEE signal processing society workshop (cat

    Fisher discriminant analysis with kernels , author=. Neural networks for signal processing IX: Proceedings of the 1999 IEEE signal processing society workshop (cat. no. 98th8468) , pages=. 1999 , organization=

    1999

  37. [37]

    The journal of machine learning research , volume=

    A kernel two-sample test , author=. The journal of machine learning research , volume=. 2012 , publisher=

    2012

  38. [38]

    Information Sciences , volume=

    Sub-domain adaptation learning methodology , author=. Information Sciences , volume=. 2015 , publisher=

    2015