Pith. sign in

REVIEW 4 major objections 4 minor 50 references

Augmenting code with minimal label-flipping counterfactual pairs makes a GNN-based vulnerability detector far more robust and its explanations align with semantically relevant statements.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

LLM-generated counterfactual code pairs with flipped vulnerability labels, used to train a GNN, sharply improve CWE-20 detection and attribution on the released CWE-20-CFA benchmark.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The headline gains are close to the trivial ceiling set by the test-set confound, so the central robustness claim is not supported as written. the 4 major comments →

arxiv 2508.18933 v1 pith:RV6I6OGR submitted 2025-08-26 cs.AI cs.CRcs.CYcs.LG

VISION: Robust and Interpretable Code Vulnerability Detection Leveraging Counterfactual Augmentation

David Egea , Barproda Halder , Sanghamitra Dutta This is my paper
classification cs.AI cs.CRcs.CYcs.LG
keywords counterfactual augmentationvulnerability detectiongraph neural networksspurious correlationcode property graphsinterpretabilityworst-group accuracyCWE-20
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that training a graph neural network on paired code examples—where each pair is a minimally edited, label-flipped version of the same function—removes most of the superficial shortcuts that make vulnerability detectors fail in the wild. It proposes VISION, which uses a large language model to generate these counterfactual pairs, converts code into property graphs, trains a GNN on the pairs, and uses a graph explainer to show which statements drove the prediction. On the CWE-20 Improper Input Validation subset, balancing original and counterfactual data raises overall accuracy from 51.8% to 97.8%, pairwise contrast accuracy from 4.5% to 95.8%, and worst-group accuracy from 0.7% to 85.5%. The paper also releases a balanced benchmark of 27,556 functions and two new attribution-based metrics for measuring whether explanations are stable and class-separated.

Core claim

The paper's central claim is that a graph neural network trained on original code samples paired with counterfactual versions—functions minimally edited so their vulnerability label flips—stops relying on spurious correlations and starts encoding the semantic difference between safe and unsafe handling of input. The authors build VISION around a GNN code classifier fed with code property graphs; a large language model produces the counterfactual rewrites, and training is done on balanced original/counterfactual pairs. On the CWE-20 (Improper Input Validation) subset, they report overall accuracy rising from 51.8% to 97.8%, pairwise contrast accuracy from 4.5% to 95.8%, and worst-group accura

What carries the argument

The load-bearing object is the counterfactual pair: an original function and a minimally edited version with the opposite vulnerability label. The pair is converted to a code property graph—a unified graph of syntax, control flow, and data dependencies—so that the model sees two nearly identical graphs that differ exactly where the vulnerability semantics change. Paired training on these examples forces the classifier to make fine-grained distinctions; the paper finds a roughly 50/50 mixture of original and generated counterfactual examples is the operating point where accuracy, pairwise contrast, and worst-group accuracy are all high. A post-hoc subgraph explainer then marks which code stat

Load-bearing premise

The result collapses if the 'minimally edited label-flipping' counterfactuals are not actually minimal and semantically valid, because then the model could be learning to recognize the LLM's editing style instead of real vulnerability semantics; the paper removes unvalidated samples but does not describe how validation was done, and concedes in Section 5 that generated counterfactuals 'may occasionally introduce unrealistic or noisy modifications.'

What would settle it

Use a held-out set of counterfactuals created by a different method or verified by human experts; if pairwise contrast accuracy and worst-group accuracy drop sharply relative to the same-pipeline test set, the gains are attributable to test-set artifacts rather than to semantic learning. A second falsifier is to train a small classifier to tell original from counterfactual functions: if it can do so with high accuracy, the augmentation carries a surface signature the GNN could exploit.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Balanced counterfactual augmentation can take a vulnerability detector from near-random pairwise discrimination to roughly 96% contrast accuracy, implying the model is picking up label-flipping edits rather than class-level heuristics.
  • Worst-group accuracy above 85% implies the learned representations are not brittle on hard subgroups of the data, addressing a failure mode of imbalanced vulnerability benchmarks.
  • The released balanced dataset gives the community a benchmark on which future augmentation and explanation methods can be compared.
  • The attribution metrics give a way to test whether a model's explanations are consistent within a class and distinct between classes, not just whether predictions are correct.
  • The interactive visualization makes counterfactual explanations inspectable by a human analyst, which is relevant for actual use in security workflows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same paired-contrast training signal could plausibly be applied to other code-understanding tasks where the goal is to detect a semantic flip, such as bug localization or API misuse, but this is a conjecture beyond the paper's CWE-20 evaluation.
  • Because the counterfactuals in the test set are produced by the same LLM pipeline used for training, the reported absolute numbers should be read as upper bounds until independent or human-verified counterfactuals confirm them.
  • The node-dependency metric may be reusable as a general probe for spurious dependencies in graph neural network explanations, even outside source code.
  • If the effect replicates across other CWEs and languages, the result would suggest that synthetic minimal edits can substitute for expensive manual clean-label data in robustness training; the paper leaves that replication open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes VISION, a framework for GNN-based CWE-20 vulnerability detection that augments a PrimeVul-derived training set with LLM-generated counterfactual code pairs, trains a Devign-style GNN on paired examples, and uses Illuminati plus a visualization module for attribution. The reported contributions are a new balanced counterfactual benchmark CWE-20-CFA (27,556 functions), large gains in accuracy (51.8%→97.8%), pairwise contrast accuracy (4.5%→95.8%), worst-group accuracy (0.7%→85.5%), and new attribution metrics (intra-class variance, inter-class distance, node-score dependency). The central claim is causal: counterfactual augmentation, rather than class balancing or generation artifacts, mitigates spurious correlations and yields robust, interpretable detection.

Significance. If the causal claim were established, the paper would offer a practical, scalable way to improve vulnerability detectors and a useful new benchmark. The code and benchmark are publicly released, and the sweep over original/counterfactual training ratios is a sensible experimental design. However, the evaluation as presented confounds vulnerability semantics with the original-vs-counterfactual source of each sample, so the headline numbers cannot be interpreted as evidence of learning CWE-20 semantics. The interpretability claims likewise are not validated against ground truth. The potential significance is real, but the current evidence does not support it.

major comments (4)
  1. [Section 4, Table 1 and 'Dataset Splitting and Augmentation Strategy'] The evaluation is confounded by the near-perfect correlation between class label and data source. In CWE-20-CFA, 13,349 of 13,778 vulnerable functions are counterfactuals (96.9%) and 13,349 of 13,778 benign functions are originals (96.9%). Since each test ID contributes both its original and its counterfactual, a trivial source-detection rule—counterfactual→vulnerable, original→benign—achieves ≈96.9% overall accuracy and ≈96.9% pairwise contrast on the fixed test set. The headline results (97.8% accuracy; 95.79% P-C) are essentially at this ceiling. The paper never reports accuracy on original-only held-out functions or source-stratified test performance, so the central claim that the model learned CWE-20 semantics rather than LLM-generation style is not supported by the reported numbers.
  2. [Section 3 'Counterfactual Generation and Augmentation' and Section 5 'Conclusions and Future Work'] Counterfactual validity is load-bearing, but the validation procedure is asserted, not described. The text says samples whose counterfactuals 'could not be reliably created or validated' were removed, without specifying the reliability or validation protocol. Section 5 concedes that LLM-generated counterfactuals 'may occasionally introduce unrealistic or noisy modifications' and defers formal verification to future work. Without a concrete validation protocol or a control experiment (e.g., random LLM perturbations using the same generation pipeline), one cannot rule out that the model is detecting generation artifacts. This is exactly the kind of shortcut that would produce the near-ceiling results.
  3. [Section 4 'Worst-Group Accuracy'] The subgroup definition is circular with respect to the model being evaluated. WGA is computed by K-means clustering the trained model's own graph embeddings, so the worst group is an artifact of the model's representation, not an externally defined spurious attribute. The authors' own observation that WGA2 'almost perfectly separates the two ground-truth classes' confirms that WGA2 essentially tracks class-level accuracy. In the presence of the source–label confound, high WGA values do not provide independent evidence that spurious correlations have been mitigated.
  4. [Section 4 'Intra-Class Attribution Variance, Inter-Class Attribution Distance, Node Score Dependency'] These metrics are proposed as measures of explanation quality, but no evidence links them to the correctness of explanations. There is no human study, no alignment with ground-truth vulnerable statements, and no comparison with an established explanation-quality metric. The qualitative claim in Figure 8 is based on a single function and is not a systematic evaluation. As with the predictive metrics, the attribution metrics are computed on a test set where label and source are confounded, so they may simply reflect the same generated-vs-original shortcut.
minor comments (4)
  1. [Figure 5 and Section 4 'Dataset Splitting and Augmentation Strategy'] The composition of the training splits is hard to follow. The text says examples are 'independently upsampled per class and data source as needed,' but no actual counts are given. A table with exact split sizes would improve reproducibility.
  2. [Section 4] All reported numbers appear to be single-run point estimates. Given the strong claims, standard deviations across random seeds (and ideally multiple dataset splits) should be reported.
  3. [Section 3] Typographical issues: 'Improper Input V alidation' contains a stray space. Several references also have artifacts such as 'Y .' in author initials.
  4. [Abstract and Section 4 'Pair-Wise Accuracy'] The abstract's 'pairwise contrast accuracy' is not explicitly tied to P-C from Table 3. The formal definition appears later, but the abstract should be consistent with the exact metric.

Circularity Check

2 steps flagged

CWE-20-CFA confounds label with counterfactual source: 96.9% of vulnerable test samples are LLM-generated counterfactuals and 96.9% of benign test samples are originals, so the reported gains can be achieved by detecting generation source rather than CWE-20 semantics; WGA is also defined from the model's own embeddings.

specific steps
  1. self definitional [Table 1; Section 4, 'Dataset Splitting and Augmentation Strategy']
    "CWE-20 CFA 13,778 13,778 27,556 ... Original 13,349 429 13,778 ... Counterfactual 429 13,349 13,778 ... The test set contains both versions of each function, ensuring perfect class balance and mirrored pairings."

    In the balanced CWE-20-CFA set, 13,349/13,778 (96.9%) of vulnerable samples are counterfactuals and 13,349/13,778 of benign samples are originals. Since the fixed test set preserves every original–counterfactual pair, the target label is almost perfectly aligned with the binary feature 'is LLM-generated'. A model that labels counterfactuals vulnerable and originals benign attains ~96.9% test accuracy and ~96.9% pair-correct contrast by construction. The reported 97.8% accuracy and 95.8% P-C are therefore consistent with a shortcut on generation source, not with learning CWE-20 semantics; no original-only or counterfactual-only evaluation is reported to rule this out.

  2. other [Section 4, 'Worst-Group Accuracy']
    "Since no explicit spurious attributes are available, we adopt an unsupervised approach to define these subgroups. First, we extract latent code embeddings from the trained model and apply K-means clustering to identify groups of structurally or stylistically similar functions."

    WGA is proposed as a robustness metric for spurious correlations, but the subgroups are defined by K-means on the trained model's own embeddings. The model being evaluated therefore determines the groups against which it is evaluated; high WGA can reflect label/source separation in the model's embedding space rather than robustness to independently defined spurious structure. This makes the metric self-referential and unable to independently support the claim of spuriousness mitigation.

full rationale

The principal reported gains—overall accuracy 51.8→97.8, P-C 4.5→95.8, WGA 0.7→85.5—are computed on a test set whose labels are nearly perfectly confounded with whether a sample is an original or an LLM-generated counterfactual. From Table 1, 96.9% of vulnerable functions are counterfactuals and 96.9% of benign functions are originals; the test set is constructed to preserve original–counterfactual pairs. Hence a trivial source detector attains ~96.9% accuracy and P-C, close to the reported numbers, so the headline 'prediction' is equivalent to a dataset-construction artifact. The paper does not report original-only or counterfactual-only accuracy, and its only 'validation' of counterfactuals is an unexplained removal step. The WGA metric is additionally self-referential because subgroups are K-means clusters of the trained model's own embeddings. The proposed attribution metrics are internal and unvalidated but not themselves circular. No load-bearing self-citation chain appears: citations to prior work by the authors (e.g., Dutta et al. 2022; Halder et al. 2024) are for definitions and background, not for the uniqueness of the method. Overall, the central claim of semantic robustness is not established independently of the augmentation pipeline; it partially reduces to source detection by construction. Score 6 rather than higher because a small fraction of original vulnerable and counterfactual benign samples exist, so the method could in principle learn real semantics; however, the reported evidence does not separate that from the source shortcut.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 4 invented entities

The ledger shows 6 free parameters, 6 axioms, and 4 invented entities. The dominant load-bearing axiom is the unverified semantic validity of LLM-generated counterfactuals, which the paper itself flags in Section 5. The evaluation constructs are partly self-referential (WGA clusters from the model's own embeddings; attribution metrics defined by the authors) and the benchmark composition depends on an unspecified filtering choice. No formal derivation is present; that is expected for an empirical ML paper, but it means the novelty and soundness rest entirely on experimental design, where the gaps above sit.

free parameters (6)
  • K-means cluster count k for WGA = swept 2 through 7
    WGA is reported for k=2..7; the headline 85.5% uses k=3 for the 50/50 split while the 0.7% baseline uses a different k, so the choice of which k to headline is post hoc.
  • WGA group size threshold = 1% of total data
    Hand-chosen cutoff; groups with fewer than 1% of samples are discarded, which determines which cluster becomes the worst group.
  • Counterfactual filtering criteria = unspecified
    The paper removes samples whose counterfactuals 'could not be reliably created or validated' (Section 3) but never defines the criteria, so the benchmark composition depends on an unreported free choice.
  • LLM prompt template and decoding settings = not stated
    Counterfactual generation uses GPT-4o-mini with a 'dynamically built' prompt, but the prompt text, temperature, and sampling settings are not given; these determine the entire dataset.
  • Training split composition sweep = 10% increments from 100/0 to 0/100
    The sweep itself is a free axis; the paper reports the best configurations (60/40 for accuracy, 50/50 for pairwise) and the abstract headline mixes them.
  • Devign and Word2Vec hyperparameters = not stated in text
    Embedding dimension, GNN layers, learning rate, and epochs are not reported, preventing exact replication.
axioms (6)
  • ad hoc to paper Each LLM-generated counterfactual is a valid, label-flipping minimal edit of its source function
    The whole method rests on this. No human, test-based, or formal verification is reported; Section 5 defers formal verification to future work and concedes 'unrealistic or noisy modifications'.
  • domain assumption PrimeVul labels for CWE-20 are accurate enough for benchmarking
    The framework inherits human-verified labels from PrimeVul (Section 3); the paper itself describes label noise in other datasets, so this is a maintained assumption rather than an established fact.
  • ad hoc to paper K-means clusters on the trained model's embeddings approximate the true spurious subgroups
    WGA requires known spurious attributes (Idrissi et al. 2022); the paper substitutes unsupervised clusters from the model under evaluation, entangling the metric with the model.
  • ad hoc to paper The proposed attribution metrics measure explanation quality
    Intra-class variance, inter-class distance, and node score dependency are introduced in this paper and not validated against ground-truth vulnerable statements or human judgments, yet are used as evidence for the framework.
  • domain assumption Joern-produced CPGs and the Illuminati explainer behave as documented
    The pipeline assumes third-party tools correctly serialize code and attribute importance; standard practice, but unverified here.
  • ad hoc to paper Counterfactual generation artifacts do not provide a shortcut exploitable on the test set
    Test and training counterfactuals come from the same GPT-4o-mini pipeline; if consistent artifacts (naming, formatting) leak through, pairwise accuracy measures artifact detection rather than vulnerability understanding.
invented entities (4)
  • Intra-class attribution variance metric no independent evidence
    purpose: Measures consistency of explainer attributions within a class; used as evidence that augmentation produces more stable reasoning.
    Introduced in this paper only; no external validation that lower intra-class variance corresponds to better explanations.
  • Inter-class attribution distance metric no independent evidence
    purpose: Measures separability of average attribution vectors between benign and vulnerable classes; used as evidence of semantically discriminative reasoning.
    Introduced in this paper; the favorable 50/50 result is interpreted as support for the method with no external benchmark.
  • Node score dependency metric no independent evidence
    purpose: Quantifies how the attribution score of node j changes when node i is removed; used to expose spurious dependencies such as the EPERM focus in the 100/0 model.
    New metric; interpretation of the heatmaps is qualitative and no ground-truth dependency structure is available.
  • CWE-20-CFA benchmark independent evidence
    purpose: Released 27,556-sample balanced benchmark of original and counterfactual functions for CWE-20.
    A released artifact other groups can probe; however, its validity inherits the unverified counterfactual filtering process.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of VISION: Robust and Interpretable Code Vulnerability Detection Leveraging Counterfactual Augmentation." pith.science (2026). https://pith.science/paper/RV6I6OGR

@misc{pith2026250818933,
  author       = {Pith},
  title        = {Pith review of: VISION: Robust and Interpretable Code Vulnerability Detection Leveraging Counterfactual Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RV6I6OGR}},
  note         = {Machine review of arXiv:2508.18933}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Automated detection of vulnerabilities in source code is an essential cybersecurity challenge, underpinning trust in digital systems and services. Graph Neural Networks (GNNs) have emerged as a promising approach as they can learn structural and logical code relationships in a data-driven manner. However, their performance is severely constrained by training data imbalances and label noise. GNNs often learn 'spurious' correlations from superficial code similarities, producing detectors that fail to generalize well to unseen real-world data. In this work, we propose a unified framework for robust and interpretable vulnerability detection, called VISION, to mitigate spurious correlations by systematically augmenting a counterfactual training dataset. Counterfactuals are samples with minimal semantic modifications but opposite labels. Our framework includes: (i) generating counterfactuals by prompting a Large Language Model (LLM); (ii) targeted GNN training on paired code examples with opposite labels; and (iii) graph-based interpretability to identify the crucial code statements relevant for vulnerability predictions while ignoring spurious ones. We find that VISION reduces spurious learning and enables more robust, generalizable detection, improving overall accuracy (from 51.8% to 97.8%), pairwise contrast accuracy (from 4.5% to 95.8%), and worst-group accuracy (from 0.7% to 85.5%) on the Common Weakness Enumeration (CWE)-20 vulnerability. We further demonstrate gains using proposed metrics: intra-class attribution variance, inter-class attribution distance, and node score dependency. We also release CWE-20-CFA, a benchmark of 27,556 functions (real and counterfactual) from the high-impact CWE-20 category. Finally, VISION advances transparent and trustworthy AI-based cybersecurity systems through interactive visualization for human-in-the-loop analysis.

Figures

Figures reproduced from arXiv: 2508.18933 by Barproda Halder, David Egea, Sanghamitra Dutta.

Figure 2
Figure 2. Figure 2: CWE-20 Improper Input Validation example. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 1
Figure 1. Figure 1: Complete architecture of our VISION Framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of a counterfactual code pair used in data augmentation. The top function is benign, safely invoking [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Integrated visualization module showcasing model predictions and explanation scores for an original benign function [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Illustration of spurious correlation in source code. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: t-SNE projections of graph embeddings for benchmarks 100/0 (left), 50/50 (center), and 30/70 (right). Green points [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Node Score Dependency heatmaps for the same vulnerable function under two training regimes: Left: Benchmark [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 31 canonical work pages · 8 internal anchors

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    The Pragmatic Frames of Spurious Correlations in Machine Learning: Interpreting How and Why They Matter

    Bell, S. J.; and Wang, S. 2024. The Multiple Dimensions of Spuriousness in Machine Learning. arXiv:2411.04696

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  4. [4]

    Chen, Y.; Bian, Y.; Zhou, K.; Xie, B.; Han, B.; and Cheng, J. 2023 a . Does invariant graph learning via environment augmentation learn invariance? Advances in Neural Information Processing Systems, 36: 71486--71519

    work page 2023

  5. [5]

    Chen, Y.; Ding, Z.; Alowain, L.; Chen, X.; and Wagner, D. 2023 b . DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection. arXiv:2304.00409

    Pith/arXiv arXiv 2023

  6. [6]

    Chernis, B.; and Verma, R. 2018. Machine Learning Methods for Software Vulnerability Detection. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics, IWSPA '18, 31–39. New York, NY, USA: Association for Computing Machinery. ISBN 9781450356343

    work page 2018

  7. [7]

    Chu, Z.; Wan, Y.; Li, Q.; Wu, Y.; Zhang, H.; Sui, Y.; Xu, G.; and Jin, H. 2024. Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanation. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA ’24, 389–401. ACM

    work page 2024

  8. [8]

    Data Quality for Software Vulnerability Datasets

    Croft, R.; Babar, M. A.; and Kholoosi, M. 2023. Data Quality for Software Vulnerability Datasets. arXiv:2301.05456

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  9. [9]

    S.; Nong, Y.; Yang, X.; Wang, S.; and Cai, H

    Daneshvar, S. S.; Nong, Y.; Yang, X.; Wang, S.; and Cai, H. 2024. Exploring RAG-based Vulnerability Augmentation with LLMs. arXiv:2408.04125

    Pith/arXiv arXiv 2024

  10. [10]

    Ding, K.; Xu, Z.; Tong, H.; and Liu, H. 2022. Data augmentation for deep graph learning: A survey. ACM SIGKDD Explorations Newsletter, 24(2): 61--77

    2022

  11. [11]

    Ding, Y.; Fu, Y.; Ibrahim, O.; Sitawarin, C.; Chen, X.; Alomair, B.; Wagner, D.; Ray, B.; and Chen, Y. 2024. Vulnerability Detection with Code Language Models: How Far Are We? arXiv:2403.18624

    Pith/arXiv arXiv 2024

  12. [12]

    Dissanayake, P.; and Dutta, S. 2024. Model reconstruction using counterfactual explanations: A perspective from polytope theory. Advances in Neural Information Processing Systems, 37: 83397--83429

    work page 2024

  13. [13]

    Dutta, S.; Long, J.; Mishra, S.; Tilli, C.; and Magazzeni, D. 2022. Robust counterfactual explanations for tree-based ensembles. In International conference on machine learning, 5742--5756. PMLR

    work page 2022

  14. [14]

    Feng, Z.; Guo, D.; Tang, D.; Duan, N.; Feng, X.; Gong, M.; Shou, L.; Qin, B.; Liu, T.; Jiang, D.; and Zhou, M. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. arXiv:2002.08155

    Pith/arXiv arXiv 2020

  15. [15]

    Ganz, T.; Imgrund, E.; Harterich, M.; and Rieck, K. 2023. CodeGraphSMOTE - Data Augmentation for Vulnerability Discovery. In Data and Applications Security and Privacy XXXVII: 37th Annual IFIP WG 11.3 Conference, DBSec 2023, Sophia-Antipolis, France, July 19–21, 2023, Proceedings, 282–301. Berlin, Heidelberg: Springer-Verlag. ISBN 978-3-031-37585-9

    work page 2023

  16. [16]

    K.; Clement, C.; Drain, D.; Sundaresan, N.; Yin, J.; Jiang, D.; and Zhou, M

    Guo, D.; Ren, S.; Lu, S.; Feng, Z.; Tang, D.; Liu, S.; Zhou, L.; Duan, N.; Svyatkovskiy, A.; Fu, S.; Tufano, M.; Deng, S. K.; Clement, C.; Drain, D.; Sundaresan, N.; Yin, J.; Jiang, D.; and Zhou, M. 2021. GraphCodeBERT: Pre-training Code Representations with Data Flow. arXiv:2009.08366

    Pith/arXiv arXiv 2021

  17. [17]

    Guo, Y.; and Bettaieb, S. 2023. An Investigation of Quality Issues in Vulnerability Detection Datasets. In 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), 29–33. IEEE

    work page 2023

  18. [18]

    Haig, B. D. 2003. What is a spurious correlation? Understanding Statistics: Statistical Issues in Psychology, Education, and the Social Sciences, 2(2): 125--132

    work page 2003

  19. [19]

    Halder, B.; Hamman, F.; Dissanayake, P.; Zhang, Q.; Sucholutsky, I.; and Dutta, S. 2024. Quantifying spuriousness of biased datasets using partial information decomposition. arXiv preprint arXiv:2407.00482

    work page arXiv 2024

  20. [20]

    Hamman, F.; Noorani, E.; Mishra, S.; Magazzeni, D.; and Dutta, S. 2023. Robust Counterfactual Explanations for Neural Networks With Probabilistic Guarantees. In International Conference on Machine Learning, 12351--12367. PMLR

    work page 2023

  21. [21]

    He, H.; Ji, Y.; and Huang, H. H. 2023. Illuminati: Towards Explaining Graph Neural Networks for Cybersecurity Analysis. arXiv:2303.14836

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  22. [22]

    Y.; Arjovsky, M.; Pezeshki, M.; and Lopez-Paz, D

    Idrissi, B. Y.; Arjovsky, M.; Pezeshki, M.; and Lopez-Paz, D. 2022. Simple data balancing achieves competitive worst-group-accuracy. arXiv:2110.14503

    Pith/arXiv arXiv 2022

  23. [23]

    Kaushik, D.; Hovy, E.; and Lipton, Z. C. 2020. Learning the Difference that Makes a Difference with Counterfactually-Augmented Data. arXiv:1909.12434

    Pith/arXiv arXiv 2020

  24. [24]

    Kong, K.; Li, G.; Ding, M.; Wu, Z.; Zhu, C.; Ghanem, B.; Taylor, G.; and Goldstein, T. 2022. Robust optimization as data augmentation for large-scale graphs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 60--69

    work page 2022

  25. [25]

    Li, Y.; Wang, S.; and Nguyen, T. N. 2021. Vulnerability detection with fine-grained interpretations. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 292–303. ACM

    work page 2021

  26. [26]

    Liu, G.; Zhao, T.; Xu, J.; Luo, T.; and Jiang, M. 2022 a . Graph rationalization with environment-based augmentations. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 1069--1078

    work page 2022

  27. [27]

    Liu, L.; Li, Z.; Wen, Y.; and Chen, P. 2022 b . Investigating the Impact of Vulnerability Datasets on Deep Learning-Based Vulnerability Detectors. PeerJ Computer Science, 8: e975

    work page 2022

  28. [28]

    Liu, S.; Ma, W.; Wang, J.; Xie, X.; Feng, R.; and Liu, Y. 2024. Enhancing Code Vulnerability Detection via Vulnerability-Preserving Data Augmentation. arXiv:2404.09599

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Lundberg, S.; and Lee, S.-I. 2017. A Unified Approach to Interpreting Model Predictions. arXiv:1705.07874

    Pith/arXiv arXiv 2017

  30. [30]

    MITRE . 2022. 2022 CWE Top 25 Most Dangerous Software Weaknesses . https://cwe.mitre.org/top25/archive/2022/2022_cwe_top25.html

    work page 2022

  31. [31]

    MITRE . 2025. CWE-20: Improper Input Validation. https://cwe.mitre.org/data/definitions/20.html. Accessed: 2025-05-23

    work page 2025

  32. [32]

    Neuhaus, S.; Zimmermann, T.; Holler, C.; and Zeller, A. 2007. Predicting vulnerable software components. In Proceedings of the 14th ACM Conference on Computer and Communications Security, CCS '07, 529–540. New York, NY, USA: Association for Computing Machinery. ISBN 9781595937032

    work page 2007

  33. [33]

    Why Should I Trust You?

    Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. arXiv:1602.04938

    Pith/arXiv arXiv 2016

  34. [34]

    Ross, A.; Marasović, A.; and Peters, M. E. 2021. Explaining NLP Models via Minimal Contrastive Editing (MiCE). arXiv:2012.13985

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  35. [35]

    C.; Hagenbuchner, M.; and Monfardini, G

    Scarselli, F.; Gori, M.; Tsoi, A. C.; Hagenbuchner, M.; and Monfardini, G. 2009. The Graph Neural Network Model. IEEE Transactions on Neural Networks, 20(1): 61--80

    work page 2009

  36. [36]

    K.; Mishra, J.; Singh, A.; Govil, R.; Srivastava, G.; and Lin, J

    Sharma, D. K.; Mishra, J.; Singh, A.; Govil, R.; Srivastava, G.; and Lin, J. C.-W. 2022. Explainable Artificial Intelligence for Cybersecurity. Computers and Electrical Engineering, 103: 108356

    work page 2022

  37. [37]

    Steinmann, D.; Divo, F.; Kraus, M.; Wüst, A.; Struppek, L.; Friedrich, F.; and Kersting, K. 2024. Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation. arXiv:2412.05152

    Pith/arXiv arXiv 2024

  38. [38]

    Sultana, S.; Afreen, S.; and Eisty, N. U. 2024. Code Vulnerability Detection: A Comparative Analysis of Emerging Large Language Models. arXiv:2409.10490

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  39. [39]

    Temraz, M.; and Keane, M. T. 2021. Solving the Class Imbalance Problem Using a Counterfactual Method for Data Augmentation. arXiv:2111.03516

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  40. [40]

    Verma, S.; Dickerson, J.; and Hines, K. 2020. Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:2010.10596

    Pith/arXiv arXiv 2020

  41. [41]

    N.; and Thai, M

    Vu, M. N.; and Thai, M. T. 2020. PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks. arXiv:2010.05788

    Pith/arXiv arXiv 2020

  42. [42]

    Wachter, S.; Mittelstadt, B.; and Russell, C. 2018. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. arXiv:1711.00399

    Pith/arXiv arXiv 2018

  43. [43]

    Wang, J.; Huang, M.; Nie, Y.; Kuang, X.; Li, X.; and Zhong, W. 2023. Fine-Grained Source Code Vulnerability Detection via Graph Neural Networks

    work page 2023

  44. [44]

    Wu, Y.; Zou, D.; Dou, S.; Yang, W.; Xu, D.; and Jin, H. 2022. VulCNN: An Image-inspired Scalable Vulnerability Detection System. In 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE), 2365--2376

    work page 2022

  45. [45]

    Yamaguchi, F.; Golde, N.; Arp, D.; and Rieck, K. 2014. Modeling and Discovering Vulnerabilities with Code Property Graphs. In 2014 IEEE Symposium on Security and Privacy, 590--604

    work page 2014

  46. [46]

    Ye, W.; Zheng, G.; Cao, X.; Ma, Y.; and Zhang, A. 2024. Spurious Correlations in Machine Learning: A Survey. arXiv:2402.12715

    arXiv 2024

  47. [47]

    Ying, R.; Bourgeois, D.; You, J.; Zitnik, M.; and Leskovec, J. 2019. GNNExplainer: Generating Explanations for Graph Neural Networks. In NeurIPS

    work page 2019

  48. [48]

    Zhou, X.; Zhang, T.; and Lo, D. 2024. Large Language Model for Vulnerability Detection: Emerging Results and Future Directions. arXiv:2401.15468

    Pith/arXiv arXiv 2024

  49. [49]

    Zhou, Y.; Liu, S.; Siow, J.; Du, X.; and Liu, Y. 2019. Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks. arXiv:1909.03496

    Pith/arXiv arXiv 2019

  50. [50]

    Ziems, N.; and Wu, S. 2021. Security Vulnerability Detection Using Deep Learning Natural Language Processing. arXiv:2105.02388

    work page internal anchor Pith review Pith/arXiv arXiv 2021

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.