Pith. sign in

REVIEW 4 major objections 6 minor 58 references

Can Highlighting Help GitHub Maintainers Track Security Fixes?

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a simple TfIdf-based token highlight is more faithful to a trained patch-retrieval model than LIME, yet highlighting does not improve humans' ability to identify the correct security patch.

desk verdict Useful dataset and a clean negative human result; the headline faithfulness claim is likely confounded by the Tf-Idf chunk selection and needs an ablation. read the letter →

arxiv 2411.11646 v1 pith:5NCDDKAM submitted 2024-11-18 cs.CR cs.SE

Xueqing Liu , Yuchen Xiong , Qiushi Liu , Jiangrui Zheng This is my paper · ORCID
classification cs.CRcs.SE
keywords securityvulnerabilitypatchtracingexplainableinformationretrievalTfIdf-HighlightLIMEfaithfulnessCodeBERTCVENVDdelays
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether highlighting tokens in a commit message and diff can help security maintainers find the commit that fixes a known vulnerability. The authors build a retrieval system that ranks candidate commits for a CVE using CodeBERT or UnixCoder, then compare two ways of explaining the ranking: LIME, a standard model-agnostic explainer, and their own TfIdf-Highlight, which scores tokens by term frequency weighted by how rare they are within the CVE's candidates and across the whole dataset. They report that TfIdf-Highlight is significantly more faithful to the trained model than LIME, improving sufficiency by about 15% in most settings. But in a blind human-labeling study, neither highlight method improved annotators' accuracy at picking the patch, and with the commit message revealed the no-highlight condition was actually more accurate (0.86 vs 0.76). If true, this means token-level highlighting can reveal a black-box model's reasoning without hurting ranking performance, yet it is not enough to support the semantic judgments humans need to verify a patch.

What carries the argument

The load-bearing object is TfIdf-Highlight, a parameter-free scoring rule that ranks tokens by a product of three factors: log term frequency in the commit message and diff, an idf computed within the candidate commits of the same CVE ($\mathrm{idf}_{\mathrm{CVE}}$), and an idf computed across the entire dataset. This score selects the top-$k$ tokens to highlight. The argument also relies on a faithfulness metric defined for ranking—the change in precision@1 between the original model probabilities and probabilities computed from only the highlighted tokens (sufficiency) or from the input with highlighted tokens removed (comprehensiveness)—which lets the authors compare explainers without ground-truth token labels.

What would settle it

Run the same retrieval and highlighting pipeline on a candidate set filtered by NVD's published version tags instead of the true patched version, and measure whether retrieval recall and the faithfulness advantage of TfIdf-Highlight over LIME persist; Table I's coverage figures (60.2% at range 0, 76.7% at ±5) predict a substantial drop. Alternatively, a larger human study (more annotators, more CVE groups, and a preregistered analysis) could test whether the observed null or negative effect of highlighting on labeling accuracy is real or an artifact of three annotators and 50–100 examples.

Watch

Extended reading notes

Core claim

The central claim is that a model-agnostic token-overlap score, TfIdf-Highlight, is more faithful to a trained patch-retrieval model than LIME's locally weighted regression, and that this faithfulness does not translate into better human decision-making. Concretely, the authors define faithfulness for ranking as the change in precision@1 when the model is given only the highlighted tokens instead of the full input (sufficiency) or when the highlighted tokens are removed (comprehensiveness). Across CodeBERT and UnixCoder, on validation and test folds, TfIdf-Highlight lowers sufficiency scores by roughly 15% relative to LIME (e.g., from 0.398 to 0.286 for CodeBERT text-only validation), meaning the highlighted tokens better reproduce the model's ranking decision. A blind human experiment with three annotators and 50–100 groups of commits finds that TfIdf-Highlight receives higher helpfulness ratings than LIME, but accuracy is similar, and both are no better than no highlighting; with commit messages revealed, no-highlight accuracy is 0.86 versus 0.76 with highlighting. The authors conclude that token-level matching is insufficient for patch tracing, which requires understanding semantic relatedness between the CVE description and the commit.

Load-bearing premise

The candidate commit pool for every CVE is built using the true first patched version tag, which the system would not know in real deployment; NVD's own version tags cover only about 60% of CVEs at the exact version and 77% within five versions.

Editorial extensions

If this is right

  • A retrieval model fine-tuned on commit messages and diff chunks can rank the true patch near the top: CodeBERT and UnixCoder reach precision@1 around 0.66, roughly 15 points above pure TfIdf.
  • TfIdf-Highlight, despite being model-agnostic and simpler than LIME, more faithfully reproduces the model's ranking decisions in almost all settings tested.
  • Because humans do not label patches more accurately with highlighting, token-level highlights should not be expected to replace maintainer review; at best they shorten the search.
  • For humans, the commit message carries more signal than diff code: revealing it raised accuracy, and the helpfulness of highlighting was judged higher when the message was present.
  • The proposed faithfulness score for ranking (change in precision@1 with highlighted tokens only or excluded) can be reused to evaluate other explainable retrieval systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reliance on the true patched version tag as an oracle means the reported retrieval and faithfulness numbers are an upper bound for what a deployable system would achieve; integrating automatic version estimation (as in Table I's 60–77% coverage) would directly discount the headline gains.
  • The negative human result suggests a testable hypothesis: explanations that convey semantic relatedness (e.g., generated natural-language rationales or CWE-aware links) would outperform token highlighting, a direction the authors flag as future work.
  • TfIdf-Highlight's within-CVE idf term is a transferable trick for any retrieval explanation task: penalize words that are frequent among the candidates for a single query, since those cannot discriminate among them.
  • Because the authors find modality bias in multi-modal training (the model initially ignored code and relied on commit messages), their data augmentation forcing the model to learn from code is a reusable recipe for other code-plus-text retrieval models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies whether token-level highlighting helps maintainers trace the commit that fixes a CVE. The authors construct a dataset of 3,573 (CVE, patch) pairs with candidate commit pools derived from version tags, fine-tune CodeBERT and UnixCoder as multi-modal retrieval models, and compare two explanation methods: LIME and a proposed model-agnostic TfIdf-Highlight. The evaluation uses a faithfulness metric (sufficiency and comprehensiveness) defined for ranking, plus blind human labeling experiments with a no-highlight control. The reported findings are that TfIdf-Highlight significantly improves sufficiency over LIME by about 15% in most settings, that TfIdf-Highlight receives higher helpfulness ratings than LIME, and that highlighting does not improve human labeling accuracy over no highlighting, sometimes reducing it.

Significance. If the results hold, the paper makes a useful empirical contribution to security database maintenance: it provides a large patch-tracing dataset, shows that dual-encoder retrieval models can rank candidate commits, and, importantly, offers a negative human result that token-overlap highlighting does not help and can even hurt maintainers' decisions. The blind user study with a no-highlight control is a strength, as is the attempt to define faithfulness for ranking rather than classification. The main concerns are that the candidate pools are built with an oracle version tag and that the central faithfulness comparison is confounded by the TfIdf-based input-chunk selection; these issues make the quantitative claims conditional in ways the current paper does not fully address.

major comments (4)
  1. [Section IV-B, Table I] The evaluation of the retrieval system and of both highlighting studies is built on candidate-commit pools constructed with the true patch version tag (Method 2), which is not available in the deployment scenario described in the introduction. Table I shows that using NVD version tags alone reaches only 60.2% coverage at range 0 and 76.7% at ±5, so the reported retrieval and faithfulness numbers are conditional on an oracle that a maintainer would not have. The paper explicitly acknowledges this at the end of Section IV-B, but the consequence is that RQ1 and RQ2 do not currently demonstrate performance for the actual NVD-assisted setting; please report the main results under Method 1 or under a noisy-version-tag protocol (e.g., ranging over ±1 and ±5) to show what a real deployment would achieve.
  2. [Section V-A, Section V-C, Section VI-B] The central RQ2 claim that TfIdf-Highlight is more faithful than LIME is confounded by the input-construction pipeline. Section V-A selects, for each diff, only the top-1 64-token chunk by TfIdf cosine similarity to the CVE description, and Section V-C's TfIdf-Highlight then highlights tokens with high TfIdf overlap from that same chunk. The sufficiency metric in Section VI-B replaces the explained component with only the highlighted tokens and measures rank preservation, so TfIdf-Highlight is being rewarded for re-selecting exactly the lexical-overlap signal that determined the model's input, while LIME is evaluated through black-box perturbations without that privileged access. Because no model is trained or evaluated with a non-TfIdf-selected chunk, the reported ~15% sufficiency gain cannot be attributed to a general explanation-quality advantage; please add an ablation using randomly chosen chunks, full-diff inputs, or a non-TfIdf chunk selector, and/or restrict both explainers to the same candidate-token vocabulary.
  3. [Section VI-C, Tables V-VI] The human labeling experiment is the only external grounding for the faithfulness comparison, but it is small (100 easy groups and 50 hard groups), all annotators are authors of the paper, and the hard experiment's accuracy differences (e.g., overall 0.76 with highlighting vs 0.86 without, with commit message revealed in Table VI) are reported without confidence intervals or significance tests. The paper's Section IX acknowledges the sample-size limitation, but the conclusion that highlighting does not improve labeling accuracy is a central negative result; please report per-annotator intervals, inter-annotator agreement, and a test such as McNemar or a bootstrap so the reader can assess the strength of the null result.
  4. [Section VI-C, Figure 5] The user study does not evaluate the trained retrieval model's ranked list: in the first experiment the distractor commit is the top-2 commit under TF-IDF, and the second experiment's groups are only filtered by file type rather than being sampled from the CodeBERT/UnixCoder ranking. Since the highlights are explanations of a neural model's decisions, but the candidate sets are generated by a different TF-IDF ranking, the human results measure how highlighting helps in a small ad hoc candidate pool rather than in the explainable retrieval system the paper proposes. Please clarify why the candidate selection in the user study is not tied to the trained model's ranking, or rerun the study on candidates that are near-misses of the neural model.
minor comments (6)
  1. [Section VI-C] The sentence 'The results in Table IV indicate that TfIdf-highlight receives higher average helpfulness ratings than LIME' should refer to Table V, not Table IV; Table IV reports faithfulness scores.
  2. [Section VI-B, Summary of Findings for RQ2] The summary says 'TfIdf-Highlight significantly outperforms BERT in the sufficiency score', but the comparison is against LIME; 'BERT' should be 'LIME'.
  3. [Section V-C, TfIdf-Highlight equations] The formula for the message modality uses logtf(w,d)=0.5 log2(tf(w,d_msg))+0.5 log2(tf(w,d_diff)), whereas the code modality uses tf(w,d) without the log transform; please clarify whether this asymmetry is intentional and how it affects the highlighted-token counts.
  4. [Section V-C] The definition of dfCVE(w,q) says 'the number of comments under the CVE q' but appears to mean the number of commits under that CVE; please correct the wording.
  5. [Section VI-C, Table VI] There is a typo in the text: 'TfIdf-Highlight has aa worse accuracy than LIME' should read 'a worse accuracy'.
  6. [Related Work] The abbreviation XML is used both for 'explainable machine learning' in the introduction and for 'extreme multi-class classification' in Section VII, which can confuse readers; please disambiguate or use different abbreviations.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation reduces to its inputs; the 15% TfIdf-Highlight gain is confounded by shared Tf-Idf chunk selection, but that is a validity concern rather than circular reasoning; the only self-citation is minor and non-load-bearing.

full rationale

The paper's derivation chain does not contain a step in which a predicted quantity is defined by or fitted to the target it claims to explain. TfIdf-Highlight (Section V-C) is a fixed Tf-Idf overlap scorer; it contains no parameters fitted to the trained retrieval model or to the faithfulness metric, and it does not use the model at all. LIME is also used as an off-the-shelf explainer. The faithfulness evaluation (Section VI-B) measures how well each method's highlighted tokens preserve the model's ranking, while the blind human labeling study (Section VI-C) provides external grounding independent of the trained model. The clearest threats are the oracle construction of candidate commits (Section IV-B, Method 2: "we end up leveraging the true version tag") and the possibility that TfIdf-Highlight's advantage in Table IV is confounded with the Tf-Idf chunk selection used to build the model input (Section V-A: "we rank all chunks based on their Tf-Idf cosine similarity... we only use the top-1 chunk in the model"). These are evaluation confounds, not circular derivations: no equation identifies the faithfulness score with the chunk-selection criterion, and no fitted parameter is renamed as a prediction. The paper itself acknowledges the ground-truth rationale limitation (Section VI-B: "we have not collected the ground truth labeling and will leave it for future work"), but the human study mitigates the concern that the faithfulness comparison is purely self-referential. The only self-citation found is Reference [19] in the related-work discussion of named-entity recognition for vulnerability reports; it is not load-bearing. I therefore assign score 2, consistent with a non-circular paper that has one minor non-load-bearing self-citation.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central empirical result rests mostly on dataset-construction choices rather than mathematical axioms. The key load-bearing assumptions are that the true patch version tag can be used to build candidate sets, that database patch links are correct, that CodeBERT and UnixCoder capture semantic relatedness, and that faithfulness metrics proxy explanation quality. Hyperparameters such as alpha, beta, chunk size, and class weights are hand-chosen.

free parameters (8)
  • alpha (idfCVE penalty) = 1
    Hand-chosen threshold in TfIdf-Highlight to penalize words appearing in more than half of the commits under a CVE; central to which tokens are highlighted.
  • beta (idfall penalty) = 0.01
    Hand-chosen threshold to penalize globally frequent words; affects highlight selection.
  • chunk size for diff code = 64 tokens
    Diff chunks are truncated to 64 tokens to fit model context and to help LIME; choice affects which code is seen by the model and highlighted.
  • top-1 chunk selection = 1
    Only the highest Tf-Idf chunk per commit is used; discards most of the diff, a hand-chosen simplification.
  • Tf-Idf baseline weights = 0.8 message / 0.2 diff
    Used for the baseline and for initial chunk ranking; hand-chosen.
  • training class weight = 10:1 positive:negative
    Chosen to handle the 1:45 imbalance; affects model probabilities and thus faithfulness scores.
  • data augmentation counts = 2 per positive; 10x negative
    Added to counter modality bias; hand-chosen and affects model behavior on code-only inputs.
  • fine-tuning hyperparameters = lr=2e-5, batch=8, epochs=5, max_len=256
    Standard BERT fine-tuning choices; not swept.
assumptions (5)
  • ad hoc to paper The true patch version tag is known when forming candidate commit pools (Section IV-B, Method 2).
    The paper explicitly says this information is not provided in real applications; retrieval and highlight results depend on this oracle.
  • domain assumption The patch links extracted from BigVul, patchdb, NVD, GitHub Advisory, and OSV are correct patch commits.
    Conflict resolution gives priority to BigVul and patchdb labels; no independent verification beyond database trust.
  • domain assumption CodeBERT and UnixCoder pooled embeddings capture the semantic relatedness between CVE descriptions and commits.
    The retrieval model treats these embeddings as sufficient representations for classification and ranking.
  • domain assumption Token-level overlap between CVE description and commit is a useful signal for highlighting.
    TfIdf-Highlight selects only words in q intersect d; the human study tests and partially falsifies this for accuracy.
  • domain assumption Faithfulness (sufficiency and comprehensiveness) is a valid measure of explanation quality.
    Authors cite DeYoung et al. and Jacovi and Goldberg; no ground-truth highlight labels are collected.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can Highlighting Help GitHub Maintainers Track Security Fixes?." pith.science (2026). https://pith.science/paper/5NCDDKAM

@misc{pith2026241111646,
  author       = {Pith},
  title        = {Pith review of: Can Highlighting Help GitHub Maintainers Track Security Fixes?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5NCDDKAM}},
  note         = {Machine review of arXiv:2411.11646}
}
read the original abstract

In recent years, the rapid growth of security vulnerabilities poses great challenges to tracing and managing them. For example, it was reported that the NVD database experienced significant delays due to the shortage of maintainers. Such delay creates challenges for third-party security personnel (e.g., administrators) to trace the information related to the CVE. To help security personnel trace a vulnerability patch, we build a retrieval system that automatically retrieves the patch in the repository. Inspired by existing work on explainable machine learning, we ask the following research question: can explanations help security maintainers make decisions in patch tracing? First, we investigate using LIME (a widely used explainable machine learning method) to highlight the rationale tokens in the commit message and code. In addition, we propose an explanation method called TfIdf-Highlight, which leverages the Tf-Idf statistics to select the most informative words in the repository and the dataset. We evaluate the effectiveness of highlighting using two experiments. First, we compare LIME and TfIdf-Highlight using a faithfulness score (i.e., sufficiency and comprehensiveness) defined for ranking. We find that TfIdf-Highlight significantly outperforms LIME's sufficiency scores by 15\% and slightly outperforms the comprehensiveness scores. Second, we conduct a blind human labeling experiment by asking the annotators to guess the patch under 3 settings (TfIdf-Highlight, LIME, and no highlight). We find that the helpfulness score for TfIdf-Highlight is higher than LIME while the labeling accuracies of LIME and TfIdf-Highlight are similar. Nevertheless, highlighting does not improve the accuracy over non-highlighting.

Figures

Figures reproduced from arXiv: 2411.11646 by the authors.

Figure 1
Figure 1. An example of NVD’s missing patch link and exposing individuals to potential identity theft [4]. The attack was exploited because Equifax did not patch an Apache Strut vulnerability whose patch was already available. As a result, it is important for OSS users to be alerted of the updates of vulnerabilities and apply the patches in time. For the same reason, if a patch is available, it is important for security advis… view at source ↗
Figure 1
Figure 1. The text description often mentions a natural lan [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. The retrieval model for the multi-modal training [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The trend of explainability score vs. the number of highlighted tokens. Model: CodeBERT, Fold: valid [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: An example of the human labeling interface [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 37 canonical work pages

  1. [1]

    Update delays to nist vulnerability database alarms researchers,

    S. Hendery, “Update delays to nist vulnerability database alarms researchers,” 2024. [Online]. Available: https://www.scmagazine.com/ news/update-delays-to-nist-vulnerability-database-alarms-researchers

    work page 2024

  2. [2]

    ” why should i trust you?

    M. T. Ribeiro, S. Singh, and C. Guestrin, “” why should i trust you?” explaining the predictions of any classifier,” in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135–1144

    2016

  3. [3]

    Eraser: A benchmark to evaluate rationalized nlp models,

    J. DeYoung, S. Jain, N. F. Rajani, E. Lehman, C. Xiong, R. Socher, and B. C. Wallace, “Eraser: A benchmark to evaluate rationalized nlp models,” arXiv preprint arXiv:1911.03429 , 2019

    arXiv 1911

  4. [4]

    2017 equifax data breach,

    Wikipedia, “2017 equifax data breach,” 2024. [Online]. Available: https://en.wikipedia.org/wiki/2017 Equifax data breach

    work page 2017

  5. [5]

    National vulnerability database,

    “National vulnerability database,” 2023. [Online]. Available: https: //nvd.nist.gov/

    work page 2023

  6. [6]

    Death knell of the nvd?

    C. Hughes, “Death knell of the nvd?” 2024. [Online]. Available: https://www.resilientcyber.io/p/death-knell-of-the-nvd

    work page 2024

  7. [7]

    National vulnerability database: Opaque changes and unan- swered questions,

    E. Chin, “National vulnerability database: Opaque changes and unan- swered questions,” 2024. [Online]. Available: https://anchore.com/blog/ national-vulnerability-database-opaque-changes-and-unanswered-questions/

    work page 2024

  8. [8]

    The github advisory pull request for cve-2013-1814,

    GitHub, “The github advisory pull request for cve-2013-1814,”

    work page 2013

Show all 58 references
  1. [9]

    Github advisory database,

    ——, “Github advisory database,” 2024. [Online]. Available: https: //github.com/advisories

    2024

  2. [10]

    Where is it? tracing the vulnerability-relevant files from vulnerability reports,

    J. Sun, J. Chen, Z. Xing, Q. Lu, X. Xu, and L. Zhu, “Where is it? tracing the vulnerability-relevant files from vulnerability reports,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, 2024, pp. 1–13

    2024

  3. [11]

    Codebert: A pre-trained model for programming and natural languages,

    Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang et al., “Codebert: A pre-trained model for programming and natural languages,” arXiv preprint arXiv:2002.08155 , 2020

    2002 arXiv

  4. [12]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, vol. 364, 2019

    1907 arXiv

  5. [13]

    Deepxplore: Automated whitebox testing of deep learning systems,

    K. Pei, Y . Cao, J. Yang, and S. Jana, “Deepxplore: Automated whitebox testing of deep learning systems,” in proceedings of the 26th Symposium on Operating Systems Principles , 2017, pp. 1–18

    2017

  6. [14]

    Lemna: Explaining deep learning based security applications,

    W. Guo, D. Mu, J. Xu, P. Su, G. Wang, and X. Xing, “Lemna: Explaining deep learning based security applications,” in proceedings of the 2018 ACM SIGSAC conference on computer and communications security , 2018, pp. 364–379

    2018

  7. [15]

    Towards faithfully interpretable nlp sys- tems: How should we define and evaluate faithfulness?

    A. Jacovi and Y . Goldberg, “Towards faithfully interpretable nlp sys- tems: How should we define and evaluate faithfulness?” arXiv preprint arXiv:2004.03685, 2020

    2004 arXiv

  8. [16]

    All the reviewed cves under the maven ecosystem in github advisory,

    GitHub, “All the reviewed cves under the maven ecosystem in github advisory,” 2024. [Online]. Available: https://github.com/advisories? query=type%3Areviewed+ecosystem%3Amaven

    2024

  9. [17]

    Automated identification of libraries from vulnerability data,

    Y . Chen, A. E. Santosa, A. Sharma, and D. Lo, “Automated identification of libraries from vulnerability data,” in ACM/IEEE International Conference on Software Engineering: Software Engineering in Practice, 2020. [Online]. Available: https://dl.acm.org/doi/abs/10.1145/ 337781...

    2020

  10. [18]

    Automated identification of libraries from vulnerability data: Can we do better?

    S. A. Haryono, H. J. Kang, A. Sharma, A. Sharma, A. Santosa, A. M. Yi, and D. Lo, “Automated identification of libraries from vulnerability data: Can we do better?” in IEEE/ACM International Conference on Program Comprehension , 2022. [Online]. Available: https://dl.acm.org/do...

    2022

  11. [19]

    Few-sample named entity recognition for security vulnerability reports by fine-tuning pre-trained language models,

    G. Yang, S. Dineen, Z. Lin, and X. Liu, “Few-sample named entity recognition for security vulnerability reports by fine-tuning pre-trained language models,” in Deployable Machine Learning for Security Defense: Second International Workshop , 2021. [Online]. Available: https://...

    2021 doi

  12. [20]

    Snyk open source vulnerability database,

    Snyk, “Snyk open source vulnerability database,” 2024. [Online]. Available: https://security.snyk.io/

    2024

  13. [21]

    The nvd page for cve-2017-6056,

    NVD, “The nvd page for cve-2017-6056,” 2024. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE-2017-6056

    2017

  14. [22]

    The information page of cwe 79,

    MITRE, “The information page of cwe 79,” 2024. [Online]. Available: https://cwe.mitre.org/data/definitions/79.html

    2024

  15. [23]

    The nvd page for cve-2017-8085,

    NVD, “The nvd page for cve-2017-8085,” 2024. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE-2017-8085

    2017

  16. [24]

    Exs: Explainable search using local model ag- nostic interpretability,

    J. Singh and A. Anand, “Exs: Explainable search using local model ag- nostic interpretability,” in Proceedings of the twelfth ACM international conference on web search and data mining , 2019, pp. 770–773

    2019

  17. [25]

    Listwise explanations for ranking models using multiple explainers,

    L. Lyu and A. Anand, “Listwise explanations for ranking models using multiple explainers,” in European Conference on Information Retrieval . Springer, 2023, pp. 653–668

    2023

  18. [26]

    Explain- able information retrieval: A survey,

    A. Anand, L. Lyu, M. Idahl, Y . Wang, J. Wallat, and Z. Zhang, “Explain- able information retrieval: A survey,” arXiv preprint arXiv:2211.02405 , 2022

    2022 arXiv

  19. [27]

    A c/c++ code vulnerability dataset with code changes and cve summaries,

    J. Fan, Y . Li, S. Wang, and T. N. Nguyen, “A c/c++ code vulnerability dataset with code changes and cve summaries,” in Proceedings of the 17th International Conference on Mining Software Repositories, ser. MSR ’20. New York, NY , USA: Association for Computing Machinery, 2020...

    2020

  20. [28]

    Patchdb: A large-scale security patch dataset,

    X. Wang, S. Wang, P. Feng, K. Sun, and S. Jajodia, “Patchdb: A large-scale security patch dataset,” in 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) , 2021, pp. 149–160

    2021

  21. [29]

    Osv database,

    OSV , “Osv database,” 2024. [Online]. Available: https://osv.dev/

    2024

  22. [30]

    packaging: Core utilities for python packages,

    P. P. Authority, “packaging: Core utilities for python packages,” 2024. [Online]. Available: https://pypi.org/project/packaging/

    2024

  23. [31]

    difflib — helpers for computing deltas,

    P. S. Foundation, “difflib — helpers for computing deltas,” Python Standard Library, 2024. [Online]. Available: https://docs.python.org/3/ library/difflib.html

    2024

  24. [32]

    Interpretable machine learning for personalized medical recommendations: A lime-based ap- proach,

    Y . Wu, L. Zhang, U. A. Bhatti, and M. Huang, “Interpretable machine learning for personalized medical recommendations: A lime-based ap- proach,” Diagnostics, vol. 13, no. 16, p. 2681, 2023

    2023

  25. [33]

    Explaining local path plans using lime,

    A. Halilovic and F. Lindner, “Explaining local path plans using lime,” in International Conference on Robotics in Alpe-Adria Danube Region . Springer, 2022, pp. 106–113

    2022

  26. [34]

    Identifying vulnerable third-party libraries from textual descriptions of vulnerabilities and libraries,

    T. Chen, L. Li, B. Shan, G. Liang, D. Li, Q. Wang, and T. Xie, “Identifying vulnerable third-party libraries from textual descriptions of vulnerabilities and libraries,” arXiv preprint arXiv:2307.08206 , 2023

    2023 arXiv

  27. [35]

    Unixcoder: Unified cross-modal pre-training for code representation,

    D. Guo, S. Lu, N. Duan, Y . Wang, M. Zhou, and J. Yin, “Unixcoder: Unified cross-modal pre-training for code representation,” arXiv preprint arXiv:2203.03850, 2022

    2022 arXiv

  28. [36]

    Sentence-bert: Sentence embeddings using siamese bert- networks,

    N. Reimers, “Sentence-bert: Sentence embeddings using siamese bert- networks,” arXiv preprint arXiv:1908.10084 , 2019

    1908 arXiv

  29. [37]

    Dense passage retrieval for open-domain question answering,

    V . Karpukhin, B. O ˘guz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage retrieval for open-domain question answering,” arXiv preprint arXiv:2004.04906 , 2020

    2004 arXiv

  30. [38]

    On modality bias recognition and reduction,

    Y . Guo, L. Nie, H. Cheng, Z. Cheng, M. Kankanhalli, and A. Del Bimbo, “On modality bias recognition and reduction,” ACM Transactions on Multimedia Computing, Communications and Applications , vol. 19, no. 3, pp. 1–22, 2023

    2023

  31. [39]

    M2cvd: Multi- model collaboration for code vulnerability detection,

    Z. Wang, G. Li, J. Li, Y . Xiong, and Z. Jin, “M2cvd: Multi- model collaboration for code vulnerability detection,” arXiv preprint arXiv:2406.05940, 2024

    2024

  32. [40]

    Finding a needle in a haystack: Automated mining of silent vulnerability fixes,

    J. Zhou, M. Pacheco, Z. Wan, X. Xia, D. Lo, Y . Wang, and A. E. Hassan, “Finding a needle in a haystack: Automated mining of silent vulnerability fixes,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 2021, pp. 705–716

    2021

  33. [41]

    Large language model for vulnera- bility detection: Emerging results and future directions,

    X. Zhou, T. Zhang, and D. Lo, “Large language model for vulnera- bility detection: Emerging results and future directions,” arXiv preprint arXiv:2401.15468, 2024

    2024 arXiv

  34. [42]

    Identifying vulnerability patches by comprehending code commits with comprehensive change contexts,

    T. Chen, L. Li, T. Qian, Z. Wang, G. Liang, D. Li, Q. Wang, and T. Xie, “Identifying vulnerability patches by comprehending code commits with comprehensive change contexts,” arXiv preprint arXiv:2310.02530, 2023

    2023 arXiv

  35. [43]

    [Online]

    Wikipedia, “tf-idf,” 2024. [Online]. Available: https://en.wikipedia.org/ wiki/Tf%E2%80%93idf

    2024

  36. [44]

    Hatexplain: A benchmark dataset for explainable hate speech detection,

    B. Mathew, P. Saha, S. M. Yimam, C. Biemann, P. Goyal, and A. Mukherjee, “Hatexplain: A benchmark dataset for explainable hate speech detection,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 17, 2021, pp. 14 867–14 875

    2021

  37. [45]

    Google apps script

    Google Cloud Platform, “Google apps script.” [Online]. Available: https://developers.google.com/apps-script/guides/sheets

  38. [46]

    The mitre cve website,

    “The mitre cve website,” 2023. [Online]. Available: https://cve.mitre.org/

    2023

  39. [47]

    Vullibgen: Identifying vulnerable third-party libraries via generative pre-trained model,

    T. Chen, L. Li, L. Zhu, Z. Li, G. Liang, D. Li, Q. Wang, and T. Xie, “Vullibgen: Identifying vulnerable third-party libraries via generative pre-trained model,” arXiv preprint arXiv:2308.04662 , 2023

    2023 arXiv

  40. [48]

    A survey for in-context learning,

    Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui, “A survey for in-context learning,” arXiv preprint arXiv:2301.00234, 2022. [Online]. Available: https://arxiv.org/abs/2301. 00234

    2022 arXiv

  41. [49]

    Cleaning the nvd: Comprehensive quality assessment, improvements, and analyses,

    A. Anwar, A. Abusnaina, S. Chen, F. Li, and D. Mohaisen, “Cleaning the nvd: Comprehensive quality assessment, improvements, and analyses,” IEEE Transactions on Dependable and Secure Computing ,

  42. [50]

    OV ANA: An approach to analyze and improve the information quality of vulnerability databases,

    P. Kuehn, M. Bayer, M. Wendelborn, and C. Reuter, “OV ANA: An approach to analyze and improve the information quality of vulnerability databases,” in International Conference on Availability, Reliability and Security , 2021. [Online]. Available: https://dl.acm.org/ doi/abs/10....

    2021

  43. [51]

    Chronos: Time-aware zero-shot identification of libraries from vulnerability reports,

    Y . Lyu, T. Le-Cong, H. J. Kang, R. Widyasari, Z. Zhao, X.- B. D. Le, M. Li, and D. Lo, “Chronos: Time-aware zero-shot identification of libraries from vulnerability reports,” in International Conference on Software Engineering , 2023. [Online]. Available: https://doi.org/10.1...

    2023

  44. [52]

    A survey on explainable anomaly detection,

    Z. Li, Y . Zhu, and M. Van Leeuwen, “A survey on explainable anomaly detection,” ACM Transactions on Knowledge Discovery from Data , vol. 18, no. 1, pp. 1–54, 2023

    2023

  45. [53]

    Does explainable artificial intelligence improve human decision- making?

    Y . Alufaisan, L. R. Marusich, J. Z. Bakdash, Y . Zhou, and M. Kantar- cioglu, “Does explainable artificial intelligence improve human decision- making?” in Proceedings of the AAAI Conference on Artificial Intelli- gence, vol. 35, no. 8, 2021, pp. 6618–6626

    2021

  46. [54]

    General pitfalls of model-agnostic interpretation methods for machine learning models,

    C. Molnar, G. K ¨onig, J. Herbinger, T. Freiesleben, S. Dandl, C. A. Scholbeck, G. Casalicchio, M. Grosse-Wentrup, and B. Bischl, “General pitfalls of model-agnostic interpretation methods for machine learning models,” in International Workshop on Extending Explainable AI Beyo...

    2020

  47. [55]

    Dseg-lime–improving image explanation by hierarchical data-driven segmentation,

    P. Knab, S. Marton, and C. Bartelt, “Dseg-lime–improving image explanation by hierarchical data-driven segmentation,” arXiv preprint arXiv:2403.07733, 2024

    2024

  48. [56]

    Denas: automated rule generation by knowledge extraction from neural net- works,

    S. Chen, S. Bateni, S. Grandhi, X. Li, C. Liu, and W. Yang, “Denas: automated rule generation by knowledge extraction from neural net- works,” in Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software ...

    2020

  49. [2021]

    Available: https://ieeexplore.ieee.org/abstract/document/ 9601266

    [Online]. Available: https://ieeexplore.ieee.org/abstract/document/ 9601266

  50. [2024]

    Available: https://github.com/github/advisory-database/ pull/3210

    [Online]. Available: https://github.com/github/advisory-database/ pull/3210

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.