REVIEW 3 major objections 6 minor 90 references
Stylometric Defenses Against Author Impersonation in Software Repositories
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Patch-level writing style is enough to verify authorship in the open world and to rank forged commits near the top of a maintainer review queue without retraining.
desk verdict Patch-level stylometric authorship verification is real and transfers to two incidents, but the operational value is shown only against naive attackers and the test AUC may be partly topic signal. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a cross-modal stylometric embedding space learned by supervised contrastive fine-tuning: a pre-trained encoder (UniXcoder) is fine-tuned with the supervised contrastive loss so that commits by the same author cluster together and commits by different authors separate, with test-time authors held out entirely. Each commit is linearized as a cleaned commit message concatenated with a normalized unified diff (early fusion), using explicit structural markers for files and hunks and placeholders for literals. A second mechanism, the patch-size-gated late-fusion dispatcher, routes anomaly scores among text-only, code-only, and equal-blend channels according to diff length, with thresholds $T_{\text{lo}}=1{,}000$ and $T_{\text{hi}}=6{,}000$ characters, because short patches carry style mostly in the message while large payloads carry it in the code. This dispatcher is what converts the learned verification signal into ranked review queues.
What would settle it
Run the same detector against a mimicry-aware attack: have an attacker read the victim's recent commits and rewrite a malicious payload's message and diff to match the victim's typical phrasing, patch size, and code idioms, then check whether the forged commit still appears in the top 1% of the audit queue; if the median false-positive burden rises to the level of random ranking, the paper's triage claim is broken for that adversary class.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a commit, treated as a cross-modal artifact of a sanitized commit message plus a normalized unified diff, embeds into a space where authorship is verifiable in the open world and impersonation is detectable as an anomaly. Using supervised contrastive fine-tuning of UniXcoder on 398,505 Linux kernel commits from 4,474 authors, the early-fusion encoder reaches 0.9322 ROC AUC on author-disjoint held-out verification pairs, essentially tied with the text-only encoder at 0.9308. In streaming anomaly detection under synthetic author swaps, the detector reaches 0.939 AUPRC. Without retraining, the patch-size-gated dispatcher ranks both 2021 PHP forged commits within the top 1.03% of a 26,680-commit organization-wide queue and ranks the 28 scoreable ForceMemo/GlassWorm spoofs with a median of one false positive per spoof, or 0.8% of the per-repository scored queue. The paper reads these results as evidence that patch-level stylometry can serve as a calibrated triage signal that routes suspicious commits to human review rather than blocking them autonomously.
Load-bearing premise
The operational claim rests on attackers not deliberately imitating the victim's patch and message style: the synthetic swaps and the two real incidents all look stylometrically naive, and the paper explicitly says targeted mimicry, gradual account takeover, baseline poisoning, and patch-size manipulation are not modeled.
Editorial extensions
If this is right
- A CI/CD review queue can be ranked by stylistic deviation from the author's own recent history, letting analysts examine the few most anomalous commits first instead of reading everything.
- For short forged commits the commit message is the dominant stylometric channel, while for large payload-heavy spoofs the code diff dominates, so patch-size routing is the appropriate fusion principle.
- The kernel-trained detector transfers to the PHP and ForceMemo corpora without retraining, ranking both PHP forgeries in the top 1.03% of the audit queue and typical ForceMemo spoofs after one benign commit.
- The layer protects authors with stable histories best; accounts with sparse or bulk-heavy histories receive weaker protection, arguing for complementary content-level screening of those accounts.
- The detector is framed as a triage signal that produces ranked queues for human review, not as an autonomous block decision.
- A maintainer-scoped queue under small-commit injection shows a median of 50 false positives before the injected commit among roughly 60,571 commits, a review burden that is low enough to be operationally plausible.
Reading between the lines
- An attack surface the paper does not test is deliberate stylistic mimicry; a natural experiment is to fine-tune a language model on a victim's commit messages and patch habits and re-measure whether the forged commit still lands in the top 1% of the queue.
- The near-tie between text-only and cross-modal verification suggests a cheaper deployment path: message-only scoring for short commits with code-channel gating for large diffs could preserve most of the triage value at lower compute cost.
- Because AI-assisted commits are not filtered from the corpus, the embedding space may become less author-specific over time; testing whether different language models leave distinguishable style signatures would bound that drift.
- The residual ForceMemo cases imply a pre-attack prioritization step: accounts whose histories are bulk-heavy or unstable could be flagged for content-based scanning in advance, a complement the paper mentions but does not implement.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a patch-level stylometric authorship-verification pipeline for defending against account-compromise attacks in software repositories. The authors fine-tune UniXcoder with supervised contrastive learning on 398,505 Linux kernel commits, jointly embedding commit messages and unified diffs, and report open-world verification ROC AUC of 0.932 on held-out authors. They then adapt the embeddings to a streaming anomaly detector with a patch-size-gated late-fusion router, and evaluate it on synthetic author swaps and on two retrospective incidents: the 2021 PHP backdoor (two forged commits ranked within the top 1.03% of a 26,680-commit queue) and the 2026 ForceMemo campaign (median 0.8% review burden across 28 scoreable spoofs). The paper concludes that such embeddings can serve as a behavioral triage layer in CI/CD, while acknowledging limitations including the naive-impersonation threat model and imperfect topic controls.
Significance. If the central claims hold, this is a noteworthy contribution: it is the first author-disjoint, open-world patch-level verification benchmark at Linux-kernel scale, and the incident checks provide a realistic transfer test that goes beyond synthetic benchmarks. The paper's strengths include strict author-disjoint splits, transparent reporting of review-burden metrics rather than only AUC, an explicit modality analysis showing text dominance, and candid admission of the naive-adversary scope in Section 6. The validation of the frozen pipeline on two real incidents, even with small positive counts, is a valuable data point for the security community. However, the significance depends on resolving the domain-matching asymmetry in the verification evaluation and on clarifying the scope of the operational claims against deliberate mimicry.
major comments (3)
- [§4.1 / §3.6.3 (Table 2)] The verification protocol samples ten same-author and ten different-author pairs per held-out author without any control for the subsystem or topic of the commits, whereas the training procedure in Section 3.3 deliberately restricts each mini-batch to a single coarse domain to prevent the model from learning topic boundaries as a shortcut. This asymmetry means the reported cross-modal AUC of 0.9322 may be inflated: negative pairs drawn from different subsystems are easier to separate by topic than by style, and the model was never forced to solve the same-domain version of the task during training. I request an evaluation with domain-matched negative pairs (e.g., paired within the same coarse subsystem) or, at minimum, an AUC stratified by same-subsystem versus cross-subsystem negative pairs. This is necessary to support the paper's RQ1 claim of patch-level stylometric signal rather than topic signal.
- [Abstract / §6] The abstract and conclusions state that the pipeline 'can support behavioral triage against author impersonation in real-world repositories,' but the empirical support is limited to naive impersonation. Section 6 explicitly says the synthetic swaps 'model naïve impersonation' and do not model targeted mimicry, and both incident datasets involve generic attacker-written messages and payloads that are stylistically distant from the victims. Because the text-only channel is the strongest signal (0.9308 verification AUC, 0.946 detection AUPRC, Table 3) and commit messages are public, a mimicry-aware adversary who copies the victim's message templates, lengths, and vocabulary would likely move much closer to the victim in the embedding space. The paper should either add a targeted-mimicry evaluation (e.g., style-transfer or template-copy attacks against the detector) or explicitly restrict all headline claims to the naive-impersonation setting. As written, the gap between the claims and the threat model is load-bearing for the operational contribution.
- [§3.6.4 / Appendix A.8] The ForceMemo transfer check does not use the exact frozen configuration declared in Section 3.6.4: the window size is changed from W=20 to W=5 (a coverage-oriented exception), and six of the 28 scoreable spoofs have their baselines augmented with commits from other repositories (path-1 augmentation). The paper is transparent about both choices and provides a confounding analysis, but as a result the ForceMemo ranking is not a clean 'without retraining' transfer test of the kernel-trained pipeline. Please report the same evaluation with W=20 on the 19 spoofs that remain scoreable, or explicitly present the W=5 result as a coverage-adapted variant and soften the 'without retraining' claim accordingly.
minor comments (6)
- [§4.2] Section 4.2 refers to 'Section 4.2.0.2' in two places; this appears to be a numbering error from the appendix restructuring and should point to the patch-size analysis subsection (likely 4.2.2).
- [Abstract] The abstract's 'within the top 1.03%' is the FP@all share (274/26,680); for the first spoof the share is 248/26,680 = 0.93%. Please state both values consistently.
- [§3.5] Reporting AUPRC for the balanced verification pairs is unusual because precision-recall is most informative under imbalance; the paper should either omit it or note that it is included for completeness.
- [§4.2, Table 4] Table 4's median FP/pos for the dispatcher (50) has a wide bootstrap interval [28, 225] that overlaps the text-only (81) and equal-mix (65) intervals; the text appropriately acknowledges this, but the main-text phrasing 'lowest worst-case burden' should not obscure that the dispatcher's advantage is not statistically significant on the primary metric in the synthetic simulation.
- [Code availability] The code availability statement says the source 'will be made publicly available upon publication'; for a reproducibility-focused claim, consider providing a repository URL in the preprint.
- [Throughout] The spelling of 'naïve' and 'naive' is inconsistent; please choose one form and use it consistently.
Circularity Check
No significant circularity: kernel-trained embeddings and validation-calibrated thresholds are applied to held-out and incident data without retraining; the sole self-citation is non-load-bearing background.
full rationale
The derivation chain is self-contained. The UniXcoder encoder is fine-tuned on Linux kernel commits under author-disjoint splits (Section 3.3, Table 1), and verification pairs plus synthetic author swaps are drawn from the held-out test partition, so the reported 0.9322 ROC AUC and 0.939 AUPRC are genuine open-world measurements rather than in-sample fits. The routing thresholds Tlo=1,000 and Thi=6,000 in Eq. (1) are explicitly calibrated on a separate validation split and then frozen for the kernel test and both incident corpora (Section 3.6.4), so the PHP top-1.03% and ForceMemo median-0.8% rankings are transfer checks, not predictions fitted to incident labels. Incident labels were never used to train the encoder or set the dispatcher. The paper's only self-citation, Tsfaty and Fire [18], appears in Section 2.1 as related work on MSDT and is not used in the methodology, equations, or evaluation, so it is not load-bearing. The Section 6 limitation that synthetic swaps model only naive impersonation and do not cover targeted mimicry is an honest robustness caveat about adversary modeling, not a circular reduction. Protocol adjustments for ForceMemo (W=5, path-1 augmentation) are disclosed and analyzed in Appendix A.8 as coverage choices, and they do not use spoof labels to fit the ranking. No equation or named result in the paper is equivalent to its own inputs by construction.
Assumptions & free parameters
free parameters (6)
- Tlo (patch-size routing lower threshold) =
1000 characters
- Thi (patch-size routing upper threshold) =
6000 characters
- Windows size W =
20 for kernel and PHP; 5 for ForceMemo
- Tpriors (percentile vs raw distance threshold) =
100 prior commits
- Minimum prior commits for scoreability =
50 (PHP), 5 (ForceMemo)
- Unit-based sampling u =
2
assumptions (5)
- domain assumption Version-control author labels (name/email) are a valid proxy for true authorship of a patch.
- standard math Cosine similarity in the fine-tuned embedding space is an effective same-author score.
- domain assumption Authorship style learned from Linux kernel commits transfers to other languages and ecosystems.
- domain assumption For benign commits, the commit message and the code diff are composed by the same person.
- domain assumption The 10% synthetic swap rate yields a stable, meaningful anomaly-detection benchmark.
Cite this review
Pith. "Pith review of Stylometric Defenses Against Author Impersonation in Software Repositories." pith.science (2026). https://pith.science/paper/GSOANXZG
@misc{pith2026260802695,
author = {Pith},
title = {Pith review of: Stylometric Defenses Against Author Impersonation in Software Repositories},
year = {2026},
howpublished = {\url{https://pith.science/paper/GSOANXZG}},
note = {Machine review of arXiv:2608.02695}
}
read the original abstract
Software supply-chain attacks increasingly exploit an identity gap where compromised maintainer accounts authorize malicious changes. This work evaluates patch-level authorship verification as a behavioral defense layer, showing that stylometric analysis can operate not only on full source files but also on patch-level commits. We fine-tune a cross-modal transformer on more than 20 years of Linux kernel commit history to embed code diffs and commit messages into a unified stylometric space, achieving ROC AUC of 0.93 for open-world authorship verification. We then use these representations in a streaming anomaly detector suited to continuous integration and deployment (CI/CD) settings. We validate the pipeline on two retrospective supply-chain incidents involving different patch characteristics: the 2021 PHP backdoor and the 2026 ForceMemo/GlassWorm campaign. Without retraining, the proposed detector surfaces both PHP forged commits within approximately 1% of the maintainer audit queue and ranks the 28 scoreable ForceMemo spoofs with a median per-repository review burden of 0.8%. These results indicate that cross-modal patch-level embeddings can support behavioral triage against author impersonation in real-world repositories.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Souppaya, Murugiah, Scarfone, Karen, and Dodson, Donna. Secure Software Development Framework (SSDF) version 1.1 : recommendations for mitigating the risk of software vulnerabilities. Technical Report NIST SP 800-218, National Institute of Standards and Technology (U.S.), Gaithersburg, MD, February 2022. URL https://nvlpubs.nist.gov/nistpub s/SpecialPubli...
work page 2022
-
[2]
Devphish: Exploring so- cial engineering in software supply chain attacks on de- velopers
Siadati, Hossein, Jafarikhah, Sima, Sahin, Elif, Hernan- dez, Terrence Brent, Tripp, Elijah Lorenzo, Khryashchev, Denis, and Kharraz, Amin. Devphish: Exploring so- cial engineering in software supply chain attacks on de- velopers. In2024 IEEE 15th Annual Ubiquitous Com- puting, Electronics and Mobile Communication Confer- ence (UEMCON), pages 517–523. IEE...
arXiv 2024
-
[3]
Williams, Laurie, Benedetti, Giacomo, Hamer, Sivana, Paramitha, Ranindya, Rahman, Imranur, Tamanna, Mahz- abin, Tystahl, Greg, Zahan, Nusrat, Morrison, Patrick, Acar, Yasemin, Cukier, Michel, K”astner, Christian, Kapravelos, Alexandros, Wermke, Dominik, and Enck, William. Research directions in software supply chain security.ACM Transactions on Software E...
doi:10.1145/3714464 2025
-
[4]
Update on git.php.net incident, April 2021
Popov, Nikita. Update on git.php.net incident, April 2021. URL https://news-web.php.net/php.int ernals/113981. php.internals mailing list, accessed 2026-05-12
work page 2021
-
[5]
git.php.net server compromised, move to GitHub, and delayed updates, March 2021
PHP.Watch. git.php.net server compromised, move to GitHub, and delayed updates, March 2021. URLhttps: //php.watch/news/2021/03/git-php-net -hack. Early incident report. Later mechanism details were revised in Popov’s official incident update. Accessed 2026-05-12
work page 2021
-
[6]
Re- ported supply chain compromise affecting xz utils data compression library, cve-2024-3094, 2024
Cybersecurity and Agency, Infrastructure Security. Re- ported supply chain compromise affecting xz utils data compression library, cve-2024-3094, 2024. URLhttps: //www.cisa.gov/news-events/alerts/202 4/03/29/reported-supply-chain-comprom ise-affecting-xz-utils-data-compressi on-library-cve-2024-3094
work page 2024
-
[7]
Sharma, Varun. ForceMemo: Hundreds of GitHub Python Repos Compromised via Account Takeover and Force-Push, March 2026. URL https://www.st epsecurity.io/blog/forcememo-hundred s-of-github-python-repos-compromised -via-account-takeover-and-force-push . Accessed 2026-05-15
work page 2026
-
[8]
De-anonymizing programmers via code stylometry
Caliskan-Islam, Aylin, Harang, Richard E., Liu, Andrew, Narayanan, Arvind, V oss, Clare R., Yamaguchi, Fabian, and Greenstadt, Rachel. De-anonymizing programmers via code stylometry. In24th USENIX Security Sympo- sium, pages 255–270. USENIX Association, 2015
work page 2015
Show all 90 references
-
[9]
Git blame who?: Stylistic author- ship attribution of small, incomplete source code frag- ments.Proceedings on Privacy Enhancing Technologies, 2019(3):389–408, 2019
Dauber, Edwin, Caliskan, Aylin, Harang, Richard, Shearer, Gregory, Weisman, Michael, Nelson, Frederica, and Greenstadt, Rachel. Git blame who?: Stylistic author- ship attribution of small, incomplete source code frag- ments.Proceedings on Privacy Enhancing Technologies, 2019(3...
2019 doi
-
[10]
Authorship attribution of source code: a language-agnostic approach and applicability in software engineering
Bogomolov, Egor, Kovalenko, Vladimir, Rebryk, Yurii, Bacchelli, Alberto, and Bryksin, Timofey. Authorship attribution of source code: a language-agnostic approach and applicability in software engineering. InESEC/FSE ’21: 29th ACM Joint European Software Engineering Conference...
2021
-
[11]
Unixcoder: Unified cross-modal pre-training for code representation
Guo, Daya, Lu, Shuai, Duan, Nan, Wang, Yanlin, Zhou, Ming, and Yin, Jian. Unixcoder: Unified cross-modal pre-training for code representation. InProceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (ACL), pages 7212–7225, 2022. doi: 10.1865...
2022 doi
-
[12]
SoK: Taxonomy of attacks on open- source software supply chains
Ladisa, Piergiorgio, Plate, Henrik, Martinez, Matias, and Barais, Olivier. SoK: Taxonomy of attacks on open- source software supply chains. In2023 IEEE Symposium on Security and Privacy (SP), pages 1509–1526. IEEE,
-
[13]
Backstabber’s knife collection: A review of Stylometric Defenses Against Author Impersonation in Software Repositories — 16/24 open source software supply chain attacks
Ohm, Marc, Plate, Henrik, Sykosch, Arnold, and Meier, Michael. Backstabber’s knife collection: A review of Stylometric Defenses Against Author Impersonation in Software Repositories — 16/24 open source software supply chain attacks. InDetec- tion of Intrusions and Malware, and...
2020 doi
-
[14]
A comparison of code–similarity analysers.Em- pirical Software Engineering, 23(4):2464–2519, 2018
Ragkhitwetsagul, Chaiyong, Krinke, Jens, and Clark, David. A comparison of code–similarity analysers.Em- pirical Software Engineering, 23(4):2464–2519, 2018. doi: 10.1007/s10664-017-9564-7. URL https: //doi.org/10.1007/s10664-017-9564-7
2018 doi
-
[15]
Detecting suspicious package updates
Garrett, Kalil, Ferreira, Gabriel, Jia, Limin, Sunshine, Joshua, and K”astner, Christian. Detecting suspicious package updates. In2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE–NIER), pages 13–16, 2019. doi: 10.1109/IC...
2019
-
[16]
Scalable source code similarity detection in large code repositories.EAI Endorsed Transactions on Scalable Information Systems, 6(22), 2019
Alomari, Firas and Harbi, Muhammed. Scalable source code similarity detection in large code repositories.EAI Endorsed Transactions on Scalable Information Systems, 6(22), 2019. doi: 10.4108/eai.13-7-2018.159353. URL https://doi.org/10.4108/eai.13-7-201 8.159353
2019
-
[17]
Evaluating automatically generated yara rules and enhancing their effectiveness
Naik, Nitin, Jenkins, Paul, Cooke, Roger, Gillett, Jonathan, and Jin, Yaochu. Evaluating automatically generated yara rules and enhancing their effectiveness. In2020 IEEE Symposium Series on Computational In- telligence (SSCI), pages 1146–1153. IEEE, 2020. doi: 10.1109/SSCI478...
2020
-
[18]
Malicious source code de- tection using a translation model.Patterns, 4(7):100773,
Tsfaty, Chen and Fire, Michael. Malicious source code de- tection using a translation model.Patterns, 4(7):100773,
-
[19]
SoK: A defense-oriented evaluation of software supply chain security
Ishgair, Eman Abu, Melara, Marcela S., and Torres-Arias, Santiago. SoK: A defense-oriented evaluation of software supply chain security. arXiv preprint arXiv:2405.14993,
-
[20]
URL https://doi.org/10.1016/j.patter.2023
doi: 10.1016/j.patter.2023.100773. URL https://doi.org/10.1016/j.patter.2023. 100773
2023
-
[21]
Towards robust detection of open source software supply chain poisoning attacks in in- dustry environments
Zheng, Xinyi, Wei, Chen, Wang, Shenao, Zhao, Yan- jie, Gao, Peiming, Zhang, Yuanchao, Wang, Kailong, and Wang, Haoyu. Towards robust detection of open source software supply chain poisoning attacks in in- dustry environments. In39th IEEE/ACM International Conference on Automat...
1990
-
[22]
Zhang, Junan, Huang, Kaifeng, Huang, Yiheng, Chen, Bihuan, Wang, Ruisi, Wang, Chong, and Peng, Xin. Killing two birds with one stone: Malicious package detection in NPM and PyPI using a single model of mali- cious behavior sequence.ACM Transactions on Software Engineering and ...
2025 doi
-
[23]
A survey of mal- ware detection techniques
Idika, Nwokedi and Mathur, Aditya P. A survey of mal- ware detection techniques. Technical report, Department of Computer Science, Purdue University, West Lafayette, IN, USA, 2007. URL https://www.cerias.pur due.edu/apps/reports_and_papers/view /4328/
2007
-
[24]
Attributing open-source contributions is critical but difficult: A systematic analysis of github prac- tices and their impact on software supply chain security
Holtgrave, Jan-Ulrich, Friedrich, Kay, Fischer, Fabian, Huaman, Nicolas, Busch, Niklas, Klemmer, Jan H., Fourn´e, Marcel, Wiese, Oliver, Wermke, Dominik, and Fahl, Sascha. Attributing open-source contributions is critical but difficult: A systematic analysis of github prac- ti...
2025
-
[25]
Integra- tion of static and dynamic code stylometry analysis for programmer de-anonymization
Wang, Ningfei, Ji, Shouling, and Wang, Ting. Integra- tion of static and dynamic code stylometry analysis for programmer de-anonymization. InProceedings of the 11th ACM Workshop on Artificial Intelligence and Secu- rity (AISec), pages 74–84, New York, NY , USA, 2018. ACM. doi:...
2018
-
[26]
Evading machine learning malware classifiers
Fleshman, William. Evading machine learning malware classifiers. Blog post on Towards Data Science (Medium),
-
[27]
Authorship attribution with topic models.Computational Stylometric Defenses Against Author Impersonation in Software Repositories — 17/24 Linguistics, 40(2):269–310, 2014
Seroussi, Yanir, Zukerman, Ingrid, and Bohnert, Fabian. Authorship attribution with topic models.Computational Stylometric Defenses Against Author Impersonation in Software Repositories — 17/24 Linguistics, 40(2):269–310, 2014. doi: 10.1162/coli\ a \ 00173
2014 doi
-
[28]
Dataset char- acteristics for reliable code authorship attribution.IEEE Transactions on Dependable and Secure Computing, 20 (1):506–521, 2023
Abazari, Farzaneh, Branca, Enrico, Ridley, Norah, Stakhanova, Natalia, and Preda, Mila Dalla. Dataset char- acteristics for reliable code authorship attribution.IEEE Transactions on Dependable and Secure Computing, 20 (1):506–521, 2023. doi: 10.1109/TDSC.2021.3138700
2023
-
[29]
Mis- leading authorship attribution of source code using ad- versarial learning
Quiring, Erwin, Maier, Alwin, and Rieck, Konrad. Mis- leading authorship attribution of source code using ad- versarial learning. In28th USENIX Security Sympo- sium (USENIX Security 19), pages 479–496, Santa Clara, CA, USA, 2019. USENIX Association. URL https: //www.usenix.org...
2019
-
[30]
Overview of the authorship verification task at pan 2023
Stamatatos, Efstathios, Kredens, Krzysztof, Pezik, Piotr, Heini, Annina, Bevendorff, Janek, Stein, Benno, and Potthast, Martin. Overview of the authorship verification task at pan 2023. InWorking Notes of the Conference and Labs of the Evaluation Forum (CLEF 2023), volume 3497...
2023
-
[31]
URL https://ceur-ws.org/Vol-3497/ paper-199.pdf
-
[32]
Code authorship identification using convolutional neural net- works.Future Generation Computer Systems, 95:104– 115, 2019
Abuhamad, Mohammed, su Rhim, Ji, AbuHmed, Tamer, Ullah, Sana, Kang, Sanggil, and Nyang, DaeHun. Code authorship identification using convolutional neural net- works.Future Generation Computer Systems, 95:104– 115, 2019. doi: 10.1016/j.future.2018.12.038
2019 doi
-
[33]
Deep metric learning for code authorship attribution and verification
White, Riley and Sprague, Nathan. Deep metric learning for code authorship attribution and verification. In2021 20th IEEE International Conference on Machine Learn- ing and Applications (ICMLA), pages 1089–1093, 2021. doi: 10.1109/ICMLA52953.2021.00178
2021
-
[34]
H., Tian, Yuan, and Song, Leo
Ou, Weihan, Ding, Steven H. H., Tian, Yuan, and Song, Leo. Scs-gan: Learning functionality-agnostic stylomet- ric representations for source code authorship verification. IEEE Transactions on Software Engineering, 49(4):1426– 1442, 2023. doi: 10.1109/TSE.2022.3177228
2023
-
[35]
SHIELD: Thwarting code authorship attribution.IEEE Transactions on Depend- able and Secure Computing, 22(5):4753–4767, 2025
Abuhamad, Mohammed, Jung, Changhun, Mohaisen, David, and Nyang, DaeHun. SHIELD: Thwarting code authorship attribution.IEEE Transactions on Depend- able and Secure Computing, 22(5):4753–4767, 2025. doi: 10.1109/TDSC.2025.3553753. URL https: //doi.org/10.1109/TDSC.2025.3553753
2025
-
[36]
Authorship attribution of source code by using back propagation neural network based on particle swarm optimization.PLOS ONE, 12(11):e0187204, November
Yang, Xinyu, Xu, Guoai, Li, Qi, Guo, Yanhui, and Zhang, Miao. Authorship attribution of source code by using back propagation neural network based on particle swarm optimization.PLOS ONE, 12(11):e0187204, November
-
[37]
Codebert: A pre-trained model for programming and natural lan- guages
Feng, Zhangyin, Guo, Daya, Tang, Duyu, Duan, Nan, Feng, Xiaocheng, Gong, Ming, Shou, Linjun, Qin, Bing, Liu, Ting, Jiang, Daxin, and Zhou, Ming. Codebert: A pre-trained model for programming and natural lan- guages. InFindings of the Association for Computa- tional Linguistics...
2020 doi
-
[38]
Graphcodebert: Pre- training code representations with data flow
Guo, Daya, Ren, Shuo, Lu, Shuai, Feng, Zhangyin, Tang, Duyu, Liu, Shujie, Zhou, Long, Duan, Nan, Svyatkovskiy, Alexey, Fu, Shengyu, Tufano, Michele, Deng, Shao Kun, Clement, Colin, Drain, Dawn, Sundaresan, Neel, Yin, Jian, Jiang, Daxin, and Zhou, Ming. Graphcodebert: Pre- trai...
-
[39]
Wang, Yue, Wang, Weishi, Joty, Shafiq, and Hoi, Steven C. H. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and gen- eration. InProceedings of the 2021 Conference on Em- pirical Methods in Natural Language Processing, pages 8696–8708...
2021
-
[40]
CommitBERT: Commit message gener- ation using pre-trained programming language model
Jung, Tae Hwan. CommitBERT: Commit message gener- ation using pre-trained programming language model. In Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021), pages 26–33. Association for Computational Linguistics, 2021. doi: 10.18653/...
2021 doi
-
[41]
Large-scale and robust code authorship identification with deep feature learning.ACM Transactions on Privacy and Security, 24(4):23:1–23:35,
Abuhamad, Mohammed, AbuHmed, Tamer, Mohaisen, David, and Nyang, DaeHun. Large-scale and robust code authorship identification with deep feature learning.ACM Transactions on Privacy and Security, 24(4):23:1–23:35,
-
[42]
Code representation learning at scale
Zhang, Dejiao, Ahmad, Wasi Uddin, Tan, Ming, Ding, Hantian, Nallapati, Ramesh, Roth, Dan, Ma, Xiaofei, and Xiang, Bing. Code representation learning at scale. InThe Twelfth International Conference on Learning Representations (ICLR), 2024. URL https://open review.net/forum?id=...
2024
-
[43]
Ropgen: Towards robust code authorship attribution via automatic coding style transfor- mation
Li, Zhen, Chen, Guenevere (Qian), Chen, Chen, Zou, Yayi, and Xu, Shouhuai. Ropgen: Towards robust code authorship attribution via automatic coding style transfor- mation. InProceedings of the 44th International Confer- ence on Software Engineering (ICSE), pages 1906–1918,
1906
-
[44]
Clave: A deep learning model for source code authorship verifica- tion with contrastive learning and transformer encoders
´Alvarez-Fidalgo, David and Ort´ın, Francisco. Clave: A deep learning model for source code authorship verifica- tion with contrastive learning and transformer encoders. Information Processing & Management, 62(3):104005,
-
[45]
Zero-shot source code author identifica- tion: A lexicon and layout independent approach
Hozhabrierdi, Pegah, Hitos, Dunai Fuentes, and Mohan, Chilukuri K. Zero-shot source code author identifica- tion: A lexicon and layout independent approach. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2020. doi: 10.1109/IJCNN48605.2 020.9207647
2020 doi
-
[46]
Determining if two documents are written by the same author.Journal of the Association for Information Science and Technology, 65(1):178–187, 2014
Koppel, Moshe and Winter, Yaron. Determining if two documents are written by the same author.Journal of the Association for Information Science and Technology, 65(1):178–187, 2014. doi: 10.1002/asi.22954. URL https://doi.org/10.1002/asi.22954
2014 doi
-
[47]
URL https://openreview.net/forum ?id=jLoC4ez43PZ
-
[48]
Pydriller: Python framework for mining software repositories
Spadini, Davide, Aniche, Maur ´ıcio, and Bacchelli, Al- berto. Pydriller: Python framework for mining software repositories. InProceedings of the 2018 ACM Joint Meet- ing on European Software Engineering Conference and Symposium on the Foundations of Software Engineer- ing, ES...
2018
-
[49]
URL https://aclanthology.org/2021.emnlp-m ain.685/
doi: 10.18653/v1/2021.emnlp-main.685. URL https://aclanthology.org/2021.emnlp-m ain.685/
2021 doi
-
[50]
Mining email social networks
Bird, Christian, Gourley, Alex, Devanbu, Prem, Gertz, Michael, and Swaminathan, Anand. Mining email social networks. InProceedings of the 2006 International Work- shop on Mining Software Repositories, MSR ’06, pages 137–143. ACM, 2006. doi: 10.1145/1137983.1138016. URL https:/...
2006
-
[51]
Commitbart: A large pre-trained model for github com- mits.arXiv preprint arXiv:2208.08100, 2022
Liu, Shangqing, Li, Yanzhou, Xie, Xiaofei, and Liu, Yang. Commitbart: A large pre-trained model for github com- mits.arXiv preprint arXiv:2208.08100, 2022. URL https://arxiv.org/abs/2208.08100
2022 arXiv
-
[52]
Cc2vec: distributed representations of code changes
Hoang, Thong, Kang, Hong Jin, Lo, David, and Lawall, Julia. Cc2vec: distributed representations of code changes. InProceedings of the ACM/IEEE 42nd Inter- national Conference on Software Engineering, ICSE ’20, pages 518–529, New York, NY , USA, 2020. Associa- tion for Computin...
2020
-
[53]
Ocean: Open-world Stylometric Defenses Against Author Impersonation in Software Repositories — 18/24 contrastive authorship identification
M”achtle, Felix, Serr, Jan-Niclas, Loose, Nils, Sander, Jonas, and Eisenbarth, Thomas. Ocean: Open-world Stylometric Defenses Against Author Impersonation in Software Repositories — 18/24 contrastive authorship identification. arXiv preprint arXiv:2412.05049, 2024. URL https:/...
2024 arXiv
-
[54]
Robust and accurate authorship attribution via program normaliza- tion.arXiv preprint arXiv:2007.00772, 2020
Wang, Yizhen, Alhanahnah, Mohannad, Wang, Ke, Christodorescu, Mihai, and Jha, Somesh. Robust and accurate authorship attribution via program normaliza- tion.arXiv preprint arXiv:2007.00772, 2020. URL https://arxiv.org/abs/2007.00772
2007 arXiv
-
[55]
Supervised contrastive learning
Khosla, Prannay, Teterwak, Piotr, Wang, Chen, Sarna, Aaron, Tian, Yonglong, Isola, Phillip, Maschinot, Aaron, Liu, Ce, and Krishnan, Dilip. Supervised contrastive learning. InProceedings of the 34th International Con- ference on Neural Information Processing Systems, NIPS ’20,...
2020
-
[56]
Supervised contrastive learning for multi-author writing style analysis
Ye, Zhanhong, Zhong, Changle, Qi, Haoliang, and Han, Yong. Supervised contrastive learning for multi-author writing style analysis. InWorking Notes of the Conference and Labs of the Evaluation Forum (CLEF 2023), volume 3497 ofCEUR Workshop Proceedings, 2023. URL ht tps://ceur-...
2023
-
[57]
Altakrori, Malik, Cheung, Jackie Chi Kit, and Fung, Ben- jamin C. M. The topic confusion task: A novel evaluation scenario for authorship attribution. InFindings of the As- sociation for Computational Linguistics: EMNLP 2021, pages 4242–4256. Association for Computational Lin-...
2021 doi
-
[58]
A transformer-based approach to highly granular source code authorship attribution.Ap- plied and Computational Engineering, 67:197–202, 2024
Shi, Chongzheng. A transformer-based approach to highly granular source code authorship attribution.Ap- plied and Computational Engineering, 67:197–202, 2024. doi: 10.54254/2755- 2721/67/20240686. URL https://doi.org/10.54254/2755- 2721/ 67/20240686
2024 doi
-
[59]
Multimodal machine learning: A survey and taxonomy.IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2):423–443, 2019
Baltruˇsaitis, Tadas, Ahuja, Chaitanya, and Morency, Louis-Philippe. Multimodal machine learning: A survey and taxonomy.IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2):423–443, 2019. doi: 10.1109/TPAMI.2018.2798607
2019
-
[60]
Duckdb: An embeddable analytical database
Raasveldt, Mark and M¨uhleisen, Hannes. Duckdb: An embeddable analytical database. InProceedings of the 2019 International Conference on Management of Data, SIGMOD ’19, pages 1981–1984, Amsterdam, Nether- lands, 2019. ACM. doi: 10.1145/3299869.3320212
2019
-
[61]
A survey on anomaly detection in evolving data.ACM SIGKDD Explorations Newsletter, 20(1):13–23, 2018
Salehi, Mahsa and Rashidi, Lida. A survey on anomaly detection in evolving data.ACM SIGKDD Explorations Newsletter, 20(1):13–23, 2018. doi: 10.1145/3229329.32 29332. URL https://doi.org/10.1145/3229 329.3229332
2018 doi
-
[62]
Kouters, Erik, Vasilescu, Bogdan, Serebrenik, Alexander, and van den Brand, Mark G. J. Who’s who in GNOME: Using LSA to merge software repository identities. In 2012 28th IEEE International Conference on Software Maintenance (ICSM), pages 592–595. IEEE, 2012. doi: 10.1109/ICSM...
2012
-
[63]
Index for rating diagnostic tests
Youden, William J. Index for rating diagnostic tests. Cancer, 3(1):32–35, 1950. doi: 10.1002/1097-0142(19 50)3:1⟨32::AID-CNCR2820030106⟩3.0.CO;2-3
1950 doi
-
[64]
Patchnet: A tool for deep patch classification
Hoang, Thong, Lawall, Julia L., Oentaryo, Richard J., Tian, Yuan, and Lo, David. Patchnet: A tool for deep patch classification. InProceedings of the 41st Inter- national Conference on Software Engineering: Com- panion Proceedings, pages 83–86. IEEE/ACM, 2019. doi: 10.1109/ICS...
2019
-
[65]
Chapman and Hall/CRC, 1993
Efron, Bradley and Tibshirani, Robert.An Introduction to the Bootstrap. Chapman and Hall/CRC, 1993. ISBN 978-0412042317. doi: 10.1201/9780429246593
1993 doi
-
[66]
The relationship be- tween precision-recall and ROC curves
Davis, Jesse and Goadrich, Mark. The relationship be- tween precision-recall and ROC curves. InProceed- ings of the 23rd International Conference on Machine Learning, pages 233–240, Pittsburgh, PA, 2006. doi: 10.1145/1143844.1143874
2006
-
[67]
Outside the closed world: On using machine learning for network intrusion detection
Sommer, Robin and Paxson, Vern. Outside the closed world: On using machine learning for network intrusion detection. In2010 IEEE Symposium on Security and Privacy, pages 305–316. IEEE, 2010. doi: 10.1109/SP.2 010.25
2010 doi
-
[68]
99% false positives: A qualitative study of SOC analysts’ perspectives on security alarms
Alahmadi, Bushra A., Axon, Louise, and Martinovic, Ivan. 99% false positives: A qualitative study of SOC analysts’ perspectives on security alarms. In31st USENIX Security Symposium (USENIX Security 22), pages 2783– 2800, Boston, MA, August 2022. USENIX Association. URL https:/...
2022
-
[69]
Anwar, El Saddik, Abdul- motaleb, and Kankanhalli, Mohan S
Atrey, Pradeep K., Hossain, M. Anwar, El Saddik, Abdul- motaleb, and Kankanhalli, Mohan S. Multimodal fusion for multimedia analysis: a survey.Multimedia Systems, 16(6):345–379, 2010. doi: 10.1007/s00530-010-0182-0. Stylometric Defenses Against Author Impersonation in Software...
2010 doi
-
[70]
Springer Series in Statistics
Good, Phillip I.Permutation, Parametric, and Boot- strap Tests of Hypotheses. Springer Series in Statistics. Springer, 3rd edition, 2005. ISBN 978-0387-20279-2. doi: 10.1007/b138696
2005 doi
-
[71]
Anomaly detection: A survey.ACM Computing Surveys, 41(3), 2009
Chandola, Varun, Banerjee, Arindam, and Kumar, Vipin. Anomaly detection: A survey.ACM Computing Surveys, 41(3), 2009. doi: 10.1145/1541880.1541882
2009
-
[72]
Overview of the cross-domain authorship attribu- tion task at pan 2019
Kestemont, Mike, Stamatatos, Efstathios, Manjavacas, Enrique, Daelemans, Walter, Potthast, Martin, and Stein, Benno. Overview of the cross-domain authorship attribu- tion task at pan 2019. InWorking Notes of the Conference and Labs of the Evaluation Forum (CLEF 2019), vol- ume...
2019
-
[73]
An introduction to ROC analysis.Pattern Recognition Letters, 27(8):861–874, 2006
Fawcett, Tom. An introduction to ROC analysis.Pattern Recognition Letters, 27(8):861–874, 2006. doi: 10.1016/ j.patrec.2005.10.010
2006
-
[75]
Es- timation of the youden index and its associated cutoff point.Biometrical Journal, 47(4):458–472, 2005
Fluss, Ronen, Faraggi, David, and Reiser, Benjamin. Es- timation of the youden index and its associated cutoff point.Biometrical Journal, 47(4):458–472, 2005. doi: 10.1002/bimj.200410135
2005 doi
-
[80]
The base-rate fallacy and the difficulty of intrusion detection.ACM Transactions on Information and System Security, 3(3):186–205, 2000
Axelsson, Stefan. The base-rate fallacy and the difficulty of intrusion detection.ACM Transactions on Information and System Security, 3(3):186–205, 2000. doi: 10.1145/ 357830.357849
-
[82]
Linux kernel source code, 2025
The Linux Foundation. Linux kernel source code, 2025. URL https://git.kernel.org/pub/scm/l inux/kernel/git/torvalds/linux.git. Git repository
2025
-
[84]
Sys- tematic construction of anomaly detection benchmarks from real data
Emmott, Andrew F., Das, Shubhomoy, Dietterich, Thomas G., Fern, Alan, and Wong, Weng-Keen. Sys- tematic construction of anomaly detection benchmarks from real data. InProceedings of the ACM SIGKDD Workshop on Outlier Detection and Description, ODD ’13, pages 16–21, Chicago, IL...
2013 doi
-
[85]
Character n-gram features are a standard lexical baseline in authorship evaluation [72]
Additional Experimental Details A.1 Baseline Architecture Details For the statistical baselines described in Section 3.6.2, we follow the feedforward lexical baselines of OCEAN [43]: a shared MLP on top of TF–IDF character features.FNN TF– IDFused character unigrams with a cap...
-
[85]
Character n-gram features are a standard lexical baseline in authorship evaluation [72]
Additional Experimental Details A.1 Baseline Architecture Details For the statistical baselines described in Section 3.6.2, we follow the feedforward lexical baselines of OCEAN [43]: a shared MLP on top of TF–IDF character features.FNN TF– IDFused character unigrams with a cap...
-
[86]
Input format.The original model operates on whole source files. We replaced the input with our linearized code diff (Section 3), tokenized with a SentencePiece 2Implementation source: https://github.com/L1NNA/SourceCodeAuthorshipAnalysis (branchSCS-Gan, pathsrc/SCS-Gan). Stylo...
-
[87]
The verification score is the cosine similarity between the encoder outputs, matching the evaluation protocol for the other baselines
Architecture and training.We retained SCS-GAN’s original adversarial design, a discriminator with multi- head stylistic attention trained against a generator, and trained it on our patch corpus with the original GAN objective. The verification score is the cosine similarity be...
-
[88]
Extending it to commit messages would require revalidating the attention-head design choices, which is outside the scope of this comparison
Modality.SCS-GAN was evaluated only in code-only mode, as the original model’s multi-head attention ar- chitecture for isolating stylistic tokens was designed for source-code inputs. Extending it to commit messages would require revalidating the attention-head design choices, ...
-
[88]
Extending it to commit messages would require revalidating the attention-head design choices, which is outside the scope of this comparison
Modality.SCS-GAN was evaluated only in code-only mode, as the original model’s multi-head attention ar- chitecture for isolating stylistic tokens was designed for source-code inputs. Extending it to commit messages would require revalidating the attention-head design choices, ...
-
[2017]
doi: 10.1371/journal.pone.0 187204
ISSN 1932-6203. doi: 10.1371/journal.pone.0 187204. URL https://dx.plos.org/10.1371/ journal.pone.0187204
1932 doi
-
[2019]
URL https://medium.com/data-scien ce/evading-machine-learning-malware -classifiers-ce52dabdb713 . This blog post demonstrates techniques for modifying malware samples to evade detection by machine learning malware classi- fiers, highlighting the adversarial nature of detection...
-
[2021]
doi: 10.1145/3461666
-
[2022]
doi: 10.1145/3510003.3510181
-
[2023]
URL https://doi.org/10.1109/SP46215.2023 .10179304
doi: 10.1109/SP46215.2023.10179304. URL https://doi.org/10.1109/SP46215.2023 .10179304
2023
-
[2024]
URL https://arxiv.org/abs/2405.1 4993
-
[2025]
doi: 10.1016/j.ipm.2024.104005
2024
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.