Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Wolves in the Repository: A Software Engineering Analysis of the XZ Utils Supply Chain Attack

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that the XZ Utils backdoor succeeded because the attacker spent 2.6 years performing credible maintainer work—translations, CI, code review, releases—so that only eight commits of actual malware were needed.

desk verdict A thorough, useful timeline of the XZ attack, but the evidence links for the 'practice weaponization' claim are missing from the manuscript; fixable, but as is it is conditional. read the letter →

arxiv 2504.17473 v1 pith:UGREGWEO submitted 2025-04-24 cs.SE cs.CR

Piotr Przymus (1) , Thomas Durieux (2) ((1) Nicolaus Copernicus University in Torun , Poland , (2) TU Delft & Endor Labs , The Netherlands) This is my paper · ORCID
classification cs.SEcs.CR
keywords XZUtilsCVE-2024-3094supplychainattackopensourcesoftwaresecuritymaintainertrustGitHubeventanalysisbackdoorengineeringpractices
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper examines the XZ Utils backdoor (CVE-2024-3094) and argues it is a new class of supply chain attack: instead of subverting code alone, the attacker subverted the software engineering process. Over 2.6 years a single persona contributed translations, documentation fixes, code reviews, CI/CD configuration, community management, and finally the migration to GitHub, thereby becoming the de facto maintainer. When the backdoor was pushed in early 2024, only eight commits carried malicious code; the surrounding activity looked like normal maintenance. The authors reconstruct this trajectory from public records and propose a five-phase model of trust-building, with implications for how open-source projects should govern and monitor contributor activity. If they are right, the key defensive lesson is that non-code contributions are a primary attack surface.

What carries the argument

The mechanism that carries the argument is a five-phase model of attacker progression (P1-P5), built from 1,020 commits, 2,944 GitHub events, 307 mailing-list messages, and account checks against breach data. Each phase starts with a new type of activity by the attacker—first patches and reviews, then accepted commits and calls for maintainers, then GitHub organization creation and release announcements, then the ifunc resolver, then the malicious commits. The model makes the attack legible as an ordinary-looking maintenance career, and it is what lets the authors separate the eight malicious commits from the roughly 1,000 benign ones.

What would settle it

An independent audit that recovers withheld IRC logs or private emails showing that the attacker coordinated with the maintainer or the ifunc committer, or a re-annotation of the full commit history that places malicious behavior earlier than January 2024, would contradict the phased model and the 'only eight commits' count.

Watch

Extended reading notes

Core claim

The core discovery is that the XZ Utils attack was executed through the project's maintenance workflow rather than through its code review. The attacker established long-term control by taking over community management, CI/CD configuration, translations, the build system, the website, and the GitHub organization itself; the malicious payload entered through release tarballs that differed from the audited git tree. The paper shows that most of the attacker's 2.6 years of activity falls into phases P1-P3 of trust building and infrastructure control, that the ifunc functionality was introduced by a separate low-activity account in 2023, and that the final phase P5 compressed the malicious commits into a two-month window timed for Red Hat and Debian releases. Because the backdoor lived in distribution packages rather than git, and because the malicious code was hidden inside a binary test file extracted by an obfuscated build script, standard code review and git auditing could miss it.

Load-bearing premise

The paper's load-bearing premise is that the public records (GitHub events, mailing-list archives, and the git history) are complete, so that the attacker's every meaningful action is observable, and that the authors' manual assignment of each commit and message to the attacker persona is accurate.

Editorial extensions

If this is right

  • The same analysis implies that reviewing only code diffs is insufficient for critical projects; maintainers must also monitor who controls CI configuration, issue templates, translations, and release packaging.
  • It implies that a single contributor's gradual assumption of non-code maintainer duties—especially the creation of the project's GitHub organization and default contact email—should be treated as a high-risk governance event.
  • It implies that release artifacts must be reproducible from git, since the backdoor was present only in the tarballs and not in the corresponding commits.
  • It implies that timing attacks around downstream distributors' release schedules (Red Hat, Debian) are part of the attack pattern and should trigger extra review windows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The five-phase model is a candidate template for detecting other long-term takeovers: monitor the share of community-management and infrastructure commits per contributor, and flag monotonic growth in that share with near-zero code contributions.
  • The paper's account suggests that LLM-generated translations and documentation fixes could let a future attacker compress the two-year trust-building phase into months, making non-code contribution vetting a priority.
  • The unresolved role of the ifunc introducer means either a solo campaign with a second persona or a small coordinated group; the paper does not settle this, but the distinction matters for detection.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents a case study of the XZ Utils supply chain attack (CVE-2024-3094), reconstructing a 2.6-year timeline of attacker activity and arguing that the attacker weaponized software engineering practices—community management, CI/CD configuration, translations, GitHub migration, and build-system changes—to establish legitimacy and execute a backdoor delivered through release tarballs. The authors assemble a public dataset from git history, GitHub Archive events, mailing lists, and breach databases; propose a five-phase model (P1–P5); and analyze the attack's impact on the OSS ecosystem. The central claim is that this constitutes a "new breed" of supply chain attack in which development practices themselves, rather than only code, are exploited.

Significance. If the stated evidence base is fully substantiated, the paper would be a valuable contribution to OSS security research: it provides a detailed, mixed-methods timeline of a sophisticated social-engineering-driven attack, offers a public replication package, and explicitly addresses threats to validity including attribution ambiguity and hindsight bias. The qualitative framework for categorizing attacker SE practices could inform future detection tools. However, the load-bearing evidence for the strongest claims—the eight-malicious-commit count and the practice-weaponization matrix—is not verifiable in the submitted manuscript, which substantially limits the current significance.

major comments (3)
  1. [Section III-C, Table III] The central claim that specific SE practices were 'weaponized' is supported by Table III, but most rows of the Security Implication column contain only the placeholder '[link]' (e.g., Community Management, Setup CI/CD, Build System Changes, GitHub Migration, Website Migration, Mailing List Engagement). As submitted, the manuscript does not provide the evidence links or a per-claim mapping to commits, events, or mailing-list messages in the replication package. Without this evidence, a reader cannot distinguish weaponized practices from ordinary maintenance, which is exactly the distinction the 'new breed' thesis depends on. Please replace every placeholder with a working reference or an explicit artifact identifier (commit SHA, event ID, message URL) and add a data-availability mapping table.
  2. [Section III-D and III-B (Answer to RQ1)] The claim 'eight commits were confirmed to be malicious' is stated as a fact in Answer to RQ1 and in Section III-D ('In total, eight commits were confirmed to be malicious'), but no commit identifiers, diff hashes, or selection criteria are given anywhere in the manuscript. Since Section VI-A concedes that some commits may have been prepared jointly and that private communications are missing, this count is not independently auditable. Please provide an explicit list of the eight commits (e.g., SHAs in the replication package) and a description of the classification criteria used to label them malicious rather than merely reverted or suspicious.
  3. [Abstract, Section III-B Answer to RQ2] The paper's framing as revealing 'a new breed of supply chain attack' generalizes from a single case study without a comparison baseline against other OSS attacks or against legitimate high-activity maintainers. The manuscript does not operationally define what counts as 'weaponized SE practice' as opposed to enthusiastic contribution, which makes the central conclusion difficult to evaluate or replicate. Please either temper the generalization (explicitly present it as a hypothesis or an exploratory framework) or add a control/comparison analysis, and define measurable indicators of weaponization.
minor comments (5)
  1. [Abstract] The abstract contains an incomplete sentence: 'we reconstruct the attack timeline, analyze the evolution of attacker tactics.' It lacks a main verb for the second clause; please rephrase.
  2. [Table II] Several rows in Table II contain the literal placeholder '[link]' (e.g., P1 row 1, P1 row 2, P2 row 3, P2 row 4, P3 row 2, P3 row 3). These should either be resolved to actual URLs or, if the footnote URLs are intended, the table should point to the footnotes consistently.
  3. [Figure 1] Figure 1 is dense and the dark shading for collaborative commits is hard to distinguish in the '# Commits' and '% Commits' panels; consider using a distinct color or hatch pattern and increasing font sizes for axis labels.
  4. [Data Availability] The replication package URL is given, but there is no description of its internal structure (e.g., which files contain the commit list, event logs, or annotation results). Adding a README excerpt or file tree in the manuscript would aid reproducibility.
  5. [Section I] The phrase 'highlighting in inerrant complexity of the task' appears to contain a typo; likely 'inherent complexity' is intended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the case study's claims are empirical and grounded in public data, with the only self-citation used as an annotation tool rather than as a justification of the findings.

full rationale

The paper makes no formal derivation in which an output is defined from the quantity it claims to predict. RQ1's timeline and RQ2's practice inventory are descriptive reconstructions from public GitHub events, git history, mailing lists, and security databases; the phase model is a periodization of observed activity, not a fitted parameter renamed as a result. The claim that only eight commits were malicious is presented as a confirmed outcome of the post-incident review (Section III-D: 'In total, eight commits were confirmed to be malicious'), an external fact about the incident rather than a quantity derived from the authors' own model. The only self-citation, PatchScope [15], is used as an 'automated annotation technique' to label code changes by type; the paper does not invoke [15] as proof of its central argument, and the conclusions about community management, CI/CD control, and migration are supported by manual analysis of communications and repository events. The unresolved '[link]' placeholders in Table III and the acknowledgment in Section VI-A that private communications are missing weaken the evidence base and verifiability, but they are completeness and correctness limitations, not circularity. No load-bearing premise reduces to the paper's own conclusions, and no prediction is statistically forced by a fitted input. A non-finding is therefore appropriate.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

This is an empirical case study; the central claims rest on data completeness and attribution rather than on fitted parameters or invented entities. The free-parameter count is zero, and the main assumptions are about the validity of the public record and the annotation process.

assumptions (3)
  • domain assumption The assembled dataset (Git commits, GitHub events, mailing list archives) is a complete and accurate record of the project's development activity.
    Section III-A describes the four data sources; external records like private IRC are missing, and the authors acknowledge this in Section VI-A.
  • domain assumption Contributions and communications can be reliably attributed to distinct personas (attacker, maintainer, etc.) using public metadata.
    Section VI-A admits some commits may be joint and attribution is ambiguous.
  • domain assumption Manual open coding of activities into categories is a valid representation of the attacker's software engineering practices.
    Section I-A says no established framework existed; coding was iterative and performed by the authors without reported inter-rater reliability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Wolves in the Repository: A Software Engineering Analysis of the XZ Utils Supply Chain Attack." pith.science (2026). https://pith.science/paper/UGREGWEO

@misc{pith2026250417473,
  author       = {Pith},
  title        = {Pith review of: Wolves in the Repository: A Software Engineering Analysis of the XZ Utils Supply Chain Attack},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UGREGWEO}},
  note         = {Machine review of arXiv:2504.17473}
}
read the original abstract

The digital economy runs on Open Source Software (OSS), with an estimated 90\% of modern applications containing open-source components. While this widespread adoption has revolutionized software development, it has also created critical security vulnerabilities, particularly in essential but under-resourced projects. This paper examines a sophisticated attack on the XZ Utils project (CVE-2024-3094), where attackers exploited not just code, but the entire open-source development process to inject a backdoor into a fundamental Linux compression library. Our analysis reveals a new breed of supply chain attack that manipulates software engineering practices themselves -- from community management to CI/CD configurations -- to establish legitimacy and maintain long-term control. Through a comprehensive examination of GitHub events and development artifacts, we reconstruct the attack timeline, analyze the evolution of attacker tactics. Our findings demonstrate how attackers leveraged seemingly beneficial contributions to project infrastructure and maintenance to bypass traditional security measures. This work extends beyond traditional security analysis by examining how software engineering practices themselves can be weaponized, offering insights for protecting the open-source ecosystem.

Figures

Figures reproduced from arXiv: 2504.17473 by the authors.

Figure 1
Figure 1. ♂Aand ♂PMactions, aggregated monthly and divided into commits and user interaction events. Vertical lines indicate the start of a phase with major events described in Table II. The # plots display the number of events of each type, while the % plots show the percentage of events. If the changes are a result of collaboration between ♂Aand ♂PM, they are marked with a darker color. rewrite the project’s history using p… view at source ↗
Figure 2
Figure 2. ♂AContributions by Project Directory and Type. This plot visualizes all ♂AGit commits, with each line automatically annotated and aggregated. The flow represents the total number of modified lines from commits. Nodes on the left show project directories, while the right categorizes contributions (code, documentation, tests, translations). The width of connections indicates the volume of changes, highlighting where ♂… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adversarial Bug Reports as a Security Risk in Language Model-Based Automated Program Repair

    cs.SE 2025-09 conditional novelty 6.0 of 10

    Adversarial bug reports induced attacker-desired patches in 90% of trials, while the best tested pre-repair filter caught only 47%, exposing a structural weakness in LLM-based automated program repair.

Reference graph

Works this paper leans on

23 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    GH Archive

    2011. GH Archive. https://www.gharchive.org/. [Accessed 01-07-2024]

    work page 2011

  2. [2]

    Gerardo Canfora, Andrea Di Sorbo, Sara Forootani, Antonio Pirozzi, and Corrado Aaron Visaggio. 2020. Investigating the vulnerability fixing process in OSS projects: Peculiarities and challenges. Computers & Security 99 (2020), 102067

    work page 2020

  3. [3]

    Jailton Coelho and Marco Tulio Valente. 2017. Why modern open source projects fail. In Proceedings of the 2017 11th Joint Meeting on F oundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). As- sociation for Computing Machinery, New York, NY , USA, 186–196. https://doi.org/10.1145/3106237.3106246

    arXiv 2017

  4. [4]

    Edson Dias, Paulo Meirelles, Fernando Castor, Igor Steinmacher, Igor Wiese, and Gustavo Pinto. 2021. What makes a great maintainer of open source projects?. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) . IEEE, 982–994

    work page 2021

  5. [5]

    Luis Felipe Dias, Igor Steinmacher, and Gustavo Pinto

    work page

  6. [6]

    Oliver Gasser, Ralph Holz, and Georg Carle. 2014. A deeper understanding of SSH: Results from Internet- wide scans. In 2014 IEEE Network Operations and Management Symposium (NOMS) . 1–9. https://doi.org/ 10.1109/NOMS.2014.6838249

    work page arXiv 2014

  7. [7]

    Mariam Guizani, Amreeta Chatterjee, Bianca Trinken- reich, Mary Evelyn May, Geraldine J Noa-Guevara, Liam James Russell, Griselda G Cuevas Zambrano, Daniel Izquierdo-Cortazar, Igor Steinmacher, Marco A Gerosa, et al . 2021. The long road ahead: Ongoing challenges in contributing to large oss organizations and what to do. Proceedings of the ACM on Human-Co...

    work page 2021

  8. [8]

    Hissam, Daniel Plakosh, and C Weinstock

    Scott A. Hissam, Daniel Plakosh, and C Weinstock. 2002. Trust and vulnerability in open source software. IEE Proceedings-Software 149, 1 (2002), 47–51

    work page 2002

Show all 23 references
  1. [9]

    Troy Hunt. 2019. Have i been pwned. URL: https://haveibeenpwned. com (visited on 10/15/2020) (2019)

    2019

  2. [10]

    Valentina Lenarduzzi, Davide Taibi, Davide Tosi, Luigi Lavazza, and Sandro Morasca. 2020. Open source software evaluation, selection, and adoption: a systematic literature review. In 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) . IEEE...

    2020

  3. [11]

    Hila Lifshitz-Assaf and Frank Nagle. 2021. The digital economy runs on open source. Here’s how to protect it. Harvard Business Review 2 (2021)

    2021

  4. [12]

    Haralambos Mouratidis, Paolo Giorgini, and Gordon Manson. 2005. When security meets software engineering: a case of modelling secure information systems. Infor- mation Systems 30, 8 (2005), 609–629

    2005

  5. [13]

    Patrick Howell O’Neill. [n. d.]. The Internet Runs on Free Open-Source Software. Who Pays to Fix It? https://www.technologyreview.com/2021/12/17/1042692/log4j- internet-open-source-hacking/

    2021

  6. [14]

    Christian Payne. 2002. On the security of open source software. Information systems journal 12, 1 (2002), 61– 78

    2002

  7. [15]

    Piotr Przymus, Jakub Nar˛ ebski, Mikołaj Fejzer, and Krzysztof Stencel. 2024. PatchScope – A Modular Tool for Annotating and Analyzing Contributions. https://ncusi. github.io/PatchScope/articles/patchscope_2024.pdf (under review)

    2024

  8. [16]

    The Register. 2020. What happens when the maintainer of a JS library downloaded 26m times a week goes to prison for killing someone with a motorbike? Core-js just found out. https://www.theregister.com/2020/03/26/ corejs_maintainer_jailed_code_release. [Accessed 28-06- 2024]

    2020

  9. [17]

    Simone Scalco, Ranindya Paramitha, Duc-Ly Vu, and Fabio Massacci. 2022. On the feasibility of detecting injections in malicious npm packages. In Proceedings of the 17th International Conference on Availability, Reliability and Security . 1–8

    2022

  10. [18]

    Tien Rahayu Tulili, Andrea Capiluppi, and Ayushi Rastogi

  11. [19]

    Xinda Wang, Kun Sun, Archer Batcheller, and Sushil Jajodia. 2019. Detecting" 0-day" vulnerability: An empirical study of secret security patch in OSS. In 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) . IEEE, 485– 492

    2019

  12. [20]

    Xiaoyan Zhou, Ying Zhang, Wenjia Niu, Jiqiang Liu, Haining Wang, and Qiang Li. 2024. OSS Malicious Pack- age Analysis in the Wild. arXiv preprint arXiv:2404.04991 (2024)

    2024 arXiv

  13. [21]

    Markus Zimmermann, Cristian-Alexandru Staicu, Cam Tenny, and Michael Pradel. 2019. Small world with high risks: A study of security threats in the npm ecosystem. In 28th USENIX Security symposium (USENIX security 19). 995–1010

    2019

  14. [2018]

    Who drives company-owned OSS projects: internal or external members? Journal of the Brazilian Computer Society 24, 1 (2018), 16

    2018

  15. [2023]

    Information and Software Technology 155 (2023), 107116

    Burnout in software engineering: A systematic mapping study. Information and Software Technology 155 (2023), 107116

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.