Pith. sign in

REVIEW 1 major objections 6 minor 16 references

A Global Author-Identity Map for the World of Code:62.7M Developer Identities from 106.8M Author Strings over 5.87B Commits

T0 review · 1 major / 6 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read 62.7M Developer Identities Merged from 106.8M Strings Over 5.87B Commits

desk verdict Mega-cluster-free author-identity map at 10^8 scale is a real artifact release; the splitting/clumping framing is the methodological contribution. The precision claim is narrower than it looks. read the letter →

arxiv 2607.06183 v1 pith:H4OETHLH submitted 2026-07-07 cs.SE

Audris Mockus This is my paper · ORCID
classification cs.SE
keywords authorcommitsclumpingglobalidentitiesidentityprecisionstrings
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper releases a curated author-identity map for the entire World of Code V2604 corpus — 5,866,595,698 git commits authored under 106,826,059 distinct name/email strings. The core problem the paper addresses is that at hundred-million-id scale, the naive approach to merging aliases (taking the transitive closure over all shared-attribute edges) welds roughly three million unrelated people into a single mega-cluster, silently corrupting every per-author aggregate computed on top. The released map avoids this by a six-stage pipeline that gates bad attributes, cuts structural bridges in the union graph, classifies candidate edges with a logistic model trained on 2.6 million free GitHub labels, and then recovers recall through a classifier-filtered shingle expansion. The result is a mega-cluster-free map whose largest cluster is 6,910 ids, resolving 73.5% of all commits into multi-id identities and 98.17% of human commits. Against the ALFAA human-rated gold set it scores recall 0.70 and precision 0.88, while the prior WoC map's apparent precision of 0.95 collapses to 0.52 once its 3,006,318-id mega-cluster is counted. The paper argues that the dominant evaluation failure in large-scale author disambiguation is measuring only splitting (recall) while leaving clumping (over-merge) untested, and that high precision claims in prior global-scale union maps are an artifact of never measuring the conflated region. It then demonstrates that the choice of identity method shifts downstream analytics in concrete, opposite directions: under-aliasing inflates developer head-count by 66.6% and invents 10.1M phantom one-commit developers, while over-aliasing welds strangers into giant phantom identities. Both errors distort bus-factor estimates, collaboration-network density, and centrality rankings in opposite directions.

What carries the argument

The six-stage pipeline (gate values, cut topology, classify edges, recover recall, resolve locally) and the splitting-vs-clumping evaluation framework that grades on two independent ground truths read jointly.

What would settle it

If the largest cluster in the released map were shown to exceed 10,000 ids, or if a broader human gold set (beyond OpenStack) produced precision below 0.70, the central claim of mega-cluster-free calibrated resolution would be undermined.

Watch

Extended reading notes

Core claim

The central discovery is that at 10^8-id scale, author de-aliasing has two failure families — splitting (one person scattered across aliases) and clumping (strangers merged into one identity) — and they must be solved in that order, because applying recall mechanisms to an over-merged graph only worsens the mega-cluster. The paper shows that value-based gating alone cannot dissolve the mega-cluster (neutralizing every detectable bad attribute still leaves it 94% intact), because a redundant mesh of moderately-bad values re-closes around each removed weld. What works is changing the unit of analysis: first from nodes to topology (a sampled-betweenness cut that identifies roughly 2,000 load-1,

Load-bearing premise

The precision claim of 0.88 rests on the ALFAA gold set, which is drawn from a single community (OpenStack). If that community's naming patterns are not representative of the global developer population, the measured precision may not generalize to the full 106.8M-id space.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 6 minor

Summary. This paper releases a curated author-identity map for World of Code (WoC) V2604, folding 106.8M raw author/committer strings into 62.7M canonical identities over 5.87B commits. Four co-versioned artifacts ship: a global alias map (a2AFullSUG), a per-identity quality classification (A2clsFull), a within-project resolution table (P2aAFull), and a commit-to-identity provenance table (c2AFull). The central methodological contribution is the splitting/clumping distinction: the paper argues that prior global-scale union maps report high precision only because their evaluation never reaches the over-merged (clumped) region, and demonstrates that the prior WoC V3 map's 0.95 precision collapses to 0.52 once its 3M-id mega-cluster is counted. The released map is mega-cluster-free (largest cluster 6,910 ids), scoring recall 0.70 / precision 0.88 on the ALFAA human gold set. A six-stage pipeline (link generation, value gating, structural gating via betweenness cut, edge classification, recall recovery via shingle expansion, representative selection) is described, with design choices traced to an eighteen-experiment record in a companion methodology paper. Downstream impact is quantified via an ablation ladder (L0/L1/L2) showing that under-merge and over-merge distort developer analytics in opposite directions across head-count, productivity, bus factor, collaboration networks, and centrality. A cross-corpus linking pilot to Semantic Scholar via DOI anchoring is presented as a broader-imp

Significance. The release of a mega-cluster-free, per-commit-provenance-tagged identity map over the entire WoC corpus is a significant community resource. The splitting/clumping distinction (§5) is a genuinely useful conceptual contribution: the argument that pairwise precision on curated samples can hide quadratic-mass clumping is well-framed and the demonstration that V3's 0.95 precision collapses to 0.52 when the mega-cluster is counted is compelling. The ablation ladder (§6) cleanly traces identity errors into downstream analytics, and the finding that 36.3% of apparently multi-developer projects are one person under multiple aliases is striking. The per-commit provenance tags (c2AFull) that let consumers choose their own precision/recall operating point are a practical design strength. The DOI-anchored cross-corpus linking pilot (§7), while small (7,246 authors), demonstrates a falsifiable approach to the clumping problem across corpora. The replication package and co-versioned artifacts are appropriate for a data-description paper.

major comments (1)
  1. §5, Table 4: The paper's central methodological argument is that pairwise precision can hide clumping (§5, p.5: 'sampling pairs in a way that under-weights a single giant cluster whose false-pair mass grows quadratically in its size'). However, the released map's own precision claim of 0.88 is also pairwise, evaluated on 469k ALFAA pairs from one community. The paper supplements this with the largest-cluster diagnostic (6,910), but this is a single data point. The paper does not report: (a) the full cluster-size distribution below 6,910 (only that the >10k bin is empty); (b) cluster-level purity (what fraction of clusters contain exactly one real person); (c) how many clusters exist in the 100–6,910 range and how many ids they collectively cover. If there are hundreds of clusters in the 100–1,000 range with false merges, the pairwise precision of 0.88 could overstate true cluster-level精度
minor comments (6)
  1. §3, stage 5: The two shingle-expansion thresholds (v1 at τ=0.9, v2.1 at τ=0.8) are stated without explaining how they were selected. A brief note on the selection criterion would help readers.
  2. §5: The paper mentions '~26 mislabels' in the ALFAA gold set and adjusts precision to ~0.90, but does not describe how these mislabels were identified. A one-sentence methodology would suffice.
  3. §6.3, Tables 9–10: The fan-out cap of 1,000 co-developers per project is mentioned but its sensitivity is not discussed. How many projects are truncated, and does the 32% edge-removal figure change materially with a different cap?
  4. §7: The DOI-anchored pilot links 7,246 authors, which is small relative to the 62.67M canonical identities. The paper is transparent about this coverage limitation, but the heading 'Broader Impact' may set expectations the pilot cannot meet; consider softening.
  5. Table 2: The 'partial' class (0.23%) is defined as 'otherwise-good but carries only one of name/email,' but it is unclear whether these ids are merged globally or treated like local/bad-by-attribute ids.
  6. §4, Table 3: Several entries reference experiments by number (e.g., 'Exp. 14') from the companion methodology paper [13], which is listed as a 2026 reference with no venue. Since the full experimental record is not available to the reader, the table's justifications are partially unverifiable. Consider noting availability.

Simulated Author's Rebuttal

1 responses · 1 unresolved

We thank the referee for the careful and constructive assessment. The recommendation of minor revision is appropriate, and the single major comment identifies a genuine gap in the cluster-size distribution diagnostics that we will close in the revised manuscript. We agree that reporting only the largest-cluster size (6,910) as a clumping diagnostic is insufficient without the supporting distribution context, and we will add the requested statistics. We note one honest limitation: the ALFAA gold set's community scope (OpenStack-centric) means our cluster-level purity estimate is necessarily partial, and we will state this explicitly rather than overclaim.

read point-by-point responses

  1. Referee: §5, Table 4: The paper's central methodological argument is that pairwise precision can hide clumping. However, the released map's own precision claim of 0.88 is also pairwise, evaluated on 469k ALFAA pairs from one community. The paper supplements this with the largest-cluster diagnostic (6,910), but this is a single data point. The paper does not report: (a) the full cluster-size distribution below 6,910; (b) cluster-level purity (what fraction of clusters contain exactly one real person); (c) how many clusters exist in the 100–6,910 range and how many ids they collectively cover. If there are hundreds of clusters in the 100–1,000 range with false merges, the pairwise precision of 0.88 could overstate true cluster-level accuracy.

    Authors: The referee is correct on all three sub-points. Our central argument—that pairwise precision can hide quadratic-mass clumping—cuts both ways: if our own 0.88 pairwise precision also hides moderate-sized clumped clusters in the 100–6,910 range, then our clumping diagnostic is incomplete. We will address this in the revision by adding three pieces of information to §5. (a) We will report the full cluster-size distribution (histogram of cluster sizes from 2 to 6,910), not just the maximum. (b) We will report cluster-level purity on the subset of clusters that intersect the ALFAA gold set: for each cluster containing at least one gold id, the fraction that map to a single real person. We will be explicit that this is a partial measure, bounded by the OpenStack-centric scope of ALFAA, and cannot certify purity for clusters that the gold set does not touch. (c) We will report the count of clusters in the 100–6,910 range and the total number of ids they collectively cover, so the reader can assess whether the pairwise precision is dominated by small clean clusters or inflated by a few large dirty ones. We expect the distribution to be heavily long-tailed (the vast majority of clusters are size 2–5), but the referee is right that this should be shown, not asserted. We will also add an explicit acknowledgment that the 0.88 pairwise precision shares the limitation we identify in prior work—it is a pairwise average that could, in principle, underweight clumping in moderate clusters—and that the cluster-size distribution and purity statistics are the necessary complement, just as the largest-cluster diagnostic is the necessary complement to V3's pairwise precision. One honest limitation we will state: without a global (non-community-specific) human gold set, cluster-level purity is revision: no

standing simulated objections not resolved
  • Cluster-level purity cannot be computed globally because no global human-rated ground truth exists. The ALFAA gold set is OpenStack-centric, so purity is estimable only for clusters it intersects. We will report this partial measure and label it as such, but we cannot honestly claim a global purity figure.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the map is built from independent data sources and validated against external benchmarks; the one self-citation is a methodology log, not a load-bearing premise.

full rationale

The paper's central claims (mega-cluster freedom, precision/recall on ALFAA gold, commit coverage) are validated against two external ground truths: the ALFAA human-rated gold set [1] and the Bock et al. GitHub within-handle benchmark [4]. The construction pipeline trains its edge classifier on 2.6M free within-handle GitHub-id labels — an independent label source, not the target output. The prior WoC map V3 (ALFAA lineage) is by overlapping authors, but the comparison is adversarial (showing V3's precision collapses to 0.52 when its mega-cluster is counted), not a premise that forces the new map's conclusion. The companion methodology paper [13] is self-cited for the full eighteen-experiment log, but the paper states each production decision inline with its measured justification (Table 3), and the methodology log is an external record of experiments, not a theorem or ansatz that the present result assumes. The ALFAA gold set [1] shares an author (Mockus), but it is used as an evaluation benchmark, not as a construction input — the map is not fitted to ALFAA pairs. The paper's own acknowledgment that ALFAA is OpenStack-centric and 'neither benchmark is a complete oracle' (Section 9) is a limitation disclosure, not a circularity. No step in the derivation chain reduces to its inputs by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new postulated entities (particles, forces, dimensions, etc.). It introduces data artifacts (the four tables) and a pipeline, but these are constructed objects, not axiomatic postulates. The free parameters are threshold values tuned on data, not fundamental constants. The axioms are standard domain assumptions for entity resolution in software repositories.

free parameters (4)
  • shingle expansion threshold v1 (tau) = 0.9
    Tuned threshold for pairwise shingle expansion; stated as the largest single recall gain in the record (Exp. 14).
  • shingle expansion threshold v2.1 (tau) = 0.8
    Tuned threshold for map-level shingle expansion; at 0.9 the map-level variant sacrifices GitHub recall.
  • fan-out cap for collaboration network = 1000
    Caps co-developers per project to exclude mega-project N^2 cliques; chosen by hand.
  • bus factor threshold = 0.5
    Commit-concentration threshold defining the bus factor as minimum developers covering 50% of commits.
assumptions (5)
  • domain assumption Two raw author/committer strings that share an exact email or exact name represent the same person unless evidence contradicts.
    Section 3, Stage 1 (Link generation): candidate same-person edges are emitted between strings sharing exact email, exact name, or a shared rare handle.
  • domain assumption A sampled betweenness cut on the exact union graph identifies load-bearing bridge ids that weld otherwise-disjoint communities.
    Section 3, Stage 3 (Structural gating) and Section 4 (Exp. 11): the structural cut is the mechanism that dissolves the mega-cluster.
  • domain assumption Within-handle GitHub-id labels are a valid training signal for edge classification.
    Section 3, Stage 4: the logistic edge classifier is trained on 2.6M free within-handle GitHub-id labels.
  • domain assumption The ALFAA human-rated gold set is a representative benchmark for measuring precision and recall of the global map.
    Section 5: the map's precision (0.88) and recall (0.70) are reported against this gold set. Section 9 acknowledges it is OpenStack-centric.
  • domain assumption The WoC V2604 collection is a complete census of public version control.
    Section 1: WoC is described as 'a periodically updated census of public version control, deduplicated at the git-object level.'

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Global Author-Identity Map for the World of Code:62.7M Developer Identities from 106.8M Author Strings over 5.87B Commits." pith.science (2026). https://pith.science/paper/H4OETHLH

@misc{pith2026260706183,
  author       = {Pith},
  title        = {Pith review of: A Global Author-Identity Map for the World of Code:62.7M Developer Identities from 106.8M Author Strings over 5.87B Commits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H4OETHLH}},
  note         = {Machine review of arXiv:2607.06183}
}
read the original abstract

Mining software repositories at global scale founders on author identity: the same developer commits under many name/email strings, and the same string is reused by many developers. We release a curated author-identity map for World of Code (WoC) version V2604, covering all 5,866,595,698 commits. It ships four co-versioned artifacts: a global alias map (a2AFullSUG) folding 106,826,059 raw author/committer strings into canonical identities; a per-identity classification (A2clsFull) tagging each id good, bad-by-attribute, local, bot, or partial; a within-project table (P2aAFull) recovering low-quality ids inside the one project where their reuse is unambiguous; and a commit-to-identity table (c2AFull) tagging every commit with its resolution provenance. The map is mega-cluster free, its largest cluster 6,910 ids (one GitHub noreply identity), and it resolves 73.5% of six billion commits into multi-id identities, raising human-id commit coverage to 98.17%. The design problem is clumping, not recall: a naive transitive union over shared-attribute edges welds three million unrelated people into one cluster, an over-merge that recall-only benchmarks price at zero. We report both error families, splitting and clumping, and show the high precision claimed by global-scale union maps can be an artifact of never measuring the conflated region. Against the ALFAA human-rated gold set the map scores recall 0.70 / precision 0.88, where the prior WoC map's apparent 0.95 precision collapses to 0.52 once its 3,006,318-id mega-cluster is counted. A canonical software-author identity is also a cross-corpus join key to scholarly author graphs, where clumping is again the binding constraint. All artifacts ship with the WoC V2604 release and a self-contained replication package.

Figures

Figures reproduced from arXiv: 2607.06183 by the authors.

Figure 1
Figure 1. How every one of the 5,866,595,698 V2604 commits is resolved, by provenance tag. The global map carries the bulk; the within-project layer adds the harder ids it cannot safely merge, lifting human-id coverage to 98.17%. (3) Within-project resolution P2aAFull. An id that is ambiguous across projects (root, ubuntu, user@laptop) is usually unambigu￾ous within the single deforked project where it appears, because reuse … view at source ↗
Figure 2
Figure 2. The six-stage construction. Value, structural, and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Gold-arbiter precision, recall, and clumping (lower [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

  1. [1]

    Sadika Amreen, Yuxia Zhang, Chris Bogart, Russell Zaretzki, and Audris Mockus

    work page

  2. [2]

    https://doi.org/10.1007/s10664-019-09786-7 Scaling Author Identity Disambiguation to the World of Code: A Methodology Conference’17, July 2017, Washington, DC, USA

    ALFAA: Active Learning Fingerprint Based Anti-Aliasing for correcting de- veloper identity errors in version control systems.Empirical Software Engineering 25, 2 (2019), 1136–1167. doi:10.1007/s10664-019-09786-7

    work page doi:10.1007/s10664-019-09786-7 2019

  3. [3]

    Guilherme Avelino, Leonardo Passos, Andre Hora, and Marco Tulio Valente

    work page

  4. [4]

    InProceedings of the 24th International Conference on Program Comprehension (ICPC)

    A novel approach for estimating truck factors. InProceedings of the 24th International Conference on Program Comprehension (ICPC). 1–10. doi:10.1109/ ICPC.2016.7503718

    work page arXiv 2016

  5. [5]

    Christian Bird, Alex Gourley, Premkumar Devanbu, Michael Gertz, and Anand Swaminathan. 2006. Mining Email Social Networks. InProceedings of the 3rd A Global Author-Identity Map for the World of Code: 62.7M Developer Identities from 106.8M Author Strings over 5.87B Commits Conference’17, July 2017, Washington, DC, USA International Workshop on Mining Softw...

    work page arXiv 2006

  6. [6]

    Thomas Bock, Aakanksha Agarwal, Mitchell Atkisson, Audris Mockus, and Bog- dan Vasilescu. 2025. Between Accuracy and Privacy: Rethinking Developer- Identity De-Aliasing in Git Repositories.Empirical Software Engineering (2025). Under submission. Tool: GitAuthority, https://github.com/bockthom/ gitauthority

    work page 2025

  7. [7]

    2012.Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection

    Peter Christen. 2012.Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. Springer. doi:10.1007/978-3-642-31164- 2

    work page doi:10.1007/978-3-642-31164- 2012

  8. [8]

    Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S. Weld

    work page

Show all 16 references
  1. [9]

    InProceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics (ACL)

    SPECTER: Document-level Representation Learning using Citation- informed Transformers. InProceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics (ACL). 2270–2282. doi:10.18653/v1/2020.acl- main.207

    2020 doi

  2. [10]

    Tapajit Dey, Sara Mousavi, Eduardo Ponce, Tanner Fry, Bogdan Vasilescu, Anna Filippova, and Audris Mockus. 2020. Detecting and Characterizing Bots that Commit Code. InProceedings of the 17th International Conference on Mining Software Repositories (MSR). 209–219. doi:10.1145/3...

    2020 doi

  3. [11]

    Tanner Fry, Tapajit Dey, Andrey Karnauch, and Audris Mockus. 2020. A dataset and an approach for identity resolution of 38 million author ids extracted from 2B git commits. InProceedings of the 17th International Conference on Mining Software Repositories (MSR), Data Showcase....

    2020 doi

  4. [12]

    Mathieu Goeminne and Tom Mens. 2013. A Comparison of Identity Merge Algorithms for Software Repositories.Science of Computer Programming78, 8 (2013), 971–986. doi:10.1016/j.scico.2011.11.004

    2013 doi

  5. [13]

    Christoph Gote and Christian Zingg. 2021. gambit – An Open Source Name Disambiguation Tool for Version Control Systems. InProceedings of the 18th International Conference on Mining Software Repositories (MSR). 80–84. doi:10. 1109/MSR52588.2021.00021

    2021

  6. [14]

    Erik Kouters, Bogdan Vasilescu, Alexander Serebrenik, and Mark G. J. van den Brand. 2012. Who’s Who in GNOME: Using LSA to Merge Software Repository Identities. InProceedings of the 28th IEEE International Conference on Software Maintenance (ICSM). 592–595. doi:10.1109/ICSM.20...

    2012 doi

  7. [15]

    Yuxing Ma, Tapajit Dey, Chris Bogart, Sadika Amreen, Marat Valiev, Adam Tutko, David Kennard, Russell Zaretzki, and Audris Mockus. 2021. World of Code: Enabling a Research Workflow for Mining and Analyzing the Universe of Open Source VCS Data.Empirical Software Engineering26, ...

    2021

  8. [16]

    Audris Mockus. 2026. Scaling Author Identity Disambiguation to the World of Code: A Methodology. Companion methodology log; full experimental record (Exps. 1–18)

    2026

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.