{"id":"fdcf52de-17de-4071-8aa1-26167e08d463","arxiv_id":"2607.06183","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":4,"one_line_summary":"A mega-cluster-free author-identity map for 5.87B git commits folds 106.8M raw strings into 62.7M canonical identities, validated jointly for splitting and clumping errors.","lead":"This paper releases a curated author-identity map for 5.87B git commits that folds 106.8M raw author strings into 62.7M canonical developer identities while avoiding the mega-cluster over-merge problem. A smart generalist might read it because it demonstrates that prior identity maps at this scale silently conflated millions of unrelated developers, corrupting downstream analytics.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The paper's own argument—that pairwise precision can hide clumping—applies to its own 0.88 precision claim, which is also pairwise and evaluated on a narrow gold set. The paper reports only the single largest cluster (6,910) as a clumping diagnostic, not the full cluster-size distribution or cluster","rationale":"The reader correctly identified the evaluation's narrow foundation (ALFAA is OpenStack-centric) as the weakest assumption. My concern is more specific: the paper's own central argument—that pairwise precision can systematically hide clumping—applies to its own evaluation, which is also pairwise and supplemented only by the single largest-cluster data point. The paper does not report the full cluster-size distribution or cluster-level purity, which are the direct measures of the clumping error family it claims to control. This is a real gap, but it is addressable without invalidating the core contribution. The mega-cluster-free property is verifiable by inspection of the released artifacts, the commit-coverage census is a factual claim, and the downstream impact analysis (§6) is thorough and runs end-to-end on L0 vs L2. The 18-experiment justification record is deferred to an unavailable companion paper [13], which limits independent verification of the design rationale, but the pipeline itself is shipped and reproducible. The CONDITIONAL verdict with MODERATE confidence is appropriate: the central claim is defensible, the artifacts ship, but the precision claim rests on an evaluation methodology that has a specific, identifiable weakness the paper itself described but did not close the loop on. A cluster-level purity analysis on the released map would either confirm 0.88 or reveal distributed clumping that the pairwise metric missed.","tokens_in":18133,"tokens_out":5835,"duration_ms":390840,"concrete_test":"Report the full cluster-size histogram (bins: 1, 2–10, 11–100, 101–1,000, 1,001–6,910) and compute cluster-level precision on the ALFAA gold set: for each cluster containing ≥2 ALFAA gold ids, check whether all gold ids in that cluster share the same true identity label. If any size bin's cluster-level precision falls below 0.85, the pairwise precision of 0.88 overstates the true clumping rate for that bin, and the headline precision claim should be qualified by cluster size.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central methodological contribution is the splitting/clumping distinction (§5). It argues that prior work's high precision is an artifact of 'sampling pairs in a way that under-weights a single giant cluster (whose false-pair mass grows quadratically in its size).' Yet the paper's own precision measurement (0.88) is also pairwise, evaluated on 469k ALFAA pairs from one community. The paper supplements this with the largest-cluster diagnostic (6,910), but this is a single data point. The paper does not report: (a) the full cluster-size distribution below 6,910—only that the >10k bin is empty; (b) cluster-level purity, i.e., what fraction of clusters contain exactly one real person; (c) how many clusters exist in the 100–6,910 range and how many ids they collectively cover. If there are, say, hundreds of clusters in the 100–1,000 range with false merges, the pairwise precision of 0.88 could overstate true cluster-level precision—precisely the failure mode the paper attributes to prior work. The 6,910-id cluster is explained as 'a single GitHub noreply identity,' but the paper does not independently verify all 6,910 members are the same person, nor does it report whether other large clusters exist below that ceiling. The reader correctly identified the OpenStack-centric narrowness of ALFAA, but the more specific issue is that the paper's own evaluation methodology inherits the weakness it criticizes: pairwise precision on a narrow sample, without cluster-level purity or a full size distribution, cannot rule out distributed clumping across many medium-sized clusters.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper releases a curated author-identity map for World of Code (WoC) V2604, folding 106.8M raw author/committer strings into 62.7M canonical identities over 5.87B commits. Four co-versioned artifacts ship: a global alias map (a2AFullSUG), a per-identity quality classification (A2clsFull), a within-project resolution table (P2aAFull), and a commit-to-identity provenance table (c2AFull). The central methodological contribution is the splitting/clumping distinction: the paper argues that prior global-scale union maps report high precision only because their evaluation never reaches the over-merged (clumped) region, and demonstrates that the prior WoC V3 map's 0.95 precision collapses to 0.52 once its 3M-id mega-cluster is counted. The released map is mega-cluster-free (largest cluster 6,910 ids), scoring recall 0.70 / precision 0.88 on the ALFAA human gold set. A six-stage pipeline (link generation, value gating, structural gating via betweenness cut, edge classification, recall recovery via shingle expansion, representative selection) is described, with design choices traced to an eighteen-experiment record in a companion methodology paper. Downstream impact is quantified via an ablation ladder (L0/L1/L2) showing that under-merge and over-merge distort developer analytics in opposite directions across head-count, productivity, bus factor, collaboration networks, and centrality. A cross-corpus linking pilot to Semantic Scholar via DOI anchoring is presented as a broader-imp","tokens_in":18351,"tokens_out":1300,"duration_ms":341019,"significance":"The release of a mega-cluster-free, per-commit-provenance-tagged identity map over the entire WoC corpus is a significant community resource. The splitting/clumping distinction (§5) is a genuinely useful conceptual contribution: the argument that pairwise precision on curated samples can hide quadratic-mass clumping is well-framed and the demonstration that V3's 0.95 precision collapses to 0.52 when the mega-cluster is counted is compelling. The ablation ladder (§6) cleanly traces identity errors into downstream analytics, and the finding that 36.3% of apparently multi-developer projects are one person under multiple aliases is striking. The per-commit provenance tags (c2AFull) that let consumers choose their own precision/recall operating point are a practical design strength. The DOI-anchored cross-corpus linking pilot (§7), while small (7,246 authors), demonstrates a falsifiable approach to the clumping problem across corpora. The replication package and co-versioned artifacts are appropriate for a data-description paper.","major_comments":[{"comment":"§5, Table 4: The paper's central methodological argument is that pairwise precision can hide clumping (§5, p.5: 'sampling pairs in a way that under-weights a single giant cluster whose false-pair mass grows quadratically in its size'). However, the released map's own precision claim of 0.88 is also pairwise, evaluated on 469k ALFAA pairs from one community. The paper supplements this with the largest-cluster diagnostic (6,910), but this is a single data point. The paper does not report: (a) the full cluster-size distribution below 6,910 (only that the >10k bin is empty); (b) cluster-level purity (what fraction of clusters contain exactly one real person); (c) how many clusters exist in the 100–6,910 range and how many ids they collectively cover. If there are hundreds of clusters in the 100–1,000 range with false merges, the pairwise precision of 0.88 could overstate true cluster-level精度","section":null}],"minor_comments":[{"comment":"§3, stage 5: The two shingle-expansion thresholds (v1 at τ=0.9, v2.1 at τ=0.8) are stated without explaining how they were selected. A brief note on the selection criterion would help readers.","section":null},{"comment":"§5: The paper mentions '~26 mislabels' in the ALFAA gold set and adjusts precision to ~0.90, but does not describe how these mislabels were identified. A one-sentence methodology would suffice.","section":null},{"comment":"§6.3, Tables 9–10: The fan-out cap of 1,000 co-developers per project is mentioned but its sensitivity is not discussed. How many projects are truncated, and does the 32% edge-removal figure change materially with a different cap?","section":null},{"comment":"§7: The DOI-anchored pilot links 7,246 authors, which is small relative to the 62.67M canonical identities. The paper is transparent about this coverage limitation, but the heading 'Broader Impact' may set expectations the pilot cannot meet; consider softening.","section":null},{"comment":"Table 2: The 'partial' class (0.23%) is defined as 'otherwise-good but carries only one of name/email,' but it is unclear whether these ids are merged globally or treated like local/bad-by-attribute ids.","section":null},{"comment":"§4, Table 3: Several entries reference experiments by number (e.g., 'Exp. 14') from the companion methodology paper [13], which is listed as a 2026 reference with no venue. Since the full experimental record is not available to the reader, the table's justifications are partially unverifiable. Consider noting availability.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The paper is a data-description paper releasing a large-scale artifact, and the bar for acceptance should reflect that. The splitting/clumping distinction and the ablation ladder are solid contributions. The main concern (cluster-level purity not reported) is real but addressable within revision: the authors likely have the cluster-size distribution and could add it as a table or figure. I would not hold this to a major-revision bar since the central claim (mega-cluster freedom, honestly reported splitting and clumping) is defensible even without cluster-level purity, but reporting it would substantially strengthen the paper's own argument. The companion methodology paper [13] being unavailable is a minor concern for verification but is standard for a data-description companion."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for the careful and constructive assessment. The recommendation of minor revision is appropriate, and the single major comment identifies a genuine gap in the cluster-size distribution diagnostics that we will close in the revised manuscript. We agree that reporting only the largest-cluster size (6,910) as a clumping diagnostic is insufficient without the supporting distribution context, and we will add the requested statistics. We note one honest limitation: the ALFAA gold set's community scope (OpenStack-centric) means our cluster-level purity estimate is necessarily partial, and we will state this explicitly rather than overclaim.","responses":[{"response":"The referee is correct on all three sub-points. Our central argument—that pairwise precision can hide quadratic-mass clumping—cuts both ways: if our own 0.88 pairwise precision also hides moderate-sized clumped clusters in the 100–6,910 range, then our clumping diagnostic is incomplete. We will address this in the revision by adding three pieces of information to §5. (a) We will report the full cluster-size distribution (histogram of cluster sizes from 2 to 6,910), not just the maximum. (b) We will report cluster-level purity on the subset of clusters that intersect the ALFAA gold set: for each cluster containing at least one gold id, the fraction that map to a single real person. We will be explicit that this is a partial measure, bounded by the OpenStack-centric scope of ALFAA, and cannot certify purity for clusters that the gold set does not touch. (c) We will report the count of clusters in the 100–6,910 range and the total number of ids they collectively cover, so the reader can assess whether the pairwise precision is dominated by small clean clusters or inflated by a few large dirty ones. We expect the distribution to be heavily long-tailed (the vast majority of clusters are size 2–5), but the referee is right that this should be shown, not asserted. We will also add an explicit acknowledgment that the 0.88 pairwise precision shares the limitation we identify in prior work—it is a pairwise average that could, in principle, underweight clumping in moderate clusters—and that the cluster-size distribution and purity statistics are the necessary complement, just as the largest-cluster diagnostic is the necessary complement to V3's pairwise precision. One honest limitation we will state: without a global (non-community-specific) human gold set, cluster-level purity is","revision_made":"no","referee_comment":"§5, Table 4: The paper's central methodological argument is that pairwise precision can hide clumping. However, the released map's own precision claim of 0.88 is also pairwise, evaluated on 469k ALFAA pairs from one community. The paper supplements this with the largest-cluster diagnostic (6,910), but this is a single data point. The paper does not report: (a) the full cluster-size distribution below 6,910; (b) cluster-level purity (what fraction of clusters contain exactly one real person); (c) how many clusters exist in the 100–6,910 range and how many ids they collectively cover. If there are hundreds of clusters in the 100–1,000 range with false merges, the pairwise precision of 0.88 could overstate true cluster-level accuracy."}],"tokens_in":18011,"tokens_out":754,"duration_ms":176383,"standing_objections":["Cluster-level purity cannot be computed globally because no global human-rated ground truth exists. The ALFAA gold set is OpenStack-centric, so purity is estimable only for clusters it intersects. We will report this partial measure and label it as such, but we cannot honestly claim a global purity figure."]},"desk_editor":{"model":"glm-5.2","letter":"Bottom line: this paper ships a mega-cluster-free author-identity map over all 5.87B public git commits, and the splitting/clumping distinction as an evaluation framework is genuinely useful. The artifact is real and the downstream ablation is thorough. The main soft spot is that the 0.88 precision rests on a single OpenStack-centric gold set, and the paper doesn't report enough distribution-level clumping diagnostics to fully close the loop on its own argument about pairwise precision hiding over-merge.","headline":"Mega-cluster-free author-identity map at 10^8 scale is a real artifact release; the splitting/clumping framing is the methodological contribution. The precision claim is narrower than it looks.","tokens_in":18987,"tokens_out":184,"would_cite":true,"duration_ms":91391,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"62.7M Developer Identities Merged from 106.8M Strings Over 5.87B Commits","keywords":[],"falsifier":"If the largest cluster in the released map were shown to exceed 10,000 ids, or if a broader human gold set (beyond OpenStack) produced precision below 0.70, the central claim of mega-cluster-free calibrated resolution would be undermined.","tokens_in":18306,"feed_emoji":"🔗","tokens_out":823,"duration_ms":187122,"temperature":0.7,"pith_summary":"This paper releases a curated author-identity map for the entire World of Code V2604 corpus — 5,866,595,698 git commits authored under 106,826,059 distinct name/email strings. The core problem the paper addresses is that at hundred-million-id scale, the naive approach to merging aliases (taking the transitive closure over all shared-attribute edges) welds roughly three million unrelated people into a single mega-cluster, silently corrupting every per-author aggregate computed on top. The released map avoids this by a six-stage pipeline that gates bad attributes, cuts structural bridges in the union graph, classifies candidate edges with a logistic model trained on 2.6 million free GitHub labels, and then recovers recall through a classifier-filtered shingle expansion. The result is a mega-cluster-free map whose largest cluster is 6,910 ids, resolving 73.5% of all commits into multi-id identities and 98.17% of human commits. Against the ALFAA human-rated gold set it scores recall 0.70 and precision 0.88, while the prior WoC map's apparent precision of 0.95 collapses to 0.52 once its 3,006,318-id mega-cluster is counted. The paper argues that the dominant evaluation failure in large-scale author disambiguation is measuring only splitting (recall) while leaving clumping (over-merge) untested, and that high precision claims in prior global-scale union maps are an artifact of never measuring the conflated region. It then demonstrates that the choice of identity method shifts downstream analytics in concrete, opposite directions: under-aliasing inflates developer head-count by 66.6% and invents 10.1M phantom one-commit developers, while over-aliasing welds strangers into giant phantom identities. Both errors distort bus-factor estimates, collaboration-network density, and centrality rankings in opposite directions.","feed_headline":"62.7M Developer Identities Merged from 106.8M Strings Over 5.87B Commits","feed_subtitle":"A mega-cluster-free author map shows prior precision claims collapse once over-merged clusters are counted, shifting every downstream metric","key_machinery":"The six-stage pipeline (gate values, cut topology, classify edges, recover recall, resolve locally) and the splitting-vs-clumping evaluation framework that grades on two independent ground truths read jointly.","core_discovery":"The central discovery is that at 10^8-id scale, author de-aliasing has two failure families — splitting (one person scattered across aliases) and clumping (strangers merged into one identity) — and they must be solved in that order, because applying recall mechanisms to an over-merged graph only worsens the mega-cluster. The paper shows that value-based gating alone cannot dissolve the mega-cluster (neutralizing every detectable bad attribute still leaves it 94% intact), because a redundant mesh of moderately-bad values re-closes around each removed weld. What works is changing the unit of analysis: first from nodes to topology (a sampled-betweenness cut that identifies roughly 2,000 load-1,","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["New mega-cluster-free map resolves 62.7M developer identities","62.7M dev identities mapped from 106.8M raw author strings","Prior author de-aliasing precision collapses under mega-cluster counts","De-aliasing 62.7M devs: splitting must be solved before clumping","Topology, not value gating, breaks massive id clusters in code commits"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The precision claim of 0.88 rests on the ALFAA gold set, which is drawn from a single community (OpenStack). If that community's naming patterns are not representative of the global developer population, the measured precision may not generalize to the full 106.8M-id space.","fun_headline_variants_meta":{"raw":{"variants":["New mega-cluster-free map resolves 62.7M developer identities","62.7M dev identities mapped from 106.8M raw author strings","Prior author de-aliasing precision collapses under mega-cluster counts","De-aliasing 62.7M devs: splitting must be solved before clumping","Topology, not value gating, breaks massive id clusters in code commits"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":1354,"prompt_tokens":750,"completion_tokens":604,"prompt_tokens_details":null},"tokens_in":750,"tokens_out":604,"duration_ms":49174,"temperature":1.0,"reasoning_tokens":500,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T14:00:36.510716+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the largest cluster in the released map were shown to exceed 10,000 ids, or if a broader human gold set (beyond OpenStack) produced precision below 0.70, the central claim of mega-cluster-free calibrated resolution would be undermined.","supporting_citations":[],"review_version":1}