{"id":"fda482ab-e305-45f8-b8f2-c09bb7aa0762","arxiv_id":"2607.06194","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"A scan of 5.8 billion git commits finds 17.59% carry cryptographic signatures, enabling a key-to-identity graph and trust tiers that calibrate heuristic author disambiguation.","lead":"This paper extracts cryptographic commit signatures from 5.8 billion git commits in the World of Code corpus and uses them to build trust tiers for developer identity. A smart generalist might read it to understand how we can distinguish real developers from impostors or bots in open-source software at scale.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"T2 tier is labeled 'real-world-bound' but the paper's own sampled verification shows only ~10% of person-keys can have their real-world binding confirmed, leaving the trust-tier contribution weaker than presented.","rationale":"The reader correctly identified the fanout threshold as a concern, but the paper addresses it through corroboration in §6, making it less load-bearing than it appears. The more significant soft spot is the T2 tier: the paper defines T2 as providing 'real-world binding' but the assignment appears to be based on person-key status alone, with only ~10% of sampled keys having verifiable real-world bindings. The paper is transparent about this ('presence and issuer-key-id consistency remain the corpus signal'), which is why I do not recommend changing the verdict. The core dataset contribution — extracting signatures at scale, building the key-to-identity graph, and releasing the artifacts — stands regardless of the T2 labeling issue. The trust-tier system is still useful as a relative confidence axis, but the T2 label overstates what is actually verified. This is a labeling/precision concern, not a soundness defect. The paper's honesty about limitations (selection bias, keyserver coverage, corroboration requirements) is commendable and supports keeping the ACCEPT verdict, though confidence might more appropriately be MEDIUM rather than HIGH given the T2 gap.","tokens_in":9628,"tokens_out":5660,"duration_ms":352066,"concrete_test":"Clarify the T2 assignment criterion: is T2 assigned to every identity with a person key (fanout < 50), or only to those whose real-world binding (PGP user-id email, SSH key on GitHub, X.509 subject) has been independently verified? If the former, recompute Table 4 restricting T2 to verified bindings only (using the keyserver-retrievable subset or SSH/GitHub API checks); if the T2 count drops by more than 50%, the 'real-world-bound' label for T2 should be qualified or the tier split into T2a (claimed) and T2b (verified).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper defines Tier T2 as adding 'a real-world binding: a PGP user-id email, an SSH key registered on a GitHub account, or an X.509/OIDC subject' (§7). However, the key handle extracted from the signature packet (issuer key-id/fingerprint, §5) does not itself carry the real-world identity — that requires fetching the full public key. The sampled verification in §8 reveals the gap: of 398 randomly sampled person-keys, only 78 (19.6%) were retrievable from the Ubuntu keyserver, and of those, only 39 (50%) had a user-id email matching the signed author string. This yields an effective verification rate of roughly 10%. The paper states that 'presence and issuer-key-id consistency remain the corpus signal,' which indicates T2 is assigned based on person-key status (fanout < 50) rather than on verified real-world binding. If so, the 465,398 T2 authors (Table 4) are labeled 'real-world-bound' without that binding being substantiated for the vast majority. The paper is transparent about this limitation but does not reconcile the T2 tier definition ('real-world binding') with the assignment criterion (person-key gate). This matters because the trust-tier system is a central contribution, and downstream consumers relying on T2 as a 'real-world binding' may over-trust it. The reader's concern about the fanout threshold (§5) is reasonable but less load-bearing: the paper handles the contamination at low fanout with corroboration in §6, showing that even at fanout 2, 71.9% of co-signed pairs are different people, and filtering to 9,036 confirmed edges. The T2 verification gap is the softer spot.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper releases a commit-signature dataset for the World of Code (WoC) V2604 collection, comprising 1.03 billion signed commits (17.59% of the corpus). The extraction is performed by scanning existing commit-table shards for gpgsig headers, avoiding a re-read of the object database. The paper constructs a key-to-identity graph gated by key fanout to separate person keys from shared organization/CI keys, builds a cryptographic alias gold from co-signed author strings to calibrate the heuristic WoC alias map, defines a four-tier identity trust system (A2trust: T0-T3), and applies the attestation chain to cross-corpus science-to-software identity links. All artifacts are released as a self-contained replication package.","tokens_in":10444,"tokens_out":1096,"duration_ms":222642,"significance":"The dataset is a genuine and useful contribution: it is the first commit-signature axis extracted at WoC scale, and the observation that the gpgsig header already survives in the commit-table message field makes the extraction reproducible without object-database access. The shard stability check (17.58%, 17.53%, 17.65%) supports the prevalence claim. The cryptographic alias gold and its use as an independent calibration benchmark for the heuristic alias map is well-motivated. The artifacts (c2sigFull, key2A/A2key, A2trust) are released under CC-BY-4.0 with replication code under MIT, and the falsifiable impersonation test on vanity strings is a concrete, testable contribution.","major_comments":[{"comment":"§7 and Table 4: There is a tension between the T2 tier definition and its assignment criterion. T2 is defined as adding 'a real-world binding: a PGP user-id email, an SSH key registered on a GitHub account, or an X.509/OIDC subject.' However, the sampled verification in §8 shows that of 398 person-keys, only 78 (19.6%) were retrievable from the keyserver, and of those only 39 (50%) had a matching user-id email — an effective verification rate of roughly 10%. The paper states that 'presence and issuer-key-id consistency remain the corpus signal,' which indicates T2 is assigned based on the person-key gate (fanout < 50) rather than on verified real-world binding. The 465,398 T2 authors are thus labeled 'real-world-bound' without that binding being substantiated for the vast majority. The paper is transparent about this limitation but does not reconcile the tier definition with the actual T","section":null},{"comment":"§5: The key-fanout gate threshold of 50 distinct authors is a post-hoc choice that directly affects the alias gold and the T2 tier assignment. The paper shows that two shared keys carry 95.9% of associations and that 2,651 shared keys are dropped, but provides no sensitivity analysis around the threshold. A threshold of 10 or 100 would change the person-key count and the downstream alias gold. The paper should either justify the threshold quantitatively (e.g., showing a knee in the fanout distribution) or report results at multiple thresholds to demonstrate that the calibration conclusions in Table 3 and the T2 counts in Table 4 are not artifacts of this specific cutoff.","section":null}],"minor_comments":[{"comment":"§9: The Hugging Face dataset URL is listed as 'TODO/woc-commit-signatures-v2604' and the DOI is 'to be minted on camera-ready.' These must be finalized before publication.","section":null},{"comment":"Table 2: The text notes that distinct-signer counts are 'drawn from one representative shard' but does not specify which shard or how 'representative' was determined.","section":null},{"comment":"§4: The statement that the strict header marker and loose family banner 'agree to within 0.01 percentage points' is interesting but the exact counts of disagreements are not given; a brief footnote with the numbers would strengthen the claim.","section":null},{"comment":"§8: The text mentions '10,143 same_as edges' but Table 6 reports '10,021 resolved cross-corpus same_as edges.' The difference of 122 should be explained (e.g., 122 edges did not join the trust universe).","section":null},{"comment":"Table 3 caption: 'Pair-recall' is defined in the caption but the column header reads 'map unifies'; aligning the terminology would help.","section":null},{"comment":"§3: The paper mentions 'SSH-pubkey and X.509-subject handle families are the remaining extension' for the key graph; clarifying whether these are planned or partially implemented would help users of the released artifacts.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The paper is a single-author data showcase by the WoC PI, which is appropriate for a data-track submission. The core extraction is sound and the artifacts are substantial. The main concern (T2 labeling) is a presentation/definition issue rather than a data error — the underlying person-key gate is reasonable, but the 'real-world-bound' label overstates what is verified. This is fixable by relabeling or adding a sub-tier. I lean toward minor revision rather than major revision because the central dataset contribution does not depend on the T2 label being perfect."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for the careful reading and the constructive recommendation. Both major comments identify genuine gaps that we will address in revision: the T2 tier definition vs. assignment criterion tension, and the lack of sensitivity analysis around the fanout-50 threshold. We agree with both points and will revise accordingly.","responses":[{"response":"The referee is correct that there is a genuine tension between the T2 definition as stated and the criterion actually used to assign it. We will revise the manuscript to reconcile this. Specifically, we plan to: (1) rename or subdivide the T2 tier to distinguish 'person-key attested' (the corpus-derivable signal: a key with fanout < 50 that consistently signs for one identity) from 'real-world verified' (the keyserver-confirmed binding), making clear that the 465,398 count reflects the former, not the latter; (2) add explicit language in §7 stating that T2 as assigned is a necessary-but-not-sufficient condition for real-world binding — the person-key gate establishes that a single keyholder controls the identity, but the key-to-real-world-name step is only sampled, not exhaustively verified; and (3) cross-reference the §8 sampled-verification results directly in the T2 definition so the reader sees the effective verification rate (roughly 10% of sampled person-keys) alongside the tier assignment. We agree the current wording overstates what T2 substantiates for the majority of assigned identities, and the revision should close that gap.","revision_made":"yes","referee_comment":"§7 and Table 4: Tension between T2 tier definition and assignment criterion. T2 is defined as adding a real-world binding (PGP user-id email, SSH key on GitHub account, X.509/OIDC subject), but §8 shows only ~10% effective verification rate from keyserver sampling. T2 is actually assigned based on the person-key gate (fanout < 50), not on verified real-world binding. The 465,398 T2 authors are labeled 'real-world-bound' without that binding being substantiated for the vast majority."},{"response":"This is a fair point and we will add a sensitivity analysis. We will report person-key counts, alias-gold pair counts, Table 3 recall figures, and Table 4 T2 author counts at thresholds of 10, 25, 50, 100, and 200. We expect the calibration conclusions to be robust because the fanout distribution is extremely skewed — two platform keys account for 95.9% of all associations, and the shared-key tail drops off sharply — so the exact cutoff in the 10–100 range should not materially change the person-key set or the alias gold. But we should show this rather than assert it, and the referee is right that the manuscript currently does not. We will add a table or figure showing the key fanout distribution (the 'knee' the referee suggests) and the downstream counts at each threshold, so the reader can verify that the calibration results are not artifacts of the specific cutoff.","revision_made":"yes","referee_comment":"§5: The key-fanout gate threshold of 50 is a post-hoc choice affecting the alias gold and T2 tier. No sensitivity analysis provided. A threshold of 10 or 100 would change person-key count and downstream results. Should justify quantitatively or report results at multiple thresholds to show calibration conclusions and T2 counts are not artifacts of this cutoff."}],"tokens_in":9502,"tokens_out":730,"duration_ms":174662,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The paper extracts cryptographic commit signatures from the existing WoC V2604 commit tables — no object-database re-read needed — and releases a per-commit signature map, a gated key-to-identity graph, and a per-identity trust tier. That is a genuine dataset contribution. The extraction method is clean: the gpgsig header survives into the message field as a byproduct of how WoC stores commits, so the whole thing is a scan over existing tables. The shard stability check (17.58%, 17.53%, 17.65%) is convincing. The alias-gold construction — co-signed author strings as same-person evidence — is a real idea, and the triage in Section 6 is honest about the fact that 71.9% of fanout-2 co-signed pairs are different people sharing a key, not missed merges. That candor matters and is to the author's credit. The fanout gate threshold of 50 is a post-hoc choice, but the paper shows the concentration is extreme enough (two keys carry 95.9% of associations) that the exact threshold barely matters. The reader flagged this as the weakest assumption; I think it is less load-bearing than it looks. The stress-test concern about T2 is the real soft spot. Section 7 defines T2 as adding a real-world binding (PGP user-id email, GitHub-registered SSH key, X.509/OIDC subject). But Section 8's sampled verification shows only 19.6% of person-keys were retrievable from the keyserver, and of those, 50% had a matching user-id email — roughly 10% effective verification. The paper says 'presence and issuer-key-id consistency remain the corpus signal,' which means T2 is actually assigned by the fanout gate, not by verified real-world binding. The 465,398 T2 authors in Table 4 are labeled 'real-world-bound' without that binding being substantiated for the vast majority. The paper is transparent about this gap but does not reconcile the tier definition with the assignment criterion. Downstream consumers who read 'T2 = real-world-bound' will over-trust it. This is fixable: either relabel T2 as 'person-key attested' (which is what it actually measures) or gate T2 on successful keyserver retrieval. The core dataset contribution — the signature map, the prevalence numbers, the alias gold — stands on its own regardless of the tier labeling issue. This paper is for the MSR community and anyone doing identity resolution over large commit corpora. It deserves a serious referee who should push on the T2 definition and ask for the DOI to be finalized, but the dataset release is real and useful.","headline":"First large-scale commit-signature dataset for WoC; trust tiers are useful but T2 labeling overstates what is verified","tokens_in":10643,"tokens_out":624,"would_cite":true,"duration_ms":85369,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"1 billion signed commits yield a cryptographic trust layer for developer identity","keywords":[],"falsifier":"If a key shared by a small team (say 10–30 developers) passes the fanout gate as a person key, its co-signed author strings would be falsely treated as same-person evidence, inflating recall gaps in the heuristic alias map and degrading the alias gold's precision.","tokens_in":9734,"feed_emoji":"🔑","tokens_out":1426,"duration_ms":132252,"temperature":0.7,"pith_summary":"The author argues that the author string in a git commit is an unverified claim, and that the cryptographic signature already embedded in 17.59% of all commits in the World of Code V2604 corpus — over one billion signed commits — can be extracted at scale to build a per-commit signature map, a gated key-to-identity graph, and a per-identity attestation tier system (A2trust). The central mechanism is the observation that git's gpgsig header survives unparsed into the WoC commit-table message field, so the signature axis is a scan over existing tables rather than a re-read of the object database. The paper then uses person keys (keys that sign for a small number of distinct authors, after filtering out shared platform and CI keys via a fanout gate) to construct a cryptographically grounded alias gold standard — pairs of author strings co-signed by the same key are the same person, with a guarantee that does not depend on name or email similarity. This gold standard calibrates the existing heuristic alias map independently of hand-labeled pairs. The author also assigns each of 62.6 million canonical authors to one of four trust tiers (unsigned, platform-signed, person-key attested, cross-corpus attested) and shows that the signing population is about 14x cleaner than the corpus baseline. The paper frames signatures as a precision anchor, not a coverage layer: signed commits skew toward recent and security-conscious developers, which overlaps the population that bibliography joins target but does not represent the corpus as a whole.","feed_headline":"1 billion signed commits yield a cryptographic trust layer for developer identity","feed_subtitle":"Author strings in git are unverified claims; this paper extracts the signatures already sitting in commit tables to build a four-tier attest","key_machinery":"The central objects are: (1) c2sigFull, a per-commit signature map labeling each of 1.03 billion signed commits with its signature family (PGP 98.96%, SSH 1.02%, X.509/sigstore 0.02%); (2) key2A/A2key, a key-to-author graph gated by a fanout threshold of 50 distinct authors to separate person keys from shared keys; (3) A2trust, a per-identity attestation tier (T0 unsigned, T1 platform-signed, T2 person-key attested, T3 cross-corpus attested) extending the published A2cls identity-class dataset; and (4) a cryptographic alias gold built from author strings co-signed by the same person key, used to measure precision and recall of the heuristic WoC alias map.","core_discovery":"The paper's central discovery is that commit signatures — already present but unparsed in the WoC V2604 commit tables — can be extracted at corpus scale to produce a cryptographic identity-attestation layer. The key-to-identity graph, gated by a fanout threshold of 50 distinct authors per key, separates 583,360 person keys from 2,651 shared organization/CI keys (which account for 97.6% of all author–key associations despite being 0.45% of keys). Person keys that co-sign multiple author strings yield 392,991 candidate same-person merges that the heuristic alias map holds apart, providing an independently grounded alias gold standard larger than the 469k hand-labeled pairs used previously, and","pith_inferences":["If the fanout threshold of 50 is too high, person keys shared by small teams (5–49 developers) would survive the gate and inject false same-identity evidence into the alias gold; if too low, legitimate individuals who sign under many email addresses or across organizational contexts could be dropped. The paper does not report sensitivity analysis around this threshold.","The 50% T1 rate (platform-signed only) is dominated by GitHub's web-flow signing key, which signs on behalf of users who never personally configured signing. This means the author-level signed rate overstates actual personal cryptographic commitment, and the meaningful attestation rate is the 0.74% at T2.","The sampled verification pass found that only 19.6% of person keys were retrievable from the Ubuntu keyserver, and of those, 50% carried a matching user-id email. This suggests that full real-world binding (true T2) is rarer than the corpus-level T2 count implies, and the T1-to-T2 boundary may carry a higher unproven rate than the paper's framing suggests.","If SSH and sigstore signing continue to grow (as adoption curves suggest), the signature axis could shift from a precision anchor toward a broader coverage layer over time, which would change the selection-bias profile the paper documents."],"forward_implications":["Any WoC-scale study can join the per-commit signature map by commit SHA and the per-identity trust tier by author key, adding a cryptographic confidence axis to identity resolution without changing the existing heuristic pipeline.","The 9,036 corroborated recall-repair edges (distinct author strings co-signed by one person key and sharing an email local-part, domain, or name token) are high-precision candidates to feed back into the alias map, potentially closing genuine same-person gaps the heuristic missed.","The key-dispersion measure (distinct signing keys per author string) provides an evidence-based replacement for the hand-curated bad-identity stoplist: strings signed by 20+ keys are flagged as vanity or impersonated identities from signature evidence alone.","The bibliography attestation bridge upgrades 1,207 of 10,021 science-to-software identity edges from name-match heuristics to cryptographic anchors, giving a calibration seed for name-based edge growth in cross-corpus author linking."],"fun_headline_variants":["Extracting 1 billion commit signatures to map developer identity trust","Commit signatures reveal a cryptographic trust tier for developer identity","Signed commits expose 393k alias merges hidden from heuristic maps","Separating person keys from CI keys to build a commit trust layer","Mining 1 billion signed commits for cryptographic identity attestation"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The fanout gate assumes that 50 distinct authors per key cleanly separates individual person keys from shared organization or CI keys. If this threshold is miscalibrated, keys shared by small teams could pass as person keys and inject false same-identity evidence into the cryptographic alias gold, corrupting the calibration of the heuristic alias map.","fun_headline_variants_meta":{"raw":{"variants":["Extracting 1 billion commit signatures to map developer identity trust","Commit signatures reveal a cryptographic trust tier for developer identity","Signed commits expose 393k alias merges hidden from heuristic maps","Separating person keys from CI keys to build a commit trust layer","Mining 1 billion signed commits for cryptographic identity attestation","Cryptographic signatures provide a scalable alias gold standard","From unverified claims to attested identities: mining commit signatures","Commit signatures as a precision anchor for developer identity","Gating shared keys reveals 583k person keys behind signed commits","A cryptographic attestation layer built from 1 billion signed commits"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":1436,"prompt_tokens":722,"completion_tokens":714,"prompt_tokens_details":null},"tokens_in":722,"tokens_out":714,"duration_ms":56828,"temperature":1.0,"reasoning_tokens":639,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T13:36:34.343237+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If a key shared by a small team (say 10–30 developers) passes the fanout gate as a person key, its co-signed author strings would be falsely treated as same-person evidence, inflating recall gaps in the heuristic alias map and degrading the alias gold's precision.","supporting_citations":[],"review_version":1}