Pith. sign in

REVIEW 2 major objections 6 minor 10 references

Claimed or Attested? A Commit-Signature Dataset and Identity Trust Tiers across the World of Code

T0 review · 2 major / 6 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read 1 billion signed commits yield a cryptographic trust layer for developer identity

desk verdict First large-scale commit-signature dataset for WoC; trust tiers are useful but T2 labeling overstates what is verified read the letter →

arxiv 2607.06194 v1 pith:LAUHOTID submitted 2026-07-07 cs.CR cs.SE

Audris Mockus This is my paper · ORCID
classification cs.CRcs.SE
keywords commitidentitysignaturesignedattestedaxiscommitskeys
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The author argues that the author string in a git commit is an unverified claim, and that the cryptographic signature already embedded in 17.59% of all commits in the World of Code V2604 corpus — over one billion signed commits — can be extracted at scale to build a per-commit signature map, a gated key-to-identity graph, and a per-identity attestation tier system (A2trust). The central mechanism is the observation that git's gpgsig header survives unparsed into the WoC commit-table message field, so the signature axis is a scan over existing tables rather than a re-read of the object database. The paper then uses person keys (keys that sign for a small number of distinct authors, after filtering out shared platform and CI keys via a fanout gate) to construct a cryptographically grounded alias gold standard — pairs of author strings co-signed by the same key are the same person, with a guarantee that does not depend on name or email similarity. This gold standard calibrates the existing heuristic alias map independently of hand-labeled pairs. The author also assigns each of 62.6 million canonical authors to one of four trust tiers (unsigned, platform-signed, person-key attested, cross-corpus attested) and shows that the signing population is about 14x cleaner than the corpus baseline. The paper frames signatures as a precision anchor, not a coverage layer: signed commits skew toward recent and security-conscious developers, which overlaps the population that bibliography joins target but does not represent the corpus as a whole.

What carries the argument

The central objects are: (1) c2sigFull, a per-commit signature map labeling each of 1.03 billion signed commits with its signature family (PGP 98.96%, SSH 1.02%, X.509/sigstore 0.02%); (2) key2A/A2key, a key-to-author graph gated by a fanout threshold of 50 distinct authors to separate person keys from shared keys; (3) A2trust, a per-identity attestation tier (T0 unsigned, T1 platform-signed, T2 person-key attested, T3 cross-corpus attested) extending the published A2cls identity-class dataset; and (4) a cryptographic alias gold built from author strings co-signed by the same person key, used to measure precision and recall of the heuristic WoC alias map.

What would settle it

If a key shared by a small team (say 10–30 developers) passes the fanout gate as a person key, its co-signed author strings would be falsely treated as same-person evidence, inflating recall gaps in the heuristic alias map and degrading the alias gold's precision.

Watch

Extended reading notes

Core claim

The paper's central discovery is that commit signatures — already present but unparsed in the WoC V2604 commit tables — can be extracted at corpus scale to produce a cryptographic identity-attestation layer. The key-to-identity graph, gated by a fanout threshold of 50 distinct authors per key, separates 583,360 person keys from 2,651 shared organization/CI keys (which account for 97.6% of all author–key associations despite being 0.45% of keys). Person keys that co-sign multiple author strings yield 392,991 candidate same-person merges that the heuristic alias map holds apart, providing an independently grounded alias gold standard larger than the 469k hand-labeled pairs used previously, and

Load-bearing premise

The fanout gate assumes that 50 distinct authors per key cleanly separates individual person keys from shared organization or CI keys. If this threshold is miscalibrated, keys shared by small teams could pass as person keys and inject false same-identity evidence into the cryptographic alias gold, corrupting the calibration of the heuristic alias map.

Editorial extensions

If this is right

  • Any WoC-scale study can join the per-commit signature map by commit SHA and the per-identity trust tier by author key, adding a cryptographic confidence axis to identity resolution without changing the existing heuristic pipeline.
  • The 9,036 corroborated recall-repair edges (distinct author strings co-signed by one person key and sharing an email local-part, domain, or name token) are high-precision candidates to feed back into the alias map, potentially closing genuine same-person gaps the heuristic missed.
  • The key-dispersion measure (distinct signing keys per author string) provides an evidence-based replacement for the hand-curated bad-identity stoplist: strings signed by 20+ keys are flagged as vanity or impersonated identities from signature evidence alone.
  • The bibliography attestation bridge upgrades 1,207 of 10,021 science-to-software identity edges from name-match heuristics to cryptographic anchors, giving a calibration seed for name-based edge growth in cross-corpus author linking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the fanout threshold of 50 is too high, person keys shared by small teams (5–49 developers) would survive the gate and inject false same-identity evidence into the alias gold; if too low, legitimate individuals who sign under many email addresses or across organizational contexts could be dropped. The paper does not report sensitivity analysis around this threshold.
  • The 50% T1 rate (platform-signed only) is dominated by GitHub's web-flow signing key, which signs on behalf of users who never personally configured signing. This means the author-level signed rate overstates actual personal cryptographic commitment, and the meaningful attestation rate is the 0.74% at T2.
  • The sampled verification pass found that only 19.6% of person keys were retrievable from the Ubuntu keyserver, and of those, 50% carried a matching user-id email. This suggests that full real-world binding (true T2) is rarer than the corpus-level T2 count implies, and the T1-to-T2 boundary may carry a higher unproven rate than the paper's framing suggests.
  • If SSH and sigstore signing continue to grow (as adoption curves suggest), the signature axis could shift from a precision anchor toward a broader coverage layer over time, which would change the selection-bias profile the paper documents.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This paper releases a commit-signature dataset for the World of Code (WoC) V2604 collection, comprising 1.03 billion signed commits (17.59% of the corpus). The extraction is performed by scanning existing commit-table shards for gpgsig headers, avoiding a re-read of the object database. The paper constructs a key-to-identity graph gated by key fanout to separate person keys from shared organization/CI keys, builds a cryptographic alias gold from co-signed author strings to calibrate the heuristic WoC alias map, defines a four-tier identity trust system (A2trust: T0-T3), and applies the attestation chain to cross-corpus science-to-software identity links. All artifacts are released as a self-contained replication package.

Significance. The dataset is a genuine and useful contribution: it is the first commit-signature axis extracted at WoC scale, and the observation that the gpgsig header already survives in the commit-table message field makes the extraction reproducible without object-database access. The shard stability check (17.58%, 17.53%, 17.65%) supports the prevalence claim. The cryptographic alias gold and its use as an independent calibration benchmark for the heuristic alias map is well-motivated. The artifacts (c2sigFull, key2A/A2key, A2trust) are released under CC-BY-4.0 with replication code under MIT, and the falsifiable impersonation test on vanity strings is a concrete, testable contribution.

major comments (2)
  1. §7 and Table 4: There is a tension between the T2 tier definition and its assignment criterion. T2 is defined as adding 'a real-world binding: a PGP user-id email, an SSH key registered on a GitHub account, or an X.509/OIDC subject.' However, the sampled verification in §8 shows that of 398 person-keys, only 78 (19.6%) were retrievable from the keyserver, and of those only 39 (50%) had a matching user-id email — an effective verification rate of roughly 10%. The paper states that 'presence and issuer-key-id consistency remain the corpus signal,' which indicates T2 is assigned based on the person-key gate (fanout < 50) rather than on verified real-world binding. The 465,398 T2 authors are thus labeled 'real-world-bound' without that binding being substantiated for the vast majority. The paper is transparent about this limitation but does not reconcile the tier definition with the actual T
  2. §5: The key-fanout gate threshold of 50 distinct authors is a post-hoc choice that directly affects the alias gold and the T2 tier assignment. The paper shows that two shared keys carry 95.9% of associations and that 2,651 shared keys are dropped, but provides no sensitivity analysis around the threshold. A threshold of 10 or 100 would change the person-key count and the downstream alias gold. The paper should either justify the threshold quantitatively (e.g., showing a knee in the fanout distribution) or report results at multiple thresholds to demonstrate that the calibration conclusions in Table 3 and the T2 counts in Table 4 are not artifacts of this specific cutoff.
minor comments (6)
  1. §9: The Hugging Face dataset URL is listed as 'TODO/woc-commit-signatures-v2604' and the DOI is 'to be minted on camera-ready.' These must be finalized before publication.
  2. Table 2: The text notes that distinct-signer counts are 'drawn from one representative shard' but does not specify which shard or how 'representative' was determined.
  3. §4: The statement that the strict header marker and loose family banner 'agree to within 0.01 percentage points' is interesting but the exact counts of disagreements are not given; a brief footnote with the numbers would strengthen the claim.
  4. §8: The text mentions '10,143 same_as edges' but Table 6 reports '10,021 resolved cross-corpus same_as edges.' The difference of 122 should be explained (e.g., 122 edges did not join the trust universe).
  5. Table 3 caption: 'Pair-recall' is defined in the caption but the column header reads 'map unifies'; aligning the terminology would help.
  6. §3: The paper mentions 'SSH-pubkey and X.509-subject handle families are the remaining extension' for the key graph; clarifying whether these are planned or partially implemented would help users of the released artifacts.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and the constructive recommendation. Both major comments identify genuine gaps that we will address in revision: the T2 tier definition vs. assignment criterion tension, and the lack of sensitivity analysis around the fanout-50 threshold. We agree with both points and will revise accordingly.

read point-by-point responses

  1. Referee: §7 and Table 4: Tension between T2 tier definition and assignment criterion. T2 is defined as adding a real-world binding (PGP user-id email, SSH key on GitHub account, X.509/OIDC subject), but §8 shows only ~10% effective verification rate from keyserver sampling. T2 is actually assigned based on the person-key gate (fanout < 50), not on verified real-world binding. The 465,398 T2 authors are labeled 'real-world-bound' without that binding being substantiated for the vast majority.

    Authors: The referee is correct that there is a genuine tension between the T2 definition as stated and the criterion actually used to assign it. We will revise the manuscript to reconcile this. Specifically, we plan to: (1) rename or subdivide the T2 tier to distinguish 'person-key attested' (the corpus-derivable signal: a key with fanout < 50 that consistently signs for one identity) from 'real-world verified' (the keyserver-confirmed binding), making clear that the 465,398 count reflects the former, not the latter; (2) add explicit language in §7 stating that T2 as assigned is a necessary-but-not-sufficient condition for real-world binding — the person-key gate establishes that a single keyholder controls the identity, but the key-to-real-world-name step is only sampled, not exhaustively verified; and (3) cross-reference the §8 sampled-verification results directly in the T2 definition so the reader sees the effective verification rate (roughly 10% of sampled person-keys) alongside the tier assignment. We agree the current wording overstates what T2 substantiates for the majority of assigned identities, and the revision should close that gap. revision: yes

  2. Referee: §5: The key-fanout gate threshold of 50 is a post-hoc choice affecting the alias gold and T2 tier. No sensitivity analysis provided. A threshold of 10 or 100 would change person-key count and downstream results. Should justify quantitatively or report results at multiple thresholds to show calibration conclusions and T2 counts are not artifacts of this cutoff.

    Authors: This is a fair point and we will add a sensitivity analysis. We will report person-key counts, alias-gold pair counts, Table 3 recall figures, and Table 4 T2 author counts at thresholds of 10, 25, 50, 100, and 200. We expect the calibration conclusions to be robust because the fanout distribution is extremely skewed — two platform keys account for 95.9% of all associations, and the shared-key tail drops off sharply — so the exact cutoff in the 10–100 range should not materially change the person-key set or the alias gold. But we should show this rather than assert it, and the referee is right that the manuscript currently does not. We will add a table or figure showing the key fanout distribution (the 'knee' the referee suggests) and the downstream counts at each threshold, so the reader can verify that the calibration results are not artifacts of the specific cutoff. revision: yes

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; derivation chain is self-contained against the external WoC corpus, with minor self-citation to prior work that is independently verifiable.

full rationale

The paper's derivation chain proceeds as follows: (1) signatures are extracted from existing WoC commit tables by scanning the message field for gpgsig banners — this is a mechanical scan of an external corpus, not a self-referential construction; (2) the key-to-identity graph uses a fanout threshold of 50 distinct authors to separate person keys from shared/CI keys — this is a heuristic gate, not a parameter fitted to a target and then presented as a prediction; (3) the alias gold is built from pairs of author strings co-signed by the same person key, which is cryptographic evidence independent of the name/email/login heuristics used by the ALFAA alias map [1]; (4) the heuristic alias map is then measured against this gold as precision and recall (Table 3) — the gold and the map use disjoint evidence sources, so the calibration is a genuine comparison, not a tautology; (5) trust tiers T0–T3 are assigned based on signature presence and person-key status, with the T2 'real-world binding' definition acknowledged as only partially verified (§8 sampled verification). The self-citations to ALFAA [1] and WoC [3,4] are to prior published work that serves as the object being calibrated or the infrastructure being extended, not as unverified premises that make the argument circular. The ALFAA map is an external benchmark with its own published validation, and the signature axis provides a genuinely independent cryptographic check on it. The T2 labeling gap (tier defined as 'real-world-bound' but assigned by person-key gate) is a correctness/labeling concern, not circularity — the paper does not define T2 in terms of itself, nor fit a parameter to T2 outcomes and call it a prediction. The minor self-citation load (author is on ALFAA and WoC papers) does not make the central claim circular because the signature-derived gold is constructed from evidence (cryptographic co-signing) that is disjoint from the heuristic map's evidence (name/email/login co-occurrence). Score 2 reflects the presence of self-citation to prior work that is not independently machine-checked or externally falsified outside the present paper, but the derivation remains substantively independent.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The ledger captures the key choices and assumptions. The fanout threshold is the main free parameter. The axioms are domain assumptions about the WoC data structure and PGP key semantics. The invented entities (trust tiers) are evidence-based classifications, not new physical postulates.

free parameters (1)
  • key-fanout gate threshold = 50
    The threshold of 50 distinct authors per key is chosen to separate person keys from shared keys (§5). This is a hand-tuned parameter that affects the composition of the alias gold.
assumptions (3)
  • domain assumption The WoC V2604 commit tables carry the gpgsig header verbatim in the message field.
    The extraction methodology depends on this property of the WoC table generator (§3). The paper states it is validated by the shard stability check.
  • domain assumption A PGP issuer key-id or fingerprint derivable from a signature packet reliably identifies a unique keyholder.
    The key-to-identity graph construction relies on this mapping (§5). The paper notes key rotation and shared keys as confounds but treats low-fanout keys as person anchors.
  • standard math The ALFAA heuristic alias map is a valid baseline for identity resolution.
    The paper uses the WoC alias map as the system under test, calibrated by the signature gold (§6). This is a standard assumption in this research line.
invented entities (1)
  • A2trust tiers (T0-T3) independent evidence
    purpose: To assign a per-identity attestation confidence level.
    The tiers are defined by the presence and type of cryptographic signature, which is independently verifiable from the commit data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Claimed or Attested? A Commit-Signature Dataset and Identity Trust Tiers across the World of Code." pith.science (2026). https://pith.science/paper/LAUHOTID

@misc{pith2026260706194,
  author       = {Pith},
  title        = {Pith review of: Claimed or Attested? A Commit-Signature Dataset and Identity Trust Tiers across the World of Code},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LAUHOTID}},
  note         = {Machine review of arXiv:2607.06194}
}
read the original abstract

An author string in a git commit is free text the committer typed, so identity resolution over a global commit corpus rests on a claim that nothing in the commit verifies. A cryptographically signed commit is different: it binds the commit to a key the committer controls, and when that key ties back to a real-world identity the git identity becomes attested rather than merely claimed. We release the first commit-signature axis for the World of Code (WoC), extracted for the V2604 collection. The signature travels in the commit object's gpgsig header and is already carried, unparsed, in the commit-message field of the WoC commit tables, so the axis is a scan over existing tables rather than a re-read of the object database. Over the V2604 corpus of 5,866,595,698 commits, 17.59% carry a signature (PGP dominant at 98.96%, with a growing minority of SSH and X.509/sigstore signatures), or 1,031,721,316 signed commits. We release the per-commit signature map c2sigFull, a key-to-author graph gated so that shared organization and continuous-integration keys are separated from person keys, and A2trust, a per-identity attestation tier (unsigned, signed, real-world-bound, cross-corpus attested) that extends the published A2cls identity-class dataset. The signature axis is a precision anchor, not a coverage layer: signed commits skew toward recent and security-conscious developers, a population that overlaps the scholarly authors a bibliography join targets. We use the person keys to build a cryptographically grounded alias gold that calibrates the heuristic WoC alias map independently of hand-labeled pairs, and to attach an attestation provenance to science-to-software identity links. All artifacts are released as a self-contained, in dependently hosted replication package keyed to the WoC V2604 collection.

Figures

Figures reproduced from arXiv: 2607.06194 by the authors.

Figure 1
Figure 1. The signature already lives in the commit tables. Git [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages

  1. [1]

    Sadika Amreen, Yuxia Zhang, Chris Bogart, Russell Zaretzki, and Audris Mockus

    work page

  2. [2]

    https://doi.org/10.1007/s10664-019-09786-7 Scaling Author Identity Disambiguation to the World of Code: A Methodology Conference’17, July 2017, Washington, DC, USA

    ALFAA: Active Learning Fingerprint Based Anti-Aliasing for correcting de- veloper identity errors in version control systems.Empirical Software Engineering 25, 2 (2019), 1136–1167. doi:10.1007/s10664-019-09786-7

    work page doi:10.1007/s10664-019-09786-7 2019

  3. [3]

    Thomas Bock, Aakanksha Agarwal, Mitchell Atkisson, Audris Mockus, and Bogdan Vasilescu. 2025. Between Accuracy and Privacy: Rethinking Developer-Identity De-Aliasing in Git Repositories.Empirical Software Engineering(2025). Under submission. Tool: GitAuthority, https://github.com/bockthom/gitauthority

    work page 2025

  4. [4]

    Yuxing Ma, Chris Bogart, Sadika Amreen, Russell Zaretzki, and Audris Mockus

    work page

  5. [5]

    Neural Cleanse: Identifying and mitigating back- door attacks in neural networks,

    World of Code: An Infrastructure for Mining the Universe of Open Source VCS Data. InProceedings of the 16th International Conference on Mining Software Repositories (MSR). 143–154. doi:10.1109/MSR.2019.00031

    work page doi:10.1109/msr.2019.00031 2019

  6. [6]

    Yuxing Ma, Tapajit Dey, Chris Bogart, Sadika Amreen, Marat Valiev, Adam Tutko, David Kennard, Russell Zaretzki, and Audris Mockus. 2021. World of Code: Enabling a Research Workflow for Mining and Analyzing the Universe of Open Source VCS Data.Empirical Software Engineering26, 2 (2021), 22. doi:10.1007/s10664-020-09905-9

    work page doi:10.1007/s10664-020-09905-9 2021

  7. [7]

    Audris Mockus. 2026. Deforking the World of Code: A Project-Provenance Map that Recovers Cross-Forge Fork Families that Platform Graphs Cannot See. arXiv:2606.29550 [cs.SE] doi:10.48550/arXiv.2606.29550 Companion MSR Data and Tool Showcase; commit-based deforking map and GHArchive declared-fork validation

    work page Pith review arXiv doi:10.48550/arxiv.2606.29550 2026

  8. [8]

    Audris Mockus. 2026. Was It Never Collected, or Rewritten Away? A Commit- Provenance Dataset Separating Ingestion Gaps from Upstream History Ed- its across the World of Code. Companion MSR Data and Tool Showcase; present/rewritten/never-ingested commit provenance from the GHArchive Pu- shEvent stream

    work page 2026

Show all 10 references
  1. [9]

    Audris Mockus, Diomidis Spinellis, Zoe Kotti, and Gabriel John Dusing. 2020. A Complete Set of Related Git Repositories Identified via Community Detection Approaches Based on Shared Commits. InProceedings of the 17th International Conference on Mining Software Repositories (MS...

    2020 doi

  2. [10]

    Zachary Newman, John Speed Meyers, and Santiago Torres-Arias. 2022. Sigstore: Software Signing for Everybody. InProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (CCS). 2353–2367. doi:10.1145/3548606. 3560596 Conference’17, July 2017, Washin...

    2022 doi

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.