Pith. sign in

REVIEW 3 major objections 5 minor 16 references

Sampled weight fingerprints can verify open-model lineage and recover merge recipes without full downloads.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 10:26 UTC pith:GE34NNZC

load-bearing objection Solid systems paper: element-aligned fingerprints plus constrained least-squares actually recover published merge recipes; LineageBench is clean but tiny. the 3 major comments →

arxiv 2607.10617 v1 pith:GE34NNZC submitted 2026-07-12 cs.LG

modelDNA: Calibrated Lineage Verification and Merge Decomposition from Sampled Weight Fingerprints

Muhammad Awais Bin Adil , Saad Aamir This is my paper
classification cs.LG
keywords model lineageweight fingerprintsopen-weight modelsmerge decompositioncalibrated verificationPCS samplestask-vector regression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Open-weight language models mostly document their parentage on the honor system, and more than half of Hub models list none at all. When a provenance dispute arises, analysts still re-run ad-hoc weight comparisons by hand. This paper presents modelDNA, a practical tool that fingerprints a model from only 100–300 MB of ranged HTTP reads, compares it against a reference set of foundation models using four published signal families, and returns one of eight calibrated verdict classes while preferring abstention to a false accusation. On a benchmark of real Hub models whose parentage is corroborated by the publishing organizations themselves, the system separates true parents from hard negatives with AUROC 1.0, zero false positives at its reporting threshold, and perfect top-1 parent attribution. Because mainstream merge methods are near-linear per tensor and the fingerprints sample identical element positions, the same fingerprints also let the tool recover mixture weights by sum-to-one constrained least squares, reconstructing published slerp curves and dare_ties weights without ever downloading full weights.

Core claim

A fingerprint built from four literature signal families (attention σ-curves, 1-D norm/bias vectors, PCS samples, and top singular values), sampled at positions that are pure functions of tensor identity, is enough both to produce calibrated lineage verdicts on real Hub models and to recover the mixture weights of near-linear merges by sum-to-one constrained least squares on the fingerprints alone.

What carries the argument

Element-aligned PCS fingerprints: sample positions are deterministic functions of (seed, canonical role, layer, tensor size), so sampling commutes with per-tensor linear combinations and a merged model’s fingerprint is the same mixture of its parents’ fingerprints; the sum-to-one constraint then reduces the fit to well-conditioned regression in task-vector space.

Load-bearing premise

The four published weight signals, combined by a logistic calibrator fitted on synthetic pairs, still separate real fine-tunes, quantizations, and near-linear merges from unrelated models of the same shape without false positives.

What would settle it

Add more real Hub models whose organizations document parentage; if any true parent falls below the 0.9 reporting threshold while a cross-family negative exceeds it, or if a published mergekit recipe cannot be recovered within the stated residual-gain and weight-error bounds, the central claims fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. modelDNA is a practical lineage-forensics tool for open-weight LLMs. It builds element-aligned fingerprints from 100–300 MB of ranged HTTP reads over four published signal families (attention σ-curves, 1-D norm/bias vectors, PCS samples, singular-value spectra), maps pairwise evidence to one of eight abstention-first verdict classes with a non-negative logistic calibrator, and, as a second contribution, recovers merge mixture weights from fingerprints alone by sum-to-one constrained least squares in task-vector space. On LineageBench (13 org-documented positives vs 107 cross-family hard negatives) the system reports AUROC 1.0, zero false positives at p≥0.9, and 13/13 top-1 parent attribution; against three published mergekit recipes it recovers a slerp layer-interpolation curve at r=0.999 and dare_ties weights to within 0.011. All fingerprints, benchmarks, and a 55-model atlas are public and offline-reproducible.

Significance. If the results hold, the paper supplies the missing engineering layer between published lineage signals and usable provenance tooling: a single command, a maintained reference DB, calibrated abstention, and background distributions. The merge-decomposition result is the more novel technical contribution—showing that identity-keyed sampling commutes with near-linear merges and that the sum-to-one constraint moves the regression into well-conditioned task-vector space—and is tightly validated against public mergekit configs without full weight downloads. Explicit strengths include offline reproducibility of every reported number, org-documented (not self-reported) ground truth for LineageBench, zero-FP ship gates, and honest limitation cases (depth-upscales, same-family merges, light fine-tunes vs recasts). The work is immediately useful for Hub audits and provenance disputes.

major comments (3)
  1. [Section 7.2, Abstract] Section 7.2 / abstract: LineageBench contains only 13 scored positives (and 107 cross-family hard negatives). AUROC 1.0, 0/107 FPs at p≥0.9, and 13/13 top-1 are correctly reported for this set, but the abstract and introduction present them as the primary verification claim. Same-family near-misses are excluded by design (correct for the stated negative definition, yet they are exactly the hard cases a scanner meets). The authors note the scale themselves; the manuscript should either enlarge the positive pool before claiming headline metrics or relegate the perfect AUROC to a scoped ‘no errors at this scale’ statement and lead with the merge-recovery results, which rest on stronger ground truth.
  2. [Sections 5.2, 7.1, 8] Sections 5.2 and 7.1: The logistic calibrator (non-negative coefficients, missing-feature imputation to unrelated-pair means) is bootstrap-fit exclusively on the synthetic population. Rankings on LineageBench are perfect, but probability magnitudes and the p≥0.9 reporting threshold are not independently validated on real pairs; Section 8 already flags this. Because the MERGE residual-gain threshold (≥0.30) and the eight-class taxonomy inherit the same probability scale, the paper should either refit/validate the calibrator on a held-out real subset or demote exact p-values to secondary status and treat thresholded rankings plus residual gain as the primary outputs.
  3. [Sections 6.3, 7.3] Section 6.3 / 7.3: The MERGE call requires residual gain ≥0.30, motivated by an impostor regime around 0.25 and one dare_ties case that scores 0.219 (honestly labeled AMBIGUOUS). With only three published-recipe cases, the threshold is under-constrained. A short sensitivity table (gain thresholds vs false-MERGE rate on sibling fine-tunes and true merges) would make the operating point falsifiable rather than fixed by the single Monarch example.
minor comments (5)
  1. [Section 7.2] Section 7.2: Nous-Hermes is correctly parented but classed QUANTIZED_COPY because PCS 0.9998 crosses the dtype-recast rule. A short note or delta-based discriminator (as the authors already propose) would prevent readers from over-interpreting subclass labels.
  2. [Figure 1, Section 7.3] Figure 1 caption and Section 7.3: The MLP layer-0 unidentifiability (task-vector norm collapse) is explained in text; marking that point on the figure would make the r=0.970 result self-explanatory.
  3. [Section 4.2] Section 4.2: Sampling constants (32 blocks × 16 384, top-32 singular values, 192-row band) are frozen and versioned—good—but stating the schema version string in the fingerprint header and in the paper would aid long-term cache compatibility.
  4. [Section 2] Section 2: REEF, TensorGuard, SeedPrints, and PhyloLM are correctly scoped as complementary tiers; a one-sentence roadmap note on when behavioral signals would be invoked (e.g., distillation) would tighten the out-of-scope boundary.
  5. [Section 7.1] Typos / polish: ‘defamation-risk surface’ (7.1) is vivid but informal for a methods section; ‘ship gate’ appears several times—consider ‘release criterion’ once defined.

Circularity Check

0 steps flagged

No significant circularity: lineage metrics rest on external org-documented ground truth and a synthetic-then-real train/eval split; merge recovery is ordinary constrained least squares validated against published mergekit YAML, not a tautology.

full rationale

The paper's two load-bearing claims are independently grounded. LineageBench positives are labeled from publishing organizations' technical reports and release notes, not from the base_model metadata the tool audits; hard negatives are cross-family pairs; the logistic calibrator is fit only on the synthetic population of §7.1 and then ranked on the real 13/107 pool of §7.2, so rankings are not forced by the fit. Merge decomposition follows from two external facts (mainstream mergekit methods are near-linear per tensor; sample positions are pure functions of tensor identity) that make sampling commute with the mixture; recovering α by sum-to-one constrained least squares is ordinary regression in task-vector space, and the reported r=0.999 curves and 0.011 weight errors are comparisons to published mergekit configurations, not re-statements of the design matrix. Residual-gain ≥0.30 is a threshold chosen above observed impostor scores, not a prediction of those scores. There is no self-citation chain (authors do not appear among the cited signal papers), no uniqueness theorem imported from prior work by the same authors, and no ansatz smuggled in via self-citation. The derivation is self-contained against external benchmarks; residual concerns are about sample size and survival envelopes of the cited signals, not circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The work rests on published survival claims for four weight-space signals, on the near-linearity of mainstream merge operators, and on a small set of free thresholds and sampling constants. No new physical entities are postulated; the free parameters are engineering choices (sample sizes, probability cutoffs, residual-gain threshold) that are frozen and versioned. Domain assumptions about what fine-tuning and merging do to weights are inherited from the cited papers and validated on the small real benchmark rather than derived from first principles.

free parameters (5)
  • PCS sample plan (32 blocks × 16 384 elements per 2-D role)
    Sampling constants are frozen and versioned; they determine fingerprint size and the quality of the cosine estimator. Chosen by engineering trade-off, not derived.
  • positive verdict threshold p ≥ 0.90 and abstention band 0.50–0.90
    Ship-gate thresholds that trade recall for zero false positives; set to enforce the defamation-risk surface rather than estimated from a large real calibration set.
  • MERGE residual-gain threshold ≥ 0.30
    Chosen to sit above the observed impostor-sibling regime (~0.25); directly controls false MERGE calls on ordinary fine-tunes.
  • logistic calibrator coefficients (non-negative, bootstrap-fit on synthetic pairs)
    Fitted on synthetic data; probability magnitudes on real models are acknowledged as less validated than rankings.
  • spectral estimation (top-32 singular values, 192-row band every fourth layer in fast mode)
    Approximation constants for F4; affect invariance claims under re-parameterization.
axioms (5)
  • domain assumption The four published signal families (attention σ-curves, 1-D norm/bias vectors, PCS, singular-value spectra) survive ordinary SFT, continued pretraining, LoRA-merge, and quantization with enough residual signal to separate related from unrelated pairs.
    Inherited from HonestAGI, HuRef, Ghost in the Transformer (Section 2, 4.2); modelDNA does not re-prove survival envelopes.
  • domain assumption Mainstream mergekit methods (linear, task arithmetic, slerp at high cosine, TIES/DARE in expectation) produce per-tensor weights that are linear or near-linear combinations of the parents.
    Stated in Section 6.1; enables the commuting of sampling with the merge operator.
  • standard math Sample positions that are pure functions of (seed, canonical role, layer, tensor size) yield element-aligned fingerprints across models of matching shape, so cosines and least-squares fits are unbiased estimators of the full-tensor quantities.
    Deterministic sampling + linearity of expectation / cosine estimation (Section 4.2, 6.1).
  • standard math Sum-to-one constraint moves the regression from near-collinear raw weights into well-conditioned task-vector differences.
    Section 6.2 geometric argument; standard constrained least squares.
  • ad hoc to paper Missing features should be imputed with unrelated-pair background means so absence of evidence is never treated as evidence.
    Design choice of the verdict engine (Section 5.2) that shapes calibration behavior.
invented entities (2)
  • modelDNA fingerprint schema (F1–F4 bundle with identity-keyed sample positions) independent evidence
    purpose: Make multi-signal, multi-model comparisons and merge decomposition possible from partial reads.
    Engineering construct, not a physical entity; independent evidence is the public fingerprint caches and offline benchmarks.
  • eight-class abstention-first verdict taxonomy (EXACT_COPY … INSUFFICIENT) independent evidence
    purpose: Map continuous evidence to public-facing claims while preferring honest abstention.
    Product design; validated by zero false positives on the stated benchmarks.

pith-pipeline@v1.1.0-grok45 · 15962 in / 4012 out tokens · 35830 ms · 2026-07-14T10:26:11.582519+00:00 · methodology

0 comments
read the original abstract

The lineage graph of open-weight language models is self-reported: Hugging Face's base_model metadata field is optional and unverified, and over 60% of Hub models document no parentage at all. Methods for detecting lineage from weights exist in the research literature, but each ships as paper code tied to one signal and one experiment; when a provenance dispute breaks, the analysis is redone by hand. This report describes modelDNA, a tool that fingerprints a model from roughly 100-300 MB of ranged HTTP reads (instead of a full 15 GB download for a 7B model), compares the fingerprint against a reference database of foundation models across four published signal families, and returns one of eight verdict classes with a calibrated probability, preferring honest abstention to confident error. On a benchmark of 15 real Hub models with org-documented parentage, judged against 8 candidate bases (13 positives, 107 hard negatives), the system achieves AUROC 1.0, zero false positives at its reporting threshold, and 13/13 correct top-1 parent attribution. The report's second contribution is merge decomposition. Every mainstream weight-merging method is (near-)linear per tensor, and fingerprint sample positions are deterministic functions of tensor identity, so a merged model's fingerprint is the same linear combination of its parents' fingerprints. Mixture weights can therefore be recovered from fingerprints alone by sum-to-one constrained least squares. Against merges with published mergekit configurations as ground truth, the method recovers a slerp merge's layer-interpolation curves at r = 0.999 and a dare_ties merge's mixture weights to within 0.011 of the published values, without downloading any weights beyond the fingerprints. All fingerprints, benchmarks, and the inferred lineage graph of 55 models are public and reproducible offline.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 12 linked inside Pith

  1. [1]

    Intrinsic Fingerprint of LLMs: Continue Training is NOT All You Need to Steal A Model

    HonestAGI. Intrinsic Fingerprint of LLMs: Continue Training is NOT All You Need to Steal A Model. arXiv:2507.03014, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Zeng et al

    J. Zeng et al. HuRef: HUman-REadable Fingerprint for Large Language Models. NeurIPS 2024. arXiv:2312.04828

    Pith/arXiv arXiv 2024

  3. [3]

    S. Wang, Z. Ma, X. Li, Z. Li. Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures. arXiv:2511.06390, 2025

    arXiv 2025

  4. [4]

    Horwitz et al

    E. Horwitz et al. On the Origin of Llamas: Model Tree Heritage Recovery. arXiv:2405.18432, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Horwitz et al

    E. Horwitz et al. Charting and Navigating Hugging Face’s Model Atlas. arXiv:2503.10633, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Nasery, E

    A. Nasery, E. Contente, A. Kaz, P. Viswanath, S. Oh. Are Robust LLM Fingerprints Adversarially Robust? arXiv:2509.26598, 2025

    arXiv 2025

  7. [7]

    Zhang et al

    J. Zhang et al. REEF: Representation Encoding Fingerprints for Large Language Models. ICLR 2025. arXiv:2410.14273

    Pith/arXiv arXiv 2025

  8. [8]

    Z. Wu, Y. Zhao, H. Wang. Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification (TensorGuard). arXiv:2506.01631, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Y. Tong, H. Wang, S. Li, K. Kawaguchi, T. Hu. SeedPrints: Fingerprints Can Even Tell Which Seed Your Large Language Model Was Trained From. arXiv:2509.26404, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Yax et al

    N. Yax et al. PhyloLM: Inferring the Phylogeny of Large Language Models. ICLR 2025. arXiv:2404.04671

    arXiv 2025

  11. [11]

    S. Shao, Y. Li, Y. He, H. Yao, W. Yang, D. Tao, Z. Qin. SoK: Large Language Model Copyright Auditing via Fingerprinting. arXiv:2508.19843, 2025

    arXiv 2025

  12. [12]

    Halko, P.-G

    N. Halko, P.-G. Martinsson, J. A. Tropp. Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions. SIAM Review 53(2), 2011. arXiv:0909.4061

    Pith/arXiv arXiv 2011

  13. [13]

    Goddard et al

    C. Goddard et al. Arcee’s MergeKit: A Toolkit for Merging Large Language Models. EMNLP 2024 (Industry Track). arXiv:2403.13257

    Pith/arXiv arXiv 2024

  14. [14]

    Ilharco et al

    G. Ilharco et al. Editing Models with Task Arithmetic. ICLR 2023. arXiv:2212.04089

    Pith/arXiv arXiv 2023

  15. [15]

    Yadav et al

    P. Yadav et al. TIES-Merging: Resolving Interference When Merging Models. NeurIPS 2023. arXiv:2306.01708

    Pith/arXiv arXiv 2023

  16. [16]

    Yu et al

    L. Yu et al. Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch (DARE). ICML 2024. arXiv:2311.03099. Appendix A: Reproducing every number in this report git clone https://github.com/AwaisAdilKhokhar/modelDNA && cd modelDNA pip install -e . # Section 7.1: synthetic gates python benchmarks/synthetic_bench.py # Sectio...

    Pith/arXiv arXiv 2024