Pith. sign in

REVIEW 3 major objections 6 minor 16 references

Trust scores on open-source chat LLMs shift far more across successive generations than sampling noise would allow, so they cannot be carried forward without re-measurement.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 09:30 UTC pith:ZKULB5PC

load-bearing objection Solid open-source longitudinal audit: trust scores on named release lines do not transfer, and the process recommendation holds even if the 3.60 imes null multiplier is only a reference. the 3 major comments →

arxiv 2607.02587 v1 pith:ZKULB5PC submitted 2026-07-01 cs.SE cs.LG

The Moving Target: A Longitudinal Audit of Trustworthiness Drift Across Twelve Checkpoints of Open-Source Chat LLMs

Zhichao Fan , Yanhang Li , Zexin Zhuang , Xian Sun , Yingshuo Wang This is my paper
classification cs.SE cs.LG
keywords trustworthiness driftopen-source LLMsmodel cardslongitudinal auditscore drift raterelease-line evaluationbenchmark stability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Model cards often quote trust-benchmark scores for a named release line as if the same number still holds when the next checkpoint ships under the same brand. This paper audits four open-source lines—Yi, Qwen, Mistral, and Gemma—at three successive public generations each on a fixed basket of truthfulness, fairness, and safety benchmarks under multiple prompt templates. Mean absolute adjacent-generation score change is 8 percentage points, 3.60 times an independence-based no-drift reference, and the elevation survives dropping any benchmark or line and switching to strict scoring. The authors therefore conclude that a trust score attached to a release line is not a transferable certificate; it must be reported as a checkpoint-bound, dated artefact. They package that practice as a longitudinal model card that records the evaluated checkpoint, the evaluation date, and the drift relative to the prior audited release.

Core claim

Across twelve public checkpoints of four open-source chat-LLM release lines, mean absolute adjacent-generation Score Drift Rate on a fixed trust-benchmark basket is 8.00 percentage points—3.60 times an independence-based pooled no-drift reference null—and remains elevated under strict scoring and leave-one-out perturbations. A trust score attached to a named release line should therefore not be presumed to transfer to the next generation without re-measurement; it should be treated as a checkpoint-bound, time-stamped artefact.

What carries the argument

Score Drift Rate (SDR): the percentage-point change in template-mean benchmark score between two adjacent generations of one release line, aggregated as mean absolute SDR over forty transitions and compared to a pooled no-drift reference null that draws binomial scores from rates pooled across generations.

Load-bearing premise

The paper treats an independence-based reference null that ignores item-level pairing and cross-template correlation as a good enough yardstick for saying the observed drift exceeds sampling noise.

What would settle it

A re-run that keeps item-level correctness vectors and uses a paired item-level permutation null, or multi-seed resamples of the 200-item subsets, yielding mean absolute SDR inside or near the new null’s 99.9-percentile would falsify the claim that drift is far above sampling noise.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Model cards must record the exact checkpoint hash and evaluation date for every trust score.
  • A materially new release (weights, tokenizer, scale, or recipe) should trigger re-audit rather than carry-forward of prior scores.
  • Open-source release-line evaluations used in procurement or governance should not assume transfer across later generations without re-evaluation.
  • Longitudinal model cards should include prior-release scores, observed absolute SDR, and a reference-null summary so readers can decide whether re-audit is warranted.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If closed APIs show similar drift, contracts that treat a model-card score as valid until the next named version would systematically understate risk between updates.
  • Systems that ship continuous or silent weight changes may still need calendar-based re-evaluation even if discrete generation re-audit is enough for open release lines.
  • Item-level paired nulls and multi-seed subset resampling are the natural next checks that would turn the reference comparison into a design-valid test of sampling noise.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper audits four open-source chat-LLM release lines (Yi, Qwen, Mistral, Gemma) at three successive public generations each (12 checkpoints) on a fixed basket of five trust benchmarks under three prompt templates. The pre-specified primary endpoint is mean absolute adjacent-generation Score Drift Rate, |SDR| = 8.00 pp (count-level bootstrap 95% CI [7.57, 9.12]), reported as 3.60× an independence-based pooled no-drift reference null (mean 2.22 pp; 99.9-percentile 3.30 pp) over 40 transitions. The elevation persists under strict scoring, leave-one-benchmark, leave-one-release-line, drop-low-parse, and a constant-parameter-size restriction. From this the authors conclude that trust scores attached to a named release line should not be carried forward without re-measurement, and they package a longitudinal model card (LMC) that binds scores to checkpoint identity, evaluation date, and drift-vs-prior context. Scope is repeatedly limited to the audited open-source, non-canonical, ≤12B setup.

Significance. If the result holds under the stated scope, the paper supplies a concrete, operationally usable argument against treating model-card trust numbers as transferable certificates across successive open-source checkpoints. Strengths that should be credited: a pre-specified primary endpoint; an explicit robustness bundle (Table 4, Appendix I) that keeps |SDR| in [6.36, 9.43] pp; a constant-size restriction (Appendix I.1) that does not collapse the headline; repeated, non-hand-waving scope tables (Tables 2 and 5); and an illustrative LMC template that is useful even if one rejects the formal null comparison. The contribution is not a new statistic but a longitudinal multi-family open-source audit pattern that governance and procurement workflows currently lack. That is a real, if narrow, contribution for cs.SE / evaluation practice.

major comments (3)
  1. [§3.1, Table 3, Appendix G] §3.1, Eq. (1), Table 3, and Appendix G: the headline 3.60× ratio and the “outside the top 10−3 tail” language rest on an independence-based pooled binomial reference null that the paper itself shows can over- or under-state true sampling variance (paired-item reuse lowers Var(Δ); positive cross-template covariance raises Var(Δ̄); net bias is unsigned because item-level correctness vectors were not retained). For a primary endpoint this is load-bearing. Either (i) re-run the primary comparator with the now-persisted per-item traces (paired item-level permutation or template-block bootstrap) so the direction of bias is empirical, or (ii) demote the ratio/tail language in the abstract, §5.1, and Table 3 and lead with absolute |SDR| plus the leave-one-out robustness band, treating the null only as a background sanity check. The governance conclusion does not require a formal p-value, but the
  2. [§4 Sampling, §7, Appendix I] §4 (Sampling) and §7: every reported number sits on one fixed nb = 200-item cached subset per benchmark, with no multi-seed item-subset resampling. The cell-level leave-one-benchmark / leave-one-line / drop-low-parse perturbations (Appendix I) do not substitute for item-subset uncertainty. The paper correctly flags this as “the biggest gap,” but the count-level CIs and the 8.00 pp point estimate are still presented as if subset choice were fixed truth. At minimum, either run a reduced multi-seed sensitivity on a slice of the grid and report the range of |SDR|, or move the primary numerical claim to a form that does not imply subset-stable precision (e.g., “|SDR| remains several pp above the reference null under all reported protocol perturbations”). Without one of these, the precision of the primary endpoint is overstated.
  3. [Table 1, Appendix I.1, Abstract] Table 1 and Appendix I.1: three of eight transitions mix scale change with recipe/tokenizer change (Yi G2→G3; Gemma G1→G2, G2→G3). The constant-parameter restriction (5 transitions, |SDR| = 7.68 pp) is reassuring but is relegated to an appendix and is not reflected in the abstract or the primary Table 3. Because the central claim is about “named release line” non-transferability rather than pure recipe drift, the main text should either (a) headline the constant-size result alongside the full-grid result, or (b) explicitly redefine the estimand as “any materially new public checkpoint of a named line,” including scale, so that scale-mixing is not a confound but part of the target. As written, readers can reasonably worry that scale alone drives part of the signal.
minor comments (6)
  1. [Figure 2, Table 3] Figure 2 caption and Table 3: the grey band is labelled “null 99.9%ile (+/− 3.3 pp)” while the signed histogram is of individual transitions; make explicit that the band is the aggregate-null comparator, not a per-transition critical value, to avoid readers treating ±3.3 pp as a cell-level significance threshold.
  2. [Abstract, §5.2] §5.2 and Tables 8–11: exploratory diagnostics (rank persistence, compliance flips, dimension volatility) are correctly labelled non-load-bearing, but the abstract and introduction still allude to “the gap persists…” without reminding the reader that only |SDR| is primary. A one-sentence separation in the abstract would help.
  3. [§4, Table 2, Appendix L] Table 2 / Appendix L: the BBQ-mixed ambiguous/disambiguated mix and CrowS-Pairs-FC vs PLL caveats are thorough in the appendix but easy to miss. Consider a short “variant semantics” paragraph in §4 that states what each non-canonical score does and does not measure, so absolute score levels are not over-interpreted.
  4. [§5.1, Appendix H] Parse-rate detail (Appendix H): eight low-parse cells on binary safety under T2/T3 are well documented; a single sentence in §5.1 noting that strict scoring moves |SDR| up (to 8.93 pp) rather than down would pre-empt cherry-picking concerns more visibly.
  5. [§2 Related Work] Related Work cites several concurrent/near-concurrent arXiv pieces by overlapping author sets (Li et al. 2026a,b; Zhuang et al. 2026; Wang et al. 2026a,b). Ensure each is used only for the specific methodological point claimed, and that the novelty paragraph does not lean on unpublished concurrent work as established prior art.
  6. [§3] Terminology block in §3 is clear but long; a small glossary table (release line / generation / checkpoint / SDR / LMC) would improve skimmability for practitioners who are the intended LMC audience.

Circularity Check

0 steps flagged

No significant circularity: primary |SDR| claim is an empirical measurement against a constructed reference null, not a quantity forced by definition or self-citation.

full rationale

The load-bearing chain is: (i) run 12 open-source checkpoints on fixed 200-item cached subsets under three templates; (ii) form template-mean scores s_f,g,b; (iii) define SDR as 100 times the adjacent-generation score difference (Eq. 1); (iv) compare mean |SDR| to an independence-based pooled no-drift reference null that pools (c,n) across generations and draws binomials independently. None of these steps reduces the observed aggregate to its inputs by construction. SDR is a plain absolute difference of measured scores; the null forces E[Δ*]=0 under the no-drift hypothesis and is used only as a reference comparator (the paper itself labels it a reference, not a design-valid paired test). The governance conclusion (do not carry trust scores forward without re-measurement) follows from the magnitude and robustness of the measured drifts, not from a fitted parameter renamed as a prediction or from a uniqueness theorem. Concurrent arXiv citations by overlapping authors (Li et al. 2026a,b; Zhuang et al. 2026; Wang et al. 2026a,b) appear in related-work and configuration-reproducibility context and are not load-bearing for the primary endpoint. Score 1 only for the presence of those non-load-bearing concurrent self-cites; the derivation itself is self-contained empirical measurement.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central claim rests on standard statistical machinery plus several design choices that are free parameters or domain assumptions of the audit protocol. No new physical entities are postulated; the longitudinal model card is a reporting template, not an ontological invention. The independence layers of the reference null are the most consequential modelling assumptions.

free parameters (5)
  • nb = 200 fixed cached items per benchmark
    Sample size chosen for compute feasibility; no multi-seed item-subset resampling performed; all primary numbers sit on one fixed draw.
  • Bnull = 5000 / B = 3000 bootstrap replicates
    Monte-Carlo sizes for null and CIs; conventional but arbitrary.
  • empirical-median compliance threshold θb
    Chosen to maximise flip sensitivity for the exploratory half-life diagnostic; not a deployment threshold.
  • ToxiGen toxicity_ai ≥ 2.5 rule
    Annotation-threshold that defines the synthetic-proxy label for ToxiGen-AT.
  • max_new_tokens = 64, greedy decoding, 4-bit NF4
    Inference protocol held constant; part of the audited configuration rather than a fitted scientific constant.
axioms (4)
  • ad hoc to paper Independence-based pooled binomial null is a usable reference for aggregate |SDR| even though it ignores item-level pairing and cross-template covariance
    Stated explicitly in §3.1 and Appendix G; the paper treats it as reference rather than formal test, yet the primary claim is framed relative to it.
  • domain assumption Non-canonical chat variants (BBQ-mixed, CrowS-Pairs-FC, ToxiGen-AT, XSTest-TD) are valid proxies for the trust dimensions being audited
    Four of five benchmarks depart from canonical protocols; authors flag this repeatedly but still interpret results as trustworthiness drift.
  • standard math Count-level binomial bootstrap supplies approximate descriptive uncertainty for the primary endpoint
    Standard parametric bootstrap on (c,n) records; item-level vectors not retained.
  • ad hoc to paper Materially new release = any change in weights, tokenizer, chat template, scale or instruction-tuning recipe
    Operational definition used to convert the statistical finding into the re-audit recommendation (§6).
invented entities (2)
  • Longitudinal Model Card (LMC) no independent evidence
    purpose: Packaging schema that attaches checkpoint hash, evaluation date, cached-sample hash, |SDR| vs prior and null summary to each trust score
    Illustrative reporting artefact; no independent empirical existence outside the paper’s recommendation.
  • Score Drift Rate (SDR) independent evidence
    purpose: Named aggregate of absolute percentage-point score change across adjacent generations
    Simple renaming of mean absolute difference; not a new scientific object.

pith-pipeline@v1.1.0-grok45 · 31081 in / 3369 out tokens · 32702 ms · 2026-07-12T09:30:23.698033+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of The Moving Target: A Longitudinal Audit of Trustworthiness Drift Across Twelve Checkpoints of Open-Source Chat LLMs." pith.science (2026). https://pith.science/paper/ZKULB5PC

@misc{pith2026260702587,
  author       = {Pith},
  title        = {Pith review of: The Moving Target: A Longitudinal Audit of Trustworthiness Drift Across Twelve Checkpoints of Open-Source Chat LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZKULB5PC}},
  note         = {Machine review of arXiv:2607.02587}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Model cards quote trust-benchmark scores without recording when they were measured, and the same number is routinely carried across successive checkpoints of one release line as if the model behind it had not shifted. We test whether it has shifted by auditing four open-source release lines, Yi, Qwen, Mistral, and Gemma, at three successive generations each, on a fixed basket of trust benchmarks under multiple prompt templates. Mean absolute adjacent-generation drift lands well above an independence-based no-drift reference null, and the gap persists when we drop a benchmark, drop a release line, or switch to strict scoring. We therefore conclude that a trust score attached to a release line should not be carried forward to the next checkpoint without remeasurement; it should instead be reported as a checkpoint-bound, dated artefact, which we package as a longitudinal model card. Closed APIs, larger models, canonical benchmark protocols, and fixed month-cadence rules lie outside the audited scope and require their own evaluation.

Figures

Figures reproduced from arXiv: 2607.02587 by Xian Sun, Yanhang Li, Yingshuo Wang, Zexin Zhuang, Zhichao Fan.

Figure 1
Figure 1. Figure 1: Overview of the longitudinal trust-audit pipeline: four open-source release lines (Yi, Qwen, Mistral, Gemma) at three public generations each are evaluated on five non-canonical chat benchmarks across truthfulness-, fairness-, and safety-related proxies under three prompt templates, then aggregated into a Score Drift Rate compared against an independence-based pooled no-drift reference null. Findings and c… view at source ↗
Figure 2
Figure 2. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Template-mean benchmark score (y) versus release generation (x), one line per family, one panel per benchmark. Generations are equally spaced on the x-axis; calendar intervals are not. Yi and Mistral improve near-monotonically on most benchmarks; Qwen declines on TruthfulQA between G2 and G3; Gemma has the largest single jump (+34 pp on BBQ from G2 to G3). Rank-stability diagnostics. Compliance flip rate a… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 5 linked inside Pith

  1. [2]

    Gemma Team

    URL https://arxiv.org/abs/2605.1 4473. Gemma Team. Gemma: Open models based on gemini research and technology.CoRR, abs/2403.08295, 2024a. doi: 10.48550/ARXIV.2403.08295. URL https: //doi.org/10.48550/arXiv.2403.08295. 8 Trustworthiness Drift Across LLM Release Lines Gemma Team. Gemma 2: Improving open language models at a practical size.CoRR, abs/2408.00...

    Pith/arXiv arXiv doi:10.48550/arxiv.2403.08295 2025

  2. [3]

    Jiang, X., Yang, S., Yang, W., Liu, Y ., and Ji, C

    Published at AIWILD, ICML 2026 workshop. Jiang, X., Yang, S., Yang, W., Liu, Y ., and Ji, C. SOK: A tax- onomy of attack vectors and defense strategies for agentic supply chain runtime.arXiv preprint arXiv:2602.19555, 2026b. URL https://arxiv.org/abs/2602.1 9555v2. Published at ICLR 2026 Workshop on AI for Mechanism Design and Strategic Decision Making; a...

    Pith/arXiv arXiv 2026

  3. [4]

    URL https://openreview.net/forum ?id=iO4LZibEqW. Lin, L. and Wang, Y . SHAP stability in credit risk manage- ment: A case study in credit card default model.Risks, 13(12):238, 2025. doi: 10.3390/risks13120238. URL https://www.mdpi.com/2227-9091/13/12/ 238. Lin, L., You, J., Li, Y ., Lin, L., Wang, Y ., Zhang, Z., and Zheng, M. Reflect-guard: Enhancing LLM...

    Pith/arXiv arXiv doi:10.3390/risks13120238 2025

  4. [5]

    Luo, H., Huang, H., Deng, Z., Li, X., Wang, H., Jin, Y ., Liu, Y ., Xu, W., and Liu, Z

    URL https://arxiv.org/abs/2605.0 8060. Luo, H., Huang, H., Deng, Z., Li, X., Wang, H., Jin, Y ., Liu, Y ., Xu, W., and Liu, Z. BIGbench: A unified benchmark for evaluating multi-dimensional social biases in text-to- image models.arXiv preprint arXiv:2407.15240, 2024. URLhttps://arxiv.org/abs/2407.15240. 9 Trustworthiness Drift Across LLM Release Lines Luo...

    Pith/arXiv arXiv doi:10.1145/3287560.3287596 2024

  5. [6]

    doi: 10.18653/V1/2020.EMNLP-MAIN.154

    Association for Computational Linguistics, 2020. doi: 10.18653/V1/2020.EMNLP-MAIN.154. URL https://doi.org/10.18653/v1/2020.emn lp-main.154. Oren, Y ., Meister, N., Chatterji, N. S., et al. Proving test set contamination in black-box language models. InThe Twelfth International Conference on Learning Represen- tations, ICLR 2024, Vienna, Austria, May 7-11...

    doi:10.18653/v1/2020.emnlp-main.154 2020

  6. [7]

    URL https://doi.org/10.18653/v1/2022 .findings-acl.165

    doi: 10.18653/V1/2022.FINDINGS-ACL.165. URL https://doi.org/10.18653/v1/2022 .findings-acl.165. Perez, E., Huang, S., Song, H. F., et al. Red teaming language models with language models. In Goldberg, Y ., Kozareva, Z., and Zhang, Y . (eds.),Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, EMNLP 2022, Abu Dhabi, Un...

    doi:10.18653/v1/2022.findings-acl.165 2022

  7. [8]

    doi: 10.18653/V1/2022.EMNLP-MAIN.225

    Association for Computational Linguistics, 2022. doi: 10.18653/V1/2022.EMNLP-MAIN.225. URL https://doi.org/10.18653/v1/2022.emn lp-main.225. Qian, P., Wang, S., Wang, X., Chen, Y ., Xu, W., Yu, Q., Lin, S., Zhang, S., You, J., and Wei, X. Relevant is not warranted: Evidence-force calibration for cited RAG,

    doi:10.18653/v1/2022.emnlp-main.225 2022

  8. [9]

    Qwen Team

    URL https://arxiv.org/abs/2605.2 8044. Qwen Team. Qwen1.5-7B-Chat model card. Hugging Face model repository, February 2024a. URL https:// huggingface.co/Qwen/Qwen1.5-7B-Chat . Accessed 2026-07-01. Qwen Team. Qwen2.5-7B-Instruct model card. Hugging Face model repository, September 2024b. URL https: //huggingface.co/Qwen/Qwen2.5-7B-Ins truct. Accessed 2026-...

    2026

  9. [10]

    doi: 10.18653/V1/2024.NAACL-LONG.301

    Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.NAACL-LONG.301. URL https://doi.org/10.18653/v1/2024.naa cl-long.301. Salarian, S., Zhang, Y ., Padhee, S., and Parthasarathy, S. MedEqualizer: A framework investigating bias in synthetic medical data and mitigation via augmenta- tion.arXiv preprint arXiv:2511.01054, 2025. URL https://...

    doi:10.18653/v1/2024.naacl-long.301 2024

  10. [11]

    0–9 months

    URL https://openreview.net/forum ?id=uyTL5Bvosj. Wang, S., Qian, P., Chen, Y ., You, J., Wang, X., Jiang, X., Liu, L., Yu, H., and Xu, J. When safe skills collide: Measuring compositional risk in agent skill ecosystems, 2026a. URL https://arxiv.org/abs/2606.0 0448. Wang, Y ., Sun, X., Li, Y ., Fan, Z., and Zhuang, Z. Auditing and fixing economic validity ...

    Pith/arXiv arXiv doi:10.48550/arxiv 2026

  11. [12]

    Within-record item independence: item-level outcomes are assumed iid Bernoulli at the record’s rate, ignoring item-difficulty heterogeneity

  12. [13]

    Paired-item reuse across generations: the two adjacent-generation evaluations score thesame 200-item cached sample, but the null draws them independently, ignoring the per-item correlationρ item betweenGandG ′

  13. [14]

    lower bound on the ratio

    Template dependence within a checkpoint: the three template records for the same(f, g, b) are simulated independently, ignoring cross-template correlation induced by the shared checkpoint and shared item sample. Variance decomposition for the template-averaged difference.Let ∆t =s ⋆ f g′bt −s ⋆ f gbt be the signed transition difference at templatet, and ¯...

    2020

  14. [15]

    3.Run all three prompt templatesT 1, T2, T3 (Appendix F); the aggregate depends on template-averaging

    Use the fixed 200-item cached samples( experiments/cache/{benchmark}_200.json), or document any substitution and rerun the reference-null analysis. 3.Run all three prompt templatesT 1, T2, T3 (Appendix F); the aggregate depends on template-averaging. 4.Record(c, n parsed, ntotal)per evaluation; optionally retain per-item traces

  15. [16]

    the prior releaseand compare to the pooled no-drift reference null; flag for revalidation when it exceeds the null99.9-percentile

    Compute mean |SDR| vs. the prior releaseand compare to the pooled no-drift reference null; flag for revalidation when it exceeds the null99.9-percentile. 6.Default to strict scoring(c/n total) for deployment-grade reports; annotate if lenient. 7.Compliance check: per-benchmarkˆp flip at the deployer’s actual threshold with exact Clopper–Pearson95%CI

  16. [17]

    Stampcheckpoint hash, cached-sample hash, template IDs, and evaluation date; treat audits lacking these as unsupported for carry-forward. N. The longitudinal model card (extended template) The extended longitudinal model card (LMC): the full field set extending the Mitchell et al. (2019) Model Card schema for longitudinal trust reporting. Table 6 in the m...

    2019