Pith. sign in

REVIEW 4 major objections 5 minor 23 references

Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Metrics-Weighted Averaging weights saved LoRA checkpoints by their training loss, and the merged model beats uniform averaging — and often the final checkpoint — by up to 5.05% relative on the GSM benchmark.

desk verdict A simple, plausible LoRA checkpoint-merging heuristic whose headline gains are partly a product of penalty-factor selection on the test benchmark; weaker than claimed, but worth a serious look. read the letter →

arxiv 2504.18580 v1 pith:4G2ZJFDG submitted 2025-04-23 cs.LG

Shi Jie Yu , Sehyun Choi This is my paper · ORCID
classification cs.LG
keywords checkpointmergingmodelmetrics-weightedaveragingloss-weightedLoRAparameter-efficientfine-tuningsouplargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Checkpoint merging normally averages fully trained models, but this paper asks whether cheaply weighted averages of LoRA adapters from a single fine-tuning run can do better. The proposed method, Metrics-Weighted Averaging (MWA), weights each checkpoint by its training loss (or step count), shaped by a single penalty factor, and averages the adapter parameters. Across math reasoning, preference alignment, and instruction tuning, loss-weighted merges beat the uniform average of the same checkpoints and, on the math and alignment runs, also outperform the final individual checkpoint, with up to 5.05% relative gain on the GSM weighted benchmark. If correct, the result means saved checkpoints plus their loss curves are enough to squeeze extra accuracy out of a fine-tuning run at near-zero extra compute.

What carries the argument

The load-bearing object is the rank-shaped weighting formula $\varphi_x = p^{q \cdot \mathrm{pos}(\ell_x)} \cdot \ell_x^{-1}$ for loss (or $\varphi_x = p^{q \cdot \mathrm{pos}(s_x)} \cdot s_x$ for steps), normalized by the sum over checkpoints. The penalty factor $p$ controls how fast weight decays with a checkpoint's rank, while the power factor $q$ sets the steepness of the decay; because raw losses differ by only small amounts late in training, the pure reciprocal-loss weights would be nearly uniform, and the rank term is what spreads them out. The same formula is presented as metric-agnostic through a min/max dichotomy, meaning any cheap signal — lower-is-better or higher-is-better — can drive the weights with the same single knob.

What would settle it

Score every saved checkpoint on the downstream benchmark (e.g., the GSM weighted average) for a fresh run of the same three training recipes. If the inverse training loss ranks checkpoints differently from the benchmark scores, or if the best loss-weighted soup found by sweeping the penalty factor does not beat the uniform soup of the same checkpoints and the final checkpoint, then the central claim is falsified. The paper reports merged-model scores but not per-checkpoint benchmark scores, so this check is open.

Watch

Extended reading notes

Core claim

The paper's central claim is that the relative value of a checkpoint for merging can be read off a cheap training signal, and that a geometric penalty on the checkpoint's rank turns that signal into weights that beat naively averaging the same checkpoints. Concretely, for checkpoints with losses $\ell_x$, MWA sets the unnormalized weight $\varphi_x = p^{q \cdot \mathrm{pos}(\ell_x)} / \ell_x$, where $\mathrm{pos}(\ell_x)$ is the zero-indexed rank from lowest to highest loss, $p<1$ is the single tuned penalty factor, and $q>1$ is a fixed power factor; steps weighting uses $\varphi_x = p^{q \cdot \mathrm{pos}(s_x)} \cdot s_x$ instead. After normalization, the LoRA adapters are averaged with these weights. Across the three runs the best models come from loss-weighted merges of 4-10 checkpoints, including interval-spaced ones; the best math result improves the GSM weighted average by 5.05% over the last merging checkpoint and 1.76% over the final checkpoint. The paper further claims that this one-knob procedure outperforms uniform averaging and, measured relative to each run's own reference checkpoints, outperforms the TIES, DARE-TIES, and SLERP baselines on domain benchmarks, while challenging the earlier finding that only pairwise or adjacent checkpoint merges help.

Load-bearing premise

The whole scheme assumes training loss is a faithful ranking of a checkpoint's value for merging: if a lower-loss checkpoint is not actually more useful inside the average, the weights are arbitrary.

Editorial extensions

If this is right

  • A fine-tuning run that already logs training loss can produce a better final model without retraining: merging the last 4-10 saved checkpoints with loss-derived weights is the recipe.
  • The cost of choosing merge weights stops growing with the number of checkpoints: instead of tuning $k-1$ weights, one penalty factor $p$ is swept, so the heuristic's advantage grows as the soup gets larger.
  • Merging checkpoints that are spaced apart (interval merging) can beat merging adjacent ones, and merges of 4-10 checkpoints beat pairwise merges, contradicting earlier checkpoint-merging conclusions.
  • The choice of metric decides the outcome: loss-weighted merging beat steps-weighted merging in all three experiments, so the practical rule is to weight by the signal that best tracks downstream quality.
  • The same weighting formula applies to any lower-is-better or higher-is-better metric, so other cheap training signals can be dropped into the identical one-knob procedure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The penalty factor $p$ interpolates between a nearly uniform soup and a near-one-hot selection of the best-loss checkpoint, so part of MWA's gain is likely variance reduction or implicit early stopping rather than fine-grained weighting; an ablation that fixes the selected subset and varies only the weights would separate the two effects.
  • Because the formula needs only a loss curve, the same merging trick should transfer to full fine-tuning and pretraining checkpoints, which the paper did not test; if the loss-to-quality link holds there, pretraining runs could harvest their routinely saved checkpoints the same way.
  • A cheap held-out validation loss would be a drop-in 'min' metric that may rank checkpoints more faithfully than training loss, especially late in training when training-loss differences are tiny; testing that variant is the most direct next experiment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Metrics-Weighted Averaging (MWA), a heuristic for merging LoRA checkpoints taken from a single fine-tuning run. Checkpoints are weighted by the reciprocal of their training loss (loss-weighted) or by their step count (steps-weighted), with an additional rank-based penalty factor p applied to control the weight spread. Experiments cover three tasks: math SFT (GSM8K/GSM-Plus), alignment via SimPO (ToxiGen/TruthfulQA), and instruction tuning (OpenHermes-2.5 validation loss). The authors report that MWA often beats uniform averaging of the same checkpoints and sometimes beats the final individual checkpoint, with the best math gain stated as +5.05% on the GSM weighted average. They also compare MWA against Ties, Dare-Ties, and SLERP, claiming relative superiority on domain-specific benchmarks, and they discuss limitations and cases where MWA underperforms uniform merging.

Significance. If the claims are supported, MWA would be a simple, low-overhead addition to PEFT checkpoint merging: it requires only one tunable hyperparameter per merge and uses metrics that are already recorded during training. The paper covers three distinct post-training scenarios, includes baseline methods (Ties, Dare-Ties, SLERP), and is candid in Section 5.5 about pairwise merges often underperforming uniform averaging. However, the significance is currently limited by the evaluation protocol: the penalty factor p is selected on the same benchmarks used to report success, all results come from single runs without error bars, and the baseline comparison in Section 5.4 is weakened by an unsubstantiated hardware-difference explanation. The core idea is plausible, but the evidence as presented does not yet support the abstract's claim of 'consistently' outperforming uniform averaging.

major comments (4)
  1. [Section 5.1 and Tables 2-3] The model-selection procedure in Section 5.1 evaluates uniform soups on the target benchmark, shortlists the top 1-2, and then varies the penalty factor p to produce MWA soups, with the best results reported in Tables 2-3. Because p is chosen by maximizing the same GSM, alignment, or validation-loss metric used to claim success, and no held-out validation split is used, the reported gains (e.g., +5.05% in Figure 1) are selected maxima over a small sweep rather than unbiased estimates of MWA's advantage. To support the central claim, the paper should report performance on a held-out validation set or use a nested selection procedure, and should show results for all p values swept.
  2. [Section 5.4 and Table 4] In the math benchmark, the raw baseline results are higher than the best MWA results (e.g., dare_ties_last_3_base_first achieves 0.2295 vs. 0.22826 for last_10_loss_pf-0_7), but the text dismisses this with the statement that 'the two runs used different underlying hardware accelerators' without providing any controlled experiment to support that claim. Since the base-model scores also differ between the two runs (gemma-2b 0.1251 vs. 0.12281 in Tables 2 and 4), the relative comparison does not establish that MWA is superior to Ties, Dare-Ties, or SLERP. The comparison should be conducted under identical hardware and base-model conditions, or the claim of superiority should be withdrawn.
  3. [Section 5.5 and Abstract] The abstract states that MWA 'consistently produces merged models that outperform the naive uniform average of checkpoints,' but Section 5.5 explicitly says that for pairwise merges MWA 'marginally outperforms the Uniform Soup equivalent at best, and often underperforms the latter.' The claim of consistency is therefore too broad; either the abstract and conclusion should be scoped to merges of 4-10 checkpoints, or the pairwise underperformance should be reported as a caveat in the abstract.
  4. [Tables 2-3 and Section 5.2] All reported results are single runs without error bars or significance tests. Some headline improvements are small in absolute terms: in Table 2, last_10_loss_pf-0_7 has weighted average 0.22826 versus 0.22431 for the final checkpoint, a difference of about 0.004 that on GSM8K/GSM-Plus corresponds to a handful of examples and may lie within evaluation noise. Without multiple training seeds or repeated evaluations, the improvements cannot be distinguished from random variation, especially when p is selected post hoc on the same benchmark.
minor comments (5)
  1. [Section 3.1] The power factor q is said to be 'fixed heuristically based on the distribution of checkpoint metrics values,' but the paper never reports the q values used in any experiment or describes the heuristic concretely. Please state the chosen q values and the rationale.
  2. [Figure 1 and Figure 2 captions] The figure captions define 'baseline' as the last merging checkpoint, while the abstract and text use 'uniform soup' as the baseline. Please clarify which baseline is shown in each figure to avoid inconsistent references.
  3. [Figure 3 and Section 5.3] In Figure 3, validation-loss values are labeled only to two decimal places (e.g., 17.78 vs. 17.82), but the differences discussed are as small as 0.02, making the order of bars visually ambiguous. A table or more decimal places would make the results clearer.
  4. [Table 6 footnote] The footnote states 'All models at or below the highlighted model show validation losses lower than those of both the final checkpoint and the last merging checkpoint,' but several models above the highlighted row (e.g., ties_last_3_base_last at 17.4391) also have lower validation loss than 17.4189, so the statement is inaccurate as written.
  5. [General presentation] There are several typographical errors, including 'seection' in Section 3, 'afects' in Section 3.1, and 'simpo_lora_hh-rlhf' with inconsistent hyphenation. The LoRA acronym is also written as both 'LoRA' and 'LoRa'; please standardize.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MWA is an openly heuristic weighting formula, and the reported gains are empirical comparisons rather than consequences of the definitions.

full rationale

The paper proposes Metrics-Weighted Averaging as an explicit heuristic: Eq. (1)–(3) define checkpoint weights as functions of training loss and a penalty factor p, and Eq. (4) defines steps-weighted weights. These formulas do not contain the evaluation benchmarks (GSM8K/GSM-Plus, ToxiGen/TruthfulQA, OpenHermes-2.5 validation loss) as inputs, so the benchmark results are not derived from the method by construction. The claim that lower-loss or later checkpoints are more valuable is stated as intuition (Section 3.1) and then tested empirically; it is not an assumption that logically forces the reported accuracy differences. No self-citations are load-bearing: the only checkpoint-merging reference, Liu et al. 2024a, is used to contrast prior assumptions and is not invoked as a uniqueness or justification theorem. The main validity concern, noted in Section 5.1, is that penalty factors are selected by evaluating on the same benchmarks used to report success (e.g., 'Vary the penalty factors to get different model soups' followed by reporting last_10_loss_pf-0_7 as best). This is a test-set selection / multiple-comparisons issue rather than a circular derivation, because the weighting formula itself is not equivalent to, or fitted on, the evaluation objective. Consequently, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central empirical result rests on three premises: loss is a proxy for checkpoint quality, linear averaging of LoRA adapters is valid, and a geometric decay by rank is a sensible weighting. None is derived; each is asserted heuristically. The penalty factor p and power factor q are chosen by hand, and p is selected per task by evaluating on the benchmark used for the success claim.

free parameters (3)
  • penalty factor p = 0.7 (math), 0.7 (alignment), 0.75 (OpenHermes); exact sweep values not reported
    Single hyperparameter in Eqs. 3 and 4, selected per task after evaluating merges on the target benchmarks (Section 5.1, Tables 2 and 3).
  • power factor q = not reported
    Authors state q is 'fixed heuristically based on the distribution of checkpoint metrics values' (Section 3.1), but do not report the chosen value or the heuristic.
  • evaluation aggregation weights = w_gsm8k = 0.3, w_gsmplus = 0.7; w_toxigen = 0.5, w_tqa_mc1 = 0.25, w_tqa_mc2 = 0.25
    Chosen by hand to define the reported improvement percentages (Tables 2 and 3, Section 5.2). These weights affect the headline numbers.
assumptions (3)
  • domain assumption Training loss is a reliable proxy for checkpoint quality for merging
    Invoked in Section 3.1 as the intuition for Eq. 1 and supported only empirically by the paper's own runs, not by any external or theoretical evidence.
  • domain assumption Linear (parameter-wise) averaging of LoRA weights from the same run preserves and combines capabilities
    Basis of the method, inherited from model soups literature (Wortsman et al. 2022); not independently validated within this paper beyond the merging results.
  • ad hoc to paper Rank-ordering by the metric and applying geometric decay p^(q*pos) yields meaningful weights
    Eqs. 3 and 4 are introduced heuristically with no derivation; the text states the basic formula 'will not suffice' and modifies it without formal justification (Section 3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging." pith.science (2026). https://pith.science/paper/4G2ZJFDG

@misc{pith2026250418580,
  author       = {Pith},
  title        = {Pith review of: Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4G2ZJFDG}},
  note         = {Machine review of arXiv:2504.18580}
}
read the original abstract

Checkpoint merging is a technique for combining multiple model snapshots into a single superior model, potentially reducing training time for large language models. This paper explores checkpoint merging in the context of parameter-efficient fine-tuning (PEFT), where only small adapter modules (e.g. LoRA) are trained. We propose Metrics-Weighted Averaging (MWA), a simple yet effective method to merge model checkpoints by weighting their parameters according to performance metrics. In particular, we investigate weighting by training loss and by training steps, under the intuition that lower-loss or later-step checkpoints are more valuable. We introduce a formula with a penalty factor to adjust weight distribution, requiring only one hyperparameter regardless of the number of checkpoints. Experiments on three fine-tuning tasks (mathematical reasoning, preference alignment, and general instruction tuning) show that MWA consistently produces merged models that outperform the naive uniform average of checkpoints. Notably, loss-weighted merging often yields the best results, delivering up to 5% higher task accuracy than the baseline uniform merge and even surpassing the final individual checkpoint's performance. These findings validate checkpoint merging for PEFT and demonstrate that a metric-driven weighting heuristic can efficiently boost model performance with minimal computational overhead.

Figures

Figures reproduced from arXiv: 2504.18580 by the authors.

Figure 1
Figure 1. Merged checkpoints scored on GSM Weighted Average [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Merged checkpoints scored on Alignment Weighted Av [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Merged checkpoints scored on OpenHermes-2.5 vali [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of Baseline and Metrics-Weighted Aver [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 2 canonical work pages

  1. [1]

    Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities, 2024

    Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities, 2024. URL https://arxiv.org/abs/2408.07666

    arXiv 2024

  2. [2]

    Arcee's mergekit: A toolkit for merging large language models, 2025

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee's mergekit: A toolkit for merging large language models, 2025. URL https://arxiv.org/abs/2403.13257

    arXiv 2025

  3. [3]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba...

    work page 2022

  4. [4]

    Rae, Oriol Vinyals, and Laurent Sifre

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre...

    arXiv 2022

  5. [5]

    Merging models with fisher-weighted averaging, 2022

    Michael Matena and Colin Raffel. Merging models with fisher-weighted averaging, 2022. URL https://arxiv.org/abs/2111.09832

    arXiv 2022

  6. [6]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA : Low - Rank Adaptation of Large Language Models , October 2021. URL http://arxiv.org/abs/2106.09685. arXiv:2106.09685 [cs]

    arXiv 2021

  7. [7]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=6t0Kwf8-jrj

    2023

  8. [8]

    Checkpoint Merging via Bayesian Optimization in LLM Pretraining , March 2024 a

    Deyuan Liu, Zecheng Wang, Bingning Wang, Weipeng Chen, Chunshan Li, Zhiying Tu, Dianhui Chu, Bo Li, and Dianbo Sui. Checkpoint Merging via Bayesian Optimization in LLM Pretraining , March 2024 a . URL http://arxiv.org/abs/2403.19390. arXiv:2403.19390 [cs]

    arXiv 2024

Show all 23 references
  1. [9]

    Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment, 2023

    Lingling Xu, Haoran Xie, Si-Zhao Joe Qin, Xiaohui Tao, and Fu Lee Wang. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment, 2023. URL https://arxiv.org/abs/2312.12148

    2023 arXiv

  2. [10]

    D o RA : Weight-decomposed low-rank adaptation

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. D o RA : Weight-decomposed low-rank adaptation. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Fel...

    2024

  3. [11]

    Qlora: Efficient finetuning of quantized llms, 2023

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms, 2023. URL https://arxiv.org/abs/2305.14314

    2023 arXiv

  4. [12]

    Instruction Tuning for Large Language Models : A Survey , March 2024

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. Instruction Tuning for Large Language Models : A Survey , March 2024. URL http://arxiv.org/abs/2308.10792. arXiv:2308.10792 [cs]

    2024

  5. [13]

    SimPO : Simple Preference Optimization with a Reference - Free Reward , July 2024

    Yu Meng, Mengzhou Xia, and Danqi Chen. SimPO : Simple Preference Optimization with a Reference - Free Reward , July 2024. URL http://arxiv.org/abs/2405.14734. arXiv:2405.14734 [cs]

    2024 arXiv

  6. [14]

    Modern Distributed Data - Parallel Large - Scale Pre -training Strategies For NLP models, June 2022

    Hao Bai. Modern Distributed Data - Parallel Large - Scale Pre -training Strategies For NLP models, June 2022. URL http://arxiv.org/abs/2206.06356. arXiv:2206.06356 [cs]

    2022 arXiv

  7. [15]

    Gemma Team , Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Ale...

    2024 arXiv

  8. [16]

    Orca- Math : Unlocking the potential of SLMs in Grade School Math , February 2024

    Arindam Mitra, Hamed Khanpour, Corby Rosset, and Ahmed Awadallah. Orca- Math : Unlocking the potential of SLMs in Grade School Math , February 2024. URL http://arxiv.org/abs/2402.14830. arXiv:2402.14830 [cs]

    2024 arXiv

  9. [17]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback , April 2022

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan H...

    2022 arXiv

  10. [18]

    Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023

    Teknium. Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023. URL https://huggingface.co/datasets/teknium/OpenHermes-2.5

    2023

  11. [19]

    Training Verifiers to Solve Math Word Problems , November 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems , November 2021. URL http://arxiv.org/abs...

    2021 arXiv

  12. [20]

    GSM - Plus : A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers , July 2024

    Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. GSM - Plus : A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers , July 2024. URL http://arxiv.org/abs/2402.19255. arXiv:2402.19255 [cs]

    2024 arXiv

  13. [21]

    ToxiGen : A Large - Scale Machine - Generated Dataset for Adversarial and Implicit Hate Speech Detection , July 2022

    Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. ToxiGen : A Large - Scale Machine - Generated Dataset for Adversarial and Implicit Hate Speech Detection , July 2022. URL http://arxiv.org/abs/2203.09509. arXiv:2203.09509 [cs]

    2022 arXiv

  14. [22]

    TruthfulQA : Measuring How Models Mimic Human Falsehoods , May 2022

    Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA : Measuring How Models Mimic Human Falsehoods , May 2022. URL http://arxiv.org/abs/2109.07958. arXiv:2109.07958 [cs]

    2022 arXiv

  15. [23]

    TIES - Merging : Resolving Interference When Merging Models , October 2023

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES - Merging : Resolving Interference When Merging Models , October 2023. URL http://arxiv.org/abs/2306.01708. arXiv:2306.01708 [cs]

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.