Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning an LLM on its own judged-correct responses improves task accuracy and roughly halves the generalization loss caused by standard supervised fine-tuning.

desk verdict S3FT is a plausible selection step on top of SDFT, but single-run numbers make the generalization gains provisional. read the letter →

arxiv 2502.08130 v2 pith:D6AI3WW4 submitted 2025-02-12 cs.CL

Sonam Gupta , Yatin Nandwani , Asaf Yehudai , Dinesh Khandelwal , Dinesh Raghu , Sachindra Joshi This is my paper · ORCID
classification cs.CL
keywords selectiveself-to-supervisedfine-tuningcatastrophicforgettingsupervisedself-generatedtrainingtargetsgeneralizationbenchmarksLLMjudgedistributionshiftreadingcomprehension
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Standard supervised fine-tuning (SFT) on a task like math or code raises in-domain accuracy but degrades the model's general abilities, by an average of up to 4.4 points on four broad benchmarks in this paper's experiments. The paper proposes Selective Self-to-Supervised Fine-Tuning (S3FT), which replaces gold-only training targets with the model's own responses whenever a judge confirms those responses are correct, and with a model-generated paraphrase of the gold response otherwise. On GSM8K, MBPP, and a Natural Questions variant, S3FT beats standard SFT on the fine-tuning task itself while roughly halving the average drop on MMLU, TruthfulQA, HellaSwag, and WinoGrande. If correct, this gives a practical recipe for task-specific fine-tuning that preserves general capabilities without needing an instruction-tuning replay buffer.

What carries the argument

The load-bearing component is the judge that decides equivalence between a model-generated response (or a paraphrase of the gold) and the gold answer. In the paper the judge is a heuristic for math (final-integer match with about 96 percent human-validated accuracy) and code (passing test cases), and an LLM-as-judge for reading comprehension (about 86 percent validated accuracy on a 200-sample set). The judge's verdicts determine which of three target types enters the training set: the model's own response, a model-paraphrased gold, or the gold itself, so the training distribution stays as close as possible to the model's native distribution.

What would settle it

A controlled experiment that introduces a known false-accept rate into the judge (for example, randomly flipping 20 percent of the judge's 'correct' decisions to 'incorrect' on GSM8K and NQ) and checks whether S3FT's advantage over standard SFT shrinks or disappears would settle how much of the gain depends on judge accuracy. A second check: if S3FT's benefit is distributional, its advantage should vanish when the base model's correct-response rate is near zero or near one on the fine-tuning set.

Watch

Extended reading notes

Core claim

S3FT's central claim is that the distribution of gold responses often lies far from the model's own response distribution, so training exclusively on gold targets pushes the model away from its pretrained behavior and causes catastrophic forgetting. By first generating a response with the base model and using an equivalence judge to decide whether it is correct, S3FT trains on self-generated correct responses wherever possible; for incorrect model responses it paraphrases the gold through the base model and uses that paraphrase if a judge validates it, falling back to the gold only when neither passes. The paper reports that this selection keeps the fine-tuned model closer to its original distribution while still learning the target task, yielding better in-domain accuracy than SFT and smaller accuracy drops on general benchmarks.

Load-bearing premise

The method assumes the judge reliably identifies which model-generated responses are truly correct; if the judge systematically accepts wrong responses as training targets, the reported gains could be an artifact of fitting the judge's mistakes rather than evidence of better task competence.

Editorial extensions

If this is right

  • On the three fine-tuning tasks used here—GSM8K, MBPP, and the NQ reading-comprehension variant—S3FT exceeds standard SFT in in-domain accuracy.
  • S3FT's average benchmark drop is about 2.5 points when trained on GSM8K or NQ and about 1.0 when trained on MBPP, versus 4.4, 2.7, and 5.8 for standard SFT, respectively.
  • Because most training targets in S3FT are the model's own validated responses (at least 30% directly, and over 50% when paraphrases are counted), the model can be fine-tuned without access to the original instruction-tuning dataset for replay.
  • S3FT's extra cost is inference and judging time, not stored data or additional trained models, so it scales to settings where replay buffers are unavailable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If judge fidelity is the bottleneck, S3FT's benefit should extend to open-ended generation tasks such as summarization and translation as LLM judges improve, which the paper's limitation section already hints at.
  • The distribution-matching story suggests a testable prediction: S3FT's advantage over SFT should be largest when the base model's correct-response rate is moderate, since then the training set mixes distributions; at extreme rates the method degenerates toward either pure self-training or pure gold training.
  • Because the paper uses greedy decoding for the base model's responses, sampling multiple responses and picking a correct one closer to the model's distribution—as the authors mention for future work—could push the generalization gains further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Selective Self-to-Supervised Fine-Tuning (S3FT), a fine-tuning recipe for LLMs that replaces gold training targets with the base model's own responses when those responses are judged correct, and otherwise uses a paraphrased gold response or the gold response itself. The method is evaluated by fine-tuning Mistral-7B-Instruct-v0.2 on GSM8K, MBPP, and a variant of Natural Questions, measuring in-domain accuracy as well as generalization on MMLU, TruthfulQA, HellaSwag, and WinoGrande. The authors report that S3FT outperforms standard SFT and the SDFT baseline on the fine-tuning tasks while losing less general capability, with average generalization drops of 2.5, 1.0, and 2.5 points depending on the training task, versus 4.4, 2.7, and 5.8 points for SFT.

Significance. If the reported gains are reproducible, S3FT is a simple and inexpensive way to reduce catastrophic forgetting during task-specific fine-tuning, with the practical advantage of not requiring an instruction-tuning replay buffer. Strengths include the clear formulation, documented prompts and hyperparameters, objective correctness checks for GSM8K and MBPP, and a human study quantifying judge accuracy. The main limitations are that all conclusions rest on single-run greedy evaluations with no variance estimates, and that the Natural Questions judge shares the base model's model family; the evidence therefore does not yet support the abstract's strength of claim.

major comments (3)
  1. [Section 4, Tables 2 and 3, Appendix A.3] The central comparison rests entirely on single-run point estimates. Appendix A.3 states that greedy decoding was always used and that a single run of the evaluation was performed. In Table 3, several differences are small: MMLU is 57.0 versus 58.2 for GSM8K-trained SFT versus S3FT, WinoGrande is 73.4 versus 73.5, and HellaSwag is 62.4 versus 63.2. Without multiple seeds or a measure of variability, the claim that S3FT achieves better performance than SFT while improving generalization is not statistically supported. Please add at least three to five seeds with means and standard deviations, or rephrase the conclusions as observations that require replication.
  2. [Section 3 and Appendix A.5] The NQ results rely on a Mistral-instruct-v2 (7B) judge, which belongs to the same model family as the base model and is reported to be 86% accurate on 200 samples. Because the same judge is used both to select S3FT training targets and to evaluate the test responses, systematic judge errors could favor S3FT's distribution-matched outputs in a way that does not reflect genuine task competence. Please quantify the sensitivity of the NQ results to judge errors, for example by reporting performance on the human-annotated 200-sample subset or by comparing the same models with a different judge.
  3. [Section 4, paragraph beginning 'We observe that irrespective...'] The sentence reporting the average drops is internally inconsistent: it says 'S3FT results in an average drop of only 2.5 when trained on GSM8K and MBPP and a drop of 1.0 when trained on MBPP.' According to Table 3, the average drops are 2.5 for GSM8K, 1.0 for MBPP, and 2.5 for NQ. Please correct the sentence and recheck all reported averages, since the abstract's 'reduces this drop by half' claim depends on these numbers.
minor comments (5)
  1. [Table 2 caption] The caption says 'over two different tasks' but the table reports results for three datasets; please correct the caption.
  2. [Appendix A.4, Algorithm 1] In line 8 of the algorithm, the gold paraphrase is written as M_theta0(y_i), but the surrounding text and Figure 1 condition the paraphrase on the input x_i; please correct the notation to make the conditioning explicit.
  3. [Appendix A.3] The GSM8K evaluation is described as matching the last number of the predicted response with the last number of the gold response, while Section 3 describes it as matching the final answer; please clarify whether the same final-integer matching is used for both the training-data judge and the test evaluation, since this affects reproducibility.
  4. [Figure 2] The caption states that the distribution is based on 84 examples from the MBPP training data, but it does not describe how these examples were sampled or how the 'acceptable' responses were identified; please add this information.
  5. [Section 3, Baselines] The paper compares against SDFT but does not state in the appendix whether SDFT used the same LoRA settings, training steps, and validation procedure; please provide these details for a fair comparison.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity; generalization is tested on external benchmarks. Only mild self-referentiality: the NQ judge is the same model family that curates the S3FT training targets.

  1. other [Section 3 (Experimental Setup, NQ evaluation); Section A.5 (Judges); Section 2 (Proposed Method, judge-based training-set curation)]
    "For NQ dataset, we employ Mistral-instruct-v2 (7B) as a judge. ... We experiment with Mistral-instruct-v2 (7B) (Jiang et al., 2023) as our base model."

    In Section 2, S3FT uses a judge to decide whether the base model's response is correct, and correct responses become training targets. For NQ, that judge is Mistral-instruct-v2 (7B), the same model as the base model, and Section 3 states that NQ correctness is also evaluated with Mistral-instruct-v2 (7B). Therefore the NQ in-domain score is not independent of the training-data filter: the fine-tuned model is optimized toward outputs that this specific judge approves, and then scored by the same judge. Part of the reported NQ improvement over SFT is, by construction, alignment with this judge.

full rationale

S3FT's central generalization claim is not circular: the Table 3 benchmarks (MMLU, TruthfulQA, HellaSwag, WinoGrande) are fixed, external tasks, and the method never fits any parameter or target selection to them. No equation in the paper reduces the reported performance drop to the method's inputs. The self-citation to Nandwani et al. (2020) for the 1oML framing is not load-bearing, since Table 1 independently demonstrates multiple valid responses with log-probability evidence. The only self-referential element is the NQ setup: the same Mistral-instruct-v2 (7B) judge model that filters which base-model responses become S3FT training targets is also used to score NQ correctness, so the in-domain NQ numbers partly measure agreement with that judge. This mild loop does not affect the generalization conclusion, which rests on the external benchmarks. The Section 4 sentence reporting drops ('2.5 when trained on GSM8K and MBPP and a drop of 1.0 when trained on MBPP') appears to contain a typo and should be treated as a consistency issue, not circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The method introduces no invented physical or mathematical entities. Its load-bearing assumptions are judge reliability, the closeness of model responses to the model's distribution, and single-model generality; the only chosen numerical inputs are standard LoRA/optimization hyperparameters.

free parameters (1)
  • LoRA and optimization settings = rank 8, scaling 16, dropout 0.1, learning rate 1e-4, 5000 steps (GSM8K/NQ) or 1000 steps (MBPP)
    Selected by hand without a reported sensitivity study; the paper's numbers could depend on these choices, though they are standard.
assumptions (4)
  • domain assumption The judge used to filter training examples correctly identifies which model responses are valid targets
    S3FT commits to using a judged-correct model response as a target; if the judge is wrong, incorrect text enters training. Human checks report 96 percent on 50 GSM8K samples and 86 percent on 200 NQ samples (Section 3).
  • domain assumption A response can be considered correct based on the final answer, test cases, or an LLM judge, without requiring full reasoning quality
    For GSM8K, last-number matching is used to certify correctness (Appendix A.3); a response with the right final integer but wrong reasoning would be accepted as a target.
  • domain assumption The model's own correct responses are closer to its native distribution and therefore safer to train on than gold responses
    This is the motivating hypothesis of the method, supported only by a small 84-example histogram (Figure 2), not by a controlled test.
  • domain assumption Results from Mistral-7B-Instruct-v2 extend to other LLMs
    All experiments use a single 7B instruction-tuned model, while the abstract claims behavior for LLMs generally; cross-model generality is untested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models." pith.science (2026). https://pith.science/paper/D6AI3WW4

@misc{pith2026250208130,
  author       = {Pith},
  title        = {Pith review of: Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D6AI3WW4}},
  note         = {Machine review of arXiv:2502.08130}
}
abstract

Fine-tuning Large Language Models (LLMs) on specific datasets is a common practice to improve performance on target tasks. However, this performance gain often leads to overfitting, where the model becomes too specialized in either the task or the characteristics of the training data, resulting in a loss of generalization. This paper introduces Selective Self-to-Supervised Fine-Tuning (S3FT), a fine-tuning approach that achieves better performance than the standard supervised fine-tuning (SFT) while improving generalization. S3FT leverages the existence of multiple valid responses to a query. By utilizing the model's correct responses, S3FT reduces model specialization during the fine-tuning stage. S3FT first identifies the correct model responses from the training set by deploying an appropriate judge. Then, it fine-tunes the model using the correct model responses and the gold response (or its paraphrase) for the remaining samples. The effectiveness of S3FT is demonstrated through experiments on mathematical reasoning, Python programming and reading comprehension tasks. The results show that standard SFT can lead to an average performance drop of up to $4.4$ on multiple benchmarks, such as MMLU and TruthfulQA. In contrast, S3FT reduces this drop by half, i.e. $2.5$, indicating better generalization capabilities than SFT while performing significantly better on the fine-tuning tasks.

Figures

Figures reproduced from arXiv: 2502.08130 by the authors.

Figure 1
Figure 1. An overview of our proposed approach: Given the input xi and its corresponding gold response yi , we employ the base model Mθ to transform yi such that it is correct but at the same time closer to model’s distribution. First, the model predicts the output yˆ. The judge decides whether the yi is correct. If true, it becomes part of the training dataset; otherwise, we paraphrase ([xi ; yi ]) to obtain y˜i . The judge … view at source ↗
Figure 2
Figure 2. Histogram of the log probability assigned by [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Prompt used for training the model on GSM8K [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Prompt used for predicting the base model’s [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Prompt used for predicting the base model’s [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Prompt used for paraphrasing the gold response of training partition of the MBPP dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Prompt used for paraphrasing the gold response of training partition of the GSM8K dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: The prompt used for judging the correctness of the responses generated by the model for NQ dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Prompt used for paraphrasing the gold re [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning

    cs.SE 2025-07 conditional novelty 6.0 of 10

    CodeReasoner combines a concise execution-focused dataset, instruction tuning, and GRPO RL to make 7B/14B models match or beat GPT-4o on code reasoning benchmarks.

Reference graph

Works this paper leans on

31 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732

    arXiv 2021

  4. [4]

    Sher Badshah and Hassan Sajjad. 2024. Reference-guided verdict: Llms-as-judges in automatic evaluation of free-form text, 2024. URL https://arxiv. org/abs/2408.09235

    arXiv 2024

  5. [5]

    Loubna Ben Allal, Niklas Muennighoff, Logesh Kumar Umapathi, Ben Lipkin, and Leandro von Werra. 2022. A framework for the evaluation of code generation models. https://github.com/bigcode-project/bigcode-evaluation-harness

    2022

  6. [6]

    Sarthak Bhat, Oleg Sidorov, Ulrich Paquet, and Anirudh Garg. 2022. Representation consolidation for continual learning. In International Conference on Learning Representations

    work page 2022

  7. [7]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  8. [8]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

Show all 31 references
  1. [9]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024 doi

  2. [10]

    Tyler L Hayes, Kushal Kafle, Robik Shrestha, Manoj Acharya, and Christopher Kanan. 2020. Remind your neural network to prevent catastrophic forgetting. In European Conference on Computer Vision, pages 466--483. Springer

    2020

  3. [11]

    Junxian He, Jiatao Gu, Jianfeng Shen, and Marc'Aurelio Ranzato. 2020. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations

    2020

  4. [12]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. In International Conference on Learning Representations

    2020

  5. [13]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  6. [14]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  7. [15]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for C...

    2019

  8. [16]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th An...

    2020

  9. [17]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA : Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214--3252

    2022

  10. [18]

    Tanya Mok, Luisa Wellhausen, Hyung Won Choe, and Hannaneh Hajishirzi. 2023. Large language models can be continuously updated without forgetting. arXiv preprint arXiv:2303.01926

    2023 arXiv

  11. [19]

    Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, C aglar Gul c ehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Conference on Computational Natural Language Learning. Association for Computational Linguistics (ACL)

    2016

  12. [20]

    Yatin Nandwani, Deepanshu Jindal, Parag Singla, et al. 2020. Neural learning of one-of-many solutions for combinatorial problems in structured output spaces. In International Conference on Learning Representations

    2020

  13. [21]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383--2392

    2016

  14. [22]

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. In Advances in Neural Information Processing Systems, volume 32

    2019

  15. [23]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. Winogrande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641

    2019 arXiv

  16. [24]

    Thomas Scialom, Thierry Charnois, and Sylvain Lamprier. 2022. Continual learning for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5432--5442

    2022

  17. [25]

    Aviv Slobodkin, Omer Goldman, Avi Caciularu, Ido Dagan, and Shauli Ravfogel. 2023. The curious case of hallucinatory (un) answerability: Finding truths in the hidden states of over-confident large language models. In Proceedings of the 2023 Conference on Empirical Methods in N...

    2023

  18. [26]

    Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. 2020. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10687--10698

    2020

  19. [27]

    Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024. Self-distillation bridges distribution gap in language model fine-tuning. arXiv preprint arXiv:2402.13669

    2024 arXiv

  20. [28]

    Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor, Tomas Kocisky, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, et al. 2019. Learning and evaluating general linguistic intelligence. arXiv preprint arXiv:1901.11373

    2019 arXiv

  21. [29]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791--4800

    2019

  22. [30]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2021. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations

    2021

  23. [31]

    Zijun Zhang, Yue Wu, Hao Guan, Xinlei Chen, and Yue Zhang. 2023. Continual learning with transformers: Challenges and solutions. arXiv preprint arXiv:2302.13713

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.