Pith. sign in

REVIEW 6 major objections 6 minor 38 references

Feeding LLM Annotations to BERT Classifiers at Your Own Risk

T0 review · 6 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning BERT on LLM-generated labels degrades accuracy and, more importantly, makes training unstable and prone to early plateaus.

desk verdict Core finding is real, but the instability claim overreaches and the plateau evidence is thin. read the letter →

arxiv 2504.15432 v1 pith:W3YOEYWL submitted 2025-04-21 cs.CL

Yucheng Lu , Kazimier Smith This is my paper · ORCID
classification cs.CL
keywords LLM-generatedlabelssyntheticdatatextclassificationfine-tuningRoBERTapredictioninstabilityperformanceplateauentropyfiltering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that using LLM-generated labels to fine-tune smaller classifiers such as RoBERTa costs more than accuracy: it makes training runs unpredictable and causes learning to plateau early. These effects appear across four text-classification datasets, are worst on complex or imbalanced tasks, and persist when the annotating LLM grows from 3B to 7B parameters. The authors offer a theoretical explanation based on the divergence between the LLM's label distribution and the true label distribution, and test simple mitigations that only partially close the gap. A sympathetic reader would take this as a caution that cheap LLM annotation is not a safe substitute for human labels in high-stakes applications.

What carries the argument

The central analytic object is the bias/variance-style decomposition of the classifier's expected KL error: $$\text{Error}(\hat P) \approx \mathrm{KL}(P\,\|\,P_S) + \mathbb{E}_{D_S}\left[\mathrm{KL}(P_S\,\|\,\hat P)\right].$$ The first term, the irreducible approximation error from using the LLM's conditional label distribution, is what produces the plateau; the second explains why variance inflates where the LLM is off. On the mitigation side, the paper tests an entropy-ranked filtering function $F(x)$ that mixes gold labels into high-uncertainty inputs, and a consistency ensemble that replaces flip-prone predictions, recovering 60–75% of the performance gap.

What would settle it

Train gold-label and LLM-label RoBERTa models on the same four datasets with training sizes beyond 5000 (for example 10k and 20k) and compare the slopes of their learning curves; if the gold-label curve continues improving while the synthetic-label curve flattens, the plateau stands, and if both flatten together it does not.

Watch

Extended reading notes

Core claim

The paper claims that fine-tuning a smaller encoder-only text classifier (RoBERTa-base) on LLM-generated labels, rather than human gold labels, degrades more than accuracy: runs become less reproducible (Krippendorff's alpha drops, predictions flip across seeds), and learning curves flatten prematurely as training data grows. These effects intensify with task complexity and class imbalance, and scaling the annotator from 3B to 7B parameters does not remove the instability. The authors attribute the pattern to irreducible approximation error in the LLM's label distribution $P_S(Y|X)$ that no amount of training data can average away, with estimation error amplified wherever $P_S$ diverges from the true $P$.

Load-bearing premise

The premature plateau result depends on the decision to cap every training set at 5000 data points; if the gold-label baseline also plateaus by that budget, the seeming plateau is an artifact of the cap rather than a failure mode of LLM labels.

Editorial extensions

If this is right

  • On complex or imbalanced datasets, LLM labeling can cause large accuracy drops (for example 66% versus 96% on Ecommerce with the 3B annotator), and scaling to 7B does not reliably close them.
  • Training instability is not a small-model artifact: Krippendorff's alpha and unchanged-prediction rates drop even on simple sentiment data, so a single fine-tuning run with LLM labels is not a trustworthy estimate.
  • Entropy-based filtering and consistency ensembles recover only 60–75% of the gold-label performance gap and do not restore training stability, so post-hoc fixes are not a full remedy.
  • Minority classes are systematically under-learned with synthetic labels, a pattern the paper reads as a mild form of model collapse during synthetic-data training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the plateau generalizes beyond the 5000-point cap, then acquiring more unlabeled data will not rescue a synthetic-label pipeline; the ceiling is set by annotator bias, not sample size.
  • The paper's own error decomposition suggests a testable hierarchy: label-only synthetic data should beat synthetic text-plus-label pairs for the same data budget, a comparison the paper notes but does not run.
  • Because entropy filtering failed worst on the Toxic dataset, LLM confidence scores may be least trustworthy exactly where the annotation decision is hardest; uncertainty-based fixes need calibration checks per dataset.
  • A practical extension would be to adopt instability metrics like Krippendorff's alpha and the proportion of unchanged predictions as standard diagnostics in any LLM-annotation pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. This paper empirically studies the use of LLM-generated labels (Qwen2.5-Instruct 3B/7B) to fine-tune RoBERTa-base classifiers on four text classification datasets (IMDB, ECommerce, Manifestos, Toxic). It reports that models trained on synthetic labels suffer accuracy/F1 degradation, increased prediction instability as measured by Krippendorff's alpha and the proportion of unchanged predictions, and premature performance plateaus, relative to models trained on gold labels. The paper proposes a bias-variance decomposition to interpret the degradation, and evaluates two mitigation strategies: entropy-based filtering and consistency ensembles. It concludes that these mitigations only partially recover the gap and do not fully resolve the instability and plateau issues.

Significance. If correct, the paper provides a useful empirical cautionary study that goes beyond simple accuracy comparisons, and its comparison against gold labels avoids circularity. The theoretical interpretation uses a standard Hessian-based bias-variance decomposition without fitted free parameters, and the paper gives detailed prompt and training protocols. However, the significance is limited by the absence of significance tests, the small number of runs (five), the borrowed Figure 1, and several internal inconsistencies in the reported stability and mitigation results, which currently weaken the headline claims.

major comments (6)
  1. [Section 2.2 and Table 1] The universal claim of "increased instability" is contradicted by the variance measures on IMDB, where gold σacc=0.28 and σf1=0.29 are higher than both synthetic conditions (3B: 0.16/0.16; 7B: 0.20/0.20). Since Section 2.2 defines stability via these variances, the paper must either restrict the instability claim to the αK/puc measures or explain why variance is not the operative stability measure on this dataset.
  2. [Table 1, Manifestos 3B row] σacc=0.00 with five runs is implausible given αK=52.72 and puc=50, which imply substantial prediction flips. Please verify the data and report the actual run-level values; if correct, explain how zero accuracy variance coexists with only 50% prediction overlap.
  3. [Section 3, Figures 1–2] The premature plateau claim lacks support. Figure 1 is borrowed from Csanady et al. (2024) and shows "LlamBERT" rather than the RoBERTa models used here, and Figure 2 has no error bars or significance tests. Because training is capped at 5,000 examples (Appendix A), the plateau may be an artifact of the cap; provide data beyond the cap or a statistical test (e.g., comparing learning-curve slopes) to show synthetic labels plateau before gold labels.
  4. [Section 1 vs Table 2] The introduction claims that "neither strategy stabilizes training variance," but the consistency ensemble reduces σacc and σf1 on all four datasets (e.g., Toxic σacc from 5.31 to 0.69; IMDB from 0.20 to 0.02). Please correct this misstatement or qualify it with respect to the gold baseline.
  5. [Table 2, IMDB Ensemble row] The reported µf1=83.45 is 10 points below µacc=93.46 on a balanced binary task and is far below the synthetic-label µf1=92.94, which would imply the ensemble drastically reduces macro-F1. If this is a typo (e.g., 93.45), fix it; otherwise, the claim that the ensemble "seems to work" is not supported on this dataset.
  6. [Section 1 and Table 2] The abstract's claim that mitigations "recover only 60–75% of the gold-label performance gap" is not supported by the table: on Manifestos the ensemble recovers roughly (81.58−71.51)/(83.56−71.51) ≈ 84% of the accuracy gap. Please either recompute the range or adjust the claim.
minor comments (6)
  1. [Figure 1 caption] The caption says "Plot from Csanady et al. (2024)" and the figure legend refers to "LlamBERT," not the RoBERTa models used in this paper; clarify the provenance and relation to the present experiments, or replace with the paper's own data.
  2. [Table 1 caption] "All numbers are scaled up by 100" is ambiguous for αK and puc, which are already percentages; state the units for each metric explicitly.
  3. [Section 3.1, Eq. (1)] The approximation in the bias-variance decomposition is stated without justification; cite the Hessian-based expansion in Heskes (1998) and note its validity conditions.
  4. [Section 5, first paragraph] The sentence "entropy-based filtering is theoretical more appealing simple alternative simple uncertainty estimation heuristics" is ungrammatical and should be rewritten for clarity.
  5. [Section 2.1] "as representatives LLMs in their respective weight classes" should be "as representative LLMs".
  6. [Appendix A] The statement "shrink the size of test datasets with <= 2000" is unclear; specify how many samples were used for each test set.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: central claims rest on external gold-label comparisons and a standard bias-variance decomposition; internal inconsistencies are robustness issues, not circular reasoning.

full rationale

The paper's central finding is an empirical comparison between classifiers fine-tuned on gold labels and classifiers fine-tuned on LLM-generated labels, evaluated on held-out gold labels across four datasets. The comparison target is external human annotation, so the main performance-degradation and instability claims are not defined in terms of the synthetic-label input or the paper's own fitted quantities. The theoretical interpretation in Section 3.1 applies a standard bias/variance decomposition from Heskes (1998) to write the expected KL error as KL(P||PS) plus an estimation-error term; the claim that the first term is an irreducible approximation error follows mathematically from the decomposition and is not a restatement of the conclusion. The mitigation experiments in Sections 4 and 5 intervene on the training-label construction (entropy-ranking filtering and consistency ensembles) and report the resulting classifier metrics on the same gold-labeled test sets; the outcomes are empirical and could have gone the other way, so they are not forced by construction. There are no load-bearing self-citations: the cited related work (e.g., Csanady et al., Pangakis & Wolken, Heskes) is external to the present authors, and none of the paper's premises depend on an unverified uniqueness theorem or ansatz imported from the authors' own prior work. Some internal inconsistencies exist, notably Table 1 showing lower sigma_acc for synthetic labels than gold on IMDB (gold 0.28 versus 3B 0.16 and 7B 0.20) despite the abstract's blanket claim of increased instability, and the suspicious sigma_acc = 0.00 for Manifestos 3B. These undermine the robustness of one headline claim, but they are correctness or reporting issues, not circularity. The premature-plateau conclusion also depends on the 5000-point training cap stated in the Figure 1 caption, which is a limitation of the evidence rather than a circular derivation. Overall, the paper's derivation chain is self-contained against external benchmarks and does not reduce to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. The central claim rests on the assumption that gold labels are error-free and that the LLM annotation process can be described as a fixed conditional distribution P_S(Y|X). The mitigation results depend on two unreported or underspecified parameters: the entropy threshold and the ensemble size.

free parameters (2)
  • entropy filtering percentile alpha = 5% or 25%, exact value for Table 2 not reported
    Hand-chosen threshold that determines how many synthetic labels are replaced with gold labels; affects all mitigation results in Table 2.
  • number of consistency-ensemble runs = not reported
    The ensemble protocol replaces flipped predictions with human labels, but the number of repeated annotations and the agreement rule are not specified, so the mitigation outcome depends on an unreported parameter.
assumptions (4)
  • domain assumption Gold labels are treated as ground truth P(Y|X) with no annotation noise.
    Section 2.2 and Table 1 compare synthetic labels against gold labels; if gold labels are noisy, the measured gap is not purely LLM error.
  • domain assumption LLM-generated labels are i.i.d. samples from P(X)P_S(Y|X) with a fixed prompt and demonstration set.
    Section 3.1 defines D_S ~ P(X)P_S(Y|X); the three-shot prompt template in Appendix C is fixed, but demonstrations are randomly drawn, so P_S is not actually fixed across samples.
  • standard math The Heskes (1998) bias-variance decomposition applies to the cross-entropy training objective.
    Section 3.1 invokes the decomposition to separate irreducible approximation error from estimation error; the paper does not derive the conditions under which the approximation holds.
  • domain assumption Model collapse as described by Shumailov et al. (2024b) is applicable to single-generation synthetic label training.
    Section 3.1 interprets minority-class underperformance as a mild form of model collapse, but the cited collapse mechanism is about recursively generated data over generations, not a single LLM labeling pass.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Feeding LLM Annotations to BERT Classifiers at Your Own Risk." pith.science (2026). https://pith.science/paper/W3YOEYWL

@misc{pith2026250415432,
  author       = {Pith},
  title        = {Pith review of: Feeding LLM Annotations to BERT Classifiers at Your Own Risk},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W3YOEYWL}},
  note         = {Machine review of arXiv:2504.15432}
}
read the original abstract

Using LLM-generated labels to fine-tune smaller encoder-only models for text classification has gained popularity in various settings. While this approach may be justified in simple and low-stakes applications, we conduct empirical analysis to demonstrate how the perennial curse of training on synthetic data manifests itself in this specific setup. Compared to models trained on gold labels, we observe not only the expected performance degradation in accuracy and F1 score, but also increased instability across training runs and premature performance plateaus. These findings cast doubts on the reliability of such approaches in real-world applications. We contextualize the observed phenomena through the lens of error propagation and offer several practical mitigation strategies, including entropy-based filtering and ensemble techniques. Although these heuristics offer partial relief, they do not fully resolve the inherent risks of propagating non-random errors from LLM annotations to smaller classifiers, underscoring the need for caution when applying this workflow in high-stakes text classification tasks.

Figures

Figures reproduced from arXiv: 2504.15432 by the authors.

Figure 1
Figure 1. Performance of RoBERTa trained on ”Gold labels” vs on synthetic labels (”LlamBERT”). [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Performance as data point increases 3.1 Theoretical Interpretation Framework Denote the true data generating process of text and label pair as the joint distribution P(Y, X), where Y is label/class, and X is input text. The supervised text classifier is trained to estimate the conditional distribution P(Y|X) from i.i.d. sample DP = {(yi , xi) N i=1 } by minimizing cross-entropy loss: LCE(θ, DP) = − 1 N N ∑ i=1 log P… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 8 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Comprehensive exploration of synthetic data generation: A survey, 2024

    André Bauer, Simon Trapp, Michael Stenger, Robert Leppich, Samuel Kounev, Mark Leznik, Kyle Chard, and Ian Foster. Comprehensive exploration of synthetic data generation: A survey, 2024. URL https://arxiv.org/abs/2401.02524

    arXiv 2024

  3. [3]

    Pathologies of Pre-trained Language Models in Few-shot Fine-tuning

    Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah, and Yangfeng Ji. Pathologies of pre-trained language models in few-shot fine-tuning, 2022. URL https://arxiv.org/abs/2204.08039

    work page Pith review arXiv 2022

  4. [4]

    A survey on large language models for critical societal domains: Finance, healthcare, and law, 2024

    Zhiyu Zoey Chen, Jing Ma, Xinlu Zhang, Nan Hao, An Yan, Armineh Nourbakhsh, Xianjun Yang, Julian McAuley, Linda Petzold, and William Yang Wang. A survey on large language models for critical societal domains: Finance, healthcare, and law, 2024. URL https://arxiv.org/abs/2405.01769

    arXiv 2024

  5. [5]

    Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf, Dominic Culver, Rui Melo, Caio Corro, Andre F. T. Martins, Fabrizio Esposito, Vera Lúcia Raposo, Sofia Morgado, and Michael Desa. Saullm-7b: A pioneering large language model for law, 2024. URL https://arxiv.org/abs/2403.03883

    arXiv 2024

  6. [6]

    Llambert: Large-scale low-cost data annotation in nlp, 2024

    Bálint Csanády, Lajos Muzsai, Péter Vedres, Zoltán Nádasdy, and András Lukács. Llambert: Large-scale low-cost data annotation in nlp, 2024. URL https://arxiv.org/abs/2403.15938

    arXiv 2024

  7. [7]

    Automated hate speech detection and the problem of offensive language, 2017

    Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language, 2017. URL https://arxiv.org/abs/1703.04009

    arXiv 2017

  8. [8]

    Deep Learning for Economists

    Melissa Dell. Deep learning for economists, 2024. URL https://arxiv.org/abs/2407.15339

    work page Pith review arXiv 2024

Show all 38 references
  1. [9]

    Calibration of pre-trained transformers

    Shrey Desai and Greg Durrett. Calibration of pre-trained transformers. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 295--302, Online, November 2020. Associatio...

    2020 doi

  2. [10]

    BERT : Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT : Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North A merican Chapter of...

    2019 doi

  3. [11]

    Text clustering applied to unbalanced data in legal contexts

    Lucas Jos \'e Gon c alves Freitas. Text clustering applied to unbalanced data in legal contexts. In Pablo Gamallo, Daniela Claro, Ant \'o nio Teixeira, Livy Real, Marcos Garcia, Hugo Gon c alo Oliveira, and Raquel Amaro (eds.), Proceedings of the 16th International Conference ...

    2024

  4. [12]

    Gallegos, Ryan A

    Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. Bias and fairness in large language models: A survey, 2024. URL https://arxiv.org/abs/2309.00770

    2024 arXiv

  5. [13]

    A. Gautam. E commerce text dataset (version - 2), 2019. URL https://doi.org/10.5281/zenodo.3355823

    2019 doi

  6. [14]

    Fabricator: An open source toolkit for generating labeled training data with teacher llms

    Jonas Golde, Patrick Haller, Felix Hamborg, Julian Risch, and Alan Akbik. Fabricator: An open source toolkit for generating labeled training data with teacher llms. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations...

    2023

  7. [15]

    Andrew Halterman and Katherine A. Keith. Codebook llms: Evaluating llms as measurement tools for political science concepts, 2025. URL https://arxiv.org/abs/2407.10747

    2025

  8. [16]

    Bias/variance decompositions for likelihood-based estimators

    Tom Heskes. Bias/variance decompositions for likelihood-based estimators. Neural Computation, 10 0 (6): 0 1425--1433, 08 1998. ISSN 0899-7667. doi:10.1162/089976698300017232. URL https://doi.org/10.1162/089976698300017232

    1998 doi

  9. [17]

    Hopkins, Yphtach Lelkes, and Samuel Wolken

    Daniel J. Hopkins, Yphtach Lelkes, and Samuel Wolken. The rise of and demand for identity-oriented media coverage. American Journal of Political Science, n/a 0 (n/a). doi:https://doi.org/10.1111/ajps.12875. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/ajps.12875

    doi

  10. [18]

    A survey of uncertainty estimation in llms: Theory meets practice, 2024

    Hsiu-Yuan Huang, Yutong Yang, Zhaoxi Zhang, Sanwoo Lee, and Yunfang Wu. A survey of uncertainty estimation in llms: Theory meets practice, 2024. URL https://arxiv.org/abs/2410.15326

    2024 arXiv

  11. [19]

    MedSyn: LLM-Based Synthetic Medical Text Generation Framework, pp.\ 215–230

    Gleb Kumichev, Pavel Blinov, Yulia Kuzkina, Vasily Goncharov, Galina Zubkova, Nikolai Zenovkin, Aleksei Goncharov, and Andrey Savchenko. MedSyn: LLM-Based Synthetic Medical Text Generation Framework, pp.\ 215–230. Springer Nature Switzerland, 2024. ISBN 9783031703812. doi:10.1...

    2024 doi

  12. [20]

    Not all llm-generated data are equal: Rethinking data weighting in text classification, 2024

    Hsun-Yu Kuo, Yin-Hsiang Liao, Yu-Chieh Chao, Wei-Yun Ma, and Pu-Jen Cheng. Not all llm-generated data are equal: Rethinking data weighting in text classification, 2024. URL https://arxiv.org/abs/2410.21526

    2024 arXiv

  13. [21]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems...

    2023

  14. [22]

    Synthetic data generation with large language models for text classification: Potential and limitations

    Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. Synthetic data generation with large language models for text classification: Potential and limitations. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural ...

    2023 doi

  15. [23]

    Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and Andrew M. Dai. Best practices and lessons learned on synthetic data. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?...

    2024

  16. [24]

    Roberta: A robustly optimized bert pretraining approach, 2019

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. URL https://arxiv.org/abs/1907.11692

    2019 arXiv

  17. [25]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp.\ 142--150,...

    2011

  18. [26]

    Leveraging llms for fair data labeling and validation in crowdsourcing environments [vision paper]

    Ibrahim Mohamed Serouis and Florence Sèdes. Leveraging llms for fair data labeling and validation in crowdsourcing environments [vision paper]. In 2024 IEEE International Conference on Big Data (BigData), pp.\ 468--472, 2024. doi:10.1109/BigData62323.2024.10825193

    2024

  19. [27]

    Replication Data for: The Temporal Focus of Campaign Communication , 2020

    Stefan Müller. Replication Data for: The Temporal Focus of Campaign Communication , 2020. URL https://doi.org/10.7910/DVN/7NP2XH

    2020 doi

  20. [28]

    Knowledge distillation in automated annotation: Supervised text classification with LLM -generated training labels

    Nicholas Pangakis and Sam Wolken. Knowledge distillation in automated annotation: Supervised text classification with LLM -generated training labels. In Dallas Card, Anjalie Field, Dirk Hovy, and Katherine Keith (eds.), Proceedings of the Sixth Workshop on Natural Language Pro...

    2024 doi

  21. [29]

    Keeping humans in the loop: Human-centered automated annotation with generative ai, 2024 b

    Nicholas Pangakis and Samuel Wolken. Keeping humans in the loop: Human-centered automated annotation with generative ai, 2024 b . URL https://arxiv.org/abs/2409.09467

    2024 arXiv

  22. [30]

    The curse of recursion: Training on generated data makes models forget, 2024 a

    Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget, 2024 a . URL https://arxiv.org/abs/2305.17493

    2024 arXiv

  23. [31]

    Ai models collapse when trained on recursively generated data

    Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, et al. Ai models collapse when trained on recursively generated data. Nature, 631: 0 755--759, 2024 b . doi:10.1038/s41586-024-07566-y

    2024 doi

  24. [32]

    Large language models and synthetic health data: progress and prospects

    Daniel Smolyak, Margrét V Bjarnadóttir, Kathy Crowley, and Ritu Agarwal. Large language models and synthetic health data: progress and prospects. JAMIA Open, 7 0 (4): 0 ooae114, 12 2024. ISSN 2574-2531. doi:10.1093/jamiaopen/ooae114. URL https://doi.org/10.1093/jamiaopen/ooae114

    2024 doi

  25. [33]

    How to fine-tune bert for text classification?, 2020

    Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. How to fine-tune bert for text classification?, 2020. URL https://arxiv.org/abs/1905.05583

    2020 arXiv

  26. [34]

    Want to reduce labeling cost? GPT -3 can help

    Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. Want to reduce labeling cost? GPT -3 can help. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, pp.\...

    2021 doi

  27. [35]

    Efficient guided generation for llms

    Brandon T Willard and R \'e mi Louf. Efficient guided generation for llms. arXiv preprint arXiv:2307.09702, 2023

    2023 arXiv

  28. [36]

    Transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  29. [37]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  30. [38]

    On the calibration of large language models and alignment

    Chiwei Zhu, Benfeng Xu, Quan Wang, Yongdong Zhang, and Zhendong Mao. On the calibration of large language models and alignment. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp.\ 9778--9795, Singapor...

    2023 doi

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.