Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Evaluating LLM Prompts for Data Augmentation in Multi-label Classification of Ecological Texts

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that prompting an instruction-tuned Russian-language LLM to paraphrase a text while explicitly naming its topic categories is the most consistently effective data-augmentation strategy for multi-label classification of…

desk verdict Careful, reproducible comparison of LLM augmentation prompts on Russian multi-label data; the ranking is plausible but statistical support is thin. read the letter →

arxiv 2411.14896 v1 pith:FJQQIY7P submitted 2024-11-22 cs.CL cs.CYcs.SI

Anna Glazkova , Olga Zakharova This is my paper · ORCID
classification cs.CLcs.CYcs.SI
keywords dataaugmentationlargelanguagemodelsprompt-basedmulti-labelclassificationecologicaltextsRussiansocialmediagreenpracticesclassimbalance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Prompting an instruction-tuned language model to rewrite a sentence while explicitly stating which categories it belongs to is the most consistently effective data-augmentation recipe tested here. The paper compares four prompt strategies—plain paraphrase, category-only generation, category-aware paraphrase, and category-plus-example generation—on an imbalanced Russian dataset of social-media mentions of green practices, using two fine-tuned transformer classifiers and two training-set growth factors. All augmentation strategies raised multi-label F1 relative to fine-tuning on the original data alone, and most beat random duplication and back translation. In three of the four model-by-growth configurations, the category-aware paraphrase prompt produced the highest F1, which is what the paper is trying to establish.

What carries the argument

The load-bearing mechanism is the category-conditioned paraphrase prompt, Ptext+topics, which instructs the model to paraphrase the original text while taking into account its relevance to a given list of categories. It combines the source sentence with explicit label information, so the LLM rewrites the text while keeping topical content and preserving more lexical and semantic similarity than generation-only prompts, but less than a plain paraphrase. The paper's similarity metrics show this intermediate position: Ptext+topics has ROUGE-1 of 18.94 percent and BERTScore of 70.40 percent, compared with 32.58 percent and 76.88 percent for plain paraphrase and 4.03 percent and 56.40 percent for category-only generation. This suggests the category conditioning changes the generated distribution in a way that helps the downstream multi-label classifier.

What would settle it

Take the augmented training sets used in the reported F1 experiments, sample about 100 generated sentences from each of the four prompts, and have annotators judge which of the nine green-practice labels are actually present. If Ptext+topics sentences do not contain their assigned labels more often than sentences from the other prompts, or if removing label-incorrect generations eliminates the F1 advantage, then the comparison is measuring label noise rather than prompt quality.

Watch

Extended reading notes

Core claim

The paper's central claim is that category-conditioned paraphrasing is the most useful of the four tested augmentation prompts for this task. In three out of four configurations—both classifiers and both augmentation sizes (1.5x and 2x)—the Ptext+topics prompt achieved the highest multi-label F1; the remaining best score went to the generation-only prompt. The best absolute F1 for ruELECTRA was 74.17 percent with the generation-only prompt at 1.5x growth, and for ruBERT 71.78 percent with Ptext+topics at 2x growth. All augmentation approaches improved over the original-data F1 of 69.96 percent (ruELECTRA) and 58.16 percent (ruBERT), and LLM augmentation usually outperformed random duplication and back translation.

Load-bearing premise

The paper assumes that every generated sentence carries the labels of its source sentence, or for the generation-only prompt the labels in the topic list, without checking whether the generated text actually expresses those categories.

Editorial extensions

If this is right

  • Prompt-based augmentation with this Russian instruction-tuned LLM is beneficial for imbalanced multi-label text: all four prompts improved F1 over the original-data baseline, and most beat random duplication and back translation.
  • A category-aware paraphrase prompt is a strong default for such tasks, since it delivered the best result in three of four model-by-growth configurations and never reduced performance.
  • Generation-only prompts can be competitive even though their outputs are least similar to the original texts, so lexical diversity alone is not a drawback for augmentation.
  • Paraphrase-based prompts retain higher similarity to the source, but the best F1 came from the prompt that also names categories, indicating that category conditioning, not similarity per se, is what helps.
  • The approach offers a practical template for other class-imbalanced multi-label classification settings where training data are scarce.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is to combine Ptext+topics with rare-class oversampling: the study uses random sentence selection, so rare green-practice categories remain rare, and the conclusion itself flags sampling strategy as future work.
  • The reported gains could partly reflect label inheritance rather than prompt quality, since generated sentences are added with their source labels without verification; an annotation study of label fidelity in generated texts would separate these explanations.
  • The same category-conditioned paraphrase design may transfer to other languages and domains whenever a strong instruction-tuned model is available, but that transfer is not established by these Russian-only experiments.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper compares four LLM prompt strategies for augmenting a Russian multi-label social media dataset (GreenRu) with mentions of green practices. The prompts are: paraphrasing the source text (Ptext), generating a new post from a topic list (Gtopics), paraphrasing the source while explicitly conditioning on its categories (Ptext+topics), and generating from a topic list plus a source example (Gtopics+text). Training sets are augmented to 1.5x and 2x their original size using T-lite-instruct-0.1, and ruELECTRA-large and ruBERT are fine-tuned as multi-label classifiers. Evaluation is macro-averaged multi-label F1 on a held-out test set, with three repeated runs per condition. The paper reports that every augmentation strategy improves over training on the original data, that Ptext+topics is the best prompt in three of the four model/scale combinations, and that category-aware paraphrasing is therefore the most effective strategy.

Significance. If the result holds, it offers a practical, low-cost recommendation for prompt-based data augmentation in multi-label, class-imbalanced Russian text classification: explicitly include the target categories when paraphrasing. The study is one of the first to examine this question for Russian-language data, uses an open Russian-specific instruct model, compares against standard baselines (random duplication and back translation), reports three repeated runs, and makes the generated texts publicly available. The main claims, however, currently rest on point estimates without any significance testing and on an unverified assumption that generated sentences inherit their assigned labels. Both issues bear directly on the central recommendation and need to be addressed before the conclusions are fully supported.

major comments (3)
  1. [III-C and Table IV] The central comparison assumes that every generated sentence is added to the training set with the labels of its source sentence (Ptext, Ptext+topics, Gtopics+text) or with the topic list (Gtopics). Section III-C describes adding generated sentences but reports no human or automatic verification that the generated text actually expresses the assigned green-practice categories. Table V reports only ROUGE-1, ROUGE-L, and BERTScore, which measure lexical and semantic similarity to the source, not whether the target labels are present. This matters because the prompts differ in how strongly they condition on categories: Gtopics and Ptext+topics explicitly instruct the model to write about given topics, while Ptext does not. If the LLM partially ignores the topic instruction, the four strategies introduce different levels and structures of label noise, and the observed F1 ordering could partly be an artifact of that noise rather than of prompt quality. The risk is especially acute for rare classes: Table II shows only 10 training mentions of 'repairing' and 22 of 'signing petitions', so a small number of mislabeled generated examples could dominate the learning signal for those classes. I ask the authors to assess label correctness on a sample of generated texts (e.g., human annotation or rule-based/classifier-based checks), report per-prompt label fidelity, and either filter generated examples before training or re-run the comparison with this verification in place.
  2. [Table IV and Section IV] The reported differences are not supported by significance testing. For example, at the 1.5x scale for ruELECTRA, Ptext achieves 74.09±0.86, Ptext+topics 73.47±2.07, and Gtopics 74.17±0.47; the gap between the best and second-best conditions is 0.70 points with heavily overlapping standard deviations based on only three runs. The abstract's statement that 'all strategies improved classification performance compared to the models fine-tuned only on the original dataset' is likewise a point estimate statement; with the given variances, several of the improvements over the original data may not be statistically distinguishable from zero. Because the paper's main conclusion is a rank ordering of prompts, please report pairwise significance tests (e.g., paired bootstrap or a nonparametric test across seeds) or honestly state that the observed ordering is not statistically significant at the available sample size. If no test is reported, the wording in the abstract and Section IV should be softened accordingly.
  3. [III-B and Table IV] The standard deviations in Table IV appear to reflect only variability in classifier fine-tuning, not variability in the LLM-generated augmentation data. Generation was performed once with temperature 0.5 and maximum 400 tokens, so the reported intervals are conditional on a single generated corpus per prompt. Since the paper compares prompts by the data they generate, the effective uncertainty is understated. Please either generate multiple augmentation sets per prompt and report variance across those sets, or state explicitly that the results and intervals are conditional on one generation pass.
minor comments (4)
  1. [Section IV, after Table V] The sentence 'the scores for Ptext are significantly higher than for Ptext+topics' uses 'significantly' without a statistical test; please replace it with 'substantially' or provide confidence intervals for the similarity metrics.
  2. [Figures 4 and 5] The captions for Figures 4 and 5 are missing, and the figures are referenced only indirectly in the text; please add captions identifying which bars correspond to which augmentation condition and model.
  3. [Section III-E] Please state whether the reported F1 scores come from the final fine-tuning epoch or from the best epoch on a validation split, and specify whether any early stopping was used; this affects the comparability of the three runs.
  4. [Section III-F] The paper defines the multi-label F1-score as the average of per-class scores but does not explicitly call it macro-averaged; please state this explicitly in the metric definition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports a held-out empirical comparison, and its central claim does not reduce to its inputs by construction.

full rationale

This paper does not attempt a derivation or a first-principles prediction; it is an empirical comparison of four prompt-based data augmentation strategies measured by multi-label F1-score on a held-out test subset of GreenRu. The augmented training data are generated only from sentences in the training subset (Section III-C), and the test subset is not used for generation, prompt selection, or hyperparameter tuning. The main claim, that 'in three out of four cases, the Ptext+topics prompt demonstrated the highest performance,' is a direct report of Table IV, not a quantity obtained by fitting a parameter and then re-predicting the same data. The abstract's statement that 'all strategies improved classification performance compared to the models fine-tuned only on the original dataset' is likewise an empirical summary of the same table. The authors cite their own GreenRu dataset and prior green-practice definitions, but the evaluation against held-out labels makes the dataset an external benchmark rather than a circular input to the claim; no uniqueness theorem, ansatz, or fitted parameter is imported from those citations. The skeptical concern that generated texts may not inherit their source labels is a real validity threat about label noise, not a circularity: the comparison could be affected by an unmeasured confound, but the paper does not define the augmentation strategies in terms of the outcome it predicts. No equation is shown to be equal to another by construction, and no fitted value is renamed as a prediction. Therefore the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the quality of GreenRu labels and on the assumption that generated sentences carry the same labels as their sources. These are domain assumptions, not fitted parameters, and they are not verified independently in the paper.

free parameters (4)
  • Augmentation scale multipliers = 1.5 and 2.0
    Two arbitrary targets for training set growth (Section III-C); both are reported, so the choice does not hide a single fitted outcome.
  • Generation temperature = 0.5
    Fixed for T-lite generation (Section III-B); not tuned against the test set.
  • Max generated tokens = 400
    Generation cap for T-lite (Section III-B); not fitted.
  • Fine-tuning hyperparameters = lr 4e-5, 5 epochs, batch 8, max length 256
    Fixed identically across all conditions (Section III-E); standard values, not tuned to maximize any single condition.
assumptions (4)
  • domain assumption GreenRu sentence-level labels are accurate enough to serve as training and test ground truth.
    Section III-A uses GreenRu without re-annotation; label noise would affect all conditions equally but could bias absolute F1 scores.
  • domain assumption T-lite-generated augmented samples inherit the multi-label annotations of their source sentences or prompt topics.
    Section III-C adds generated sentences directly to the training set with the source labels; no manual verification or automatic label check is performed.
  • domain assumption Macro-averaged multi-label F1 is the appropriate evaluation objective for this imbalanced task.
    Section III-F chooses this metric; per-class F1 is not reported, so gains on rare classes cannot be isolated.
  • domain assumption The Simple Transformers library implements ruELECTRA and ruBERT fine-tuning correctly.
    Section III-E relies on standard library defaults; no custom proof of correctness is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating LLM Prompts for Data Augmentation in Multi-label Classification of Ecological Texts." pith.science (2026). https://pith.science/paper/FJQQIY7P

@misc{pith2026241114896,
  author       = {Pith},
  title        = {Pith review of: Evaluating LLM Prompts for Data Augmentation in Multi-label Classification of Ecological Texts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FJQQIY7P}},
  note         = {Machine review of arXiv:2411.14896}
}
read the original abstract

Large language models (LLMs) play a crucial role in natural language processing (NLP) tasks, improving the understanding, generation, and manipulation of human language across domains such as translating, summarizing, and classifying text. Previous studies have demonstrated that instruction-based LLMs can be effectively utilized for data augmentation to generate diverse and realistic text samples. This study applied prompt-based data augmentation to detect mentions of green practices in Russian social media. Detecting green practices in social media aids in understanding their prevalence and helps formulate recommendations for scaling eco-friendly actions to mitigate environmental issues. We evaluated several prompts for augmenting texts in a multi-label classification task, either by rewriting existing datasets using LLMs, generating new data, or combining both approaches. Our results revealed that all strategies improved classification performance compared to the models fine-tuned only on the original dataset, outperforming baselines in most cases. The best results were obtained with the prompt that paraphrased the original text while clearly indicating the relevant categories.

Figures

Figures reproduced from arXiv: 2411.14896 by the authors.

Figure 1
Figure 1. The distribution of practices in GreenRu. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The mutual occurrence of practices in GreenRu. 1 - waste sorting, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The number of mentions per practice in the training set. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance growth, % (ruELECTRA) [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Performance growth, % (ruBERT). samples and original data. These metrics show how much the generated sample corresponds to the original text and similarly how much the augmented text retains the general meaning of the original text. As expected, the highest similarity …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multimodal Large Language Models for Image, Text, and Speech Data Augmentation: A Survey

    cs.CV 2025-01 conditional novelty 4.0 of 10

    A literature review cataloging LLM-based augmentation methods across image, text, and speech, with a taxonomy of techniques, limitations, and suggested fixes.

Reference graph

Works this paper leans on

31 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Survey on deep learning with class imbalance,

    J. M. Johnson and T. M. Khoshgoftaar, “Survey on deep learning with class imbalance,” Journal of big data , vol. 6, no. 1, pp. 1–54, 2019

    2019

  2. [2]

    Text data augmentation for deep learning,

    C. Shorten, T. M. Khoshgoftaar, and B. Furht, “Text data augmentation for deep learning,” Journal of big Data , vol. 8, no. 1, p. 101, 2021

    2021

  3. [3]

    Generative pre-trained transformer (GPT) in research: A systematic review on data augmentation,

    F. Sufi, “Generative pre-trained transformer (GPT) in research: A systematic review on data augmentation,” Information, vol. 15, no. 2, p. 99, 2024

    work page 2024

  4. [4]

    An empirical survey of data augmentation for limited data learning in NLP,

    J. Chen, D. Tam, C. Raffel, M. Bansal, and D. Yang, “An empirical survey of data augmentation for limited data learning in NLP,” Trans- actions of the Association for Computational Linguistics , vol. 11, pp. 191–211, 2023

    work page 2023

  5. [5]

    Green values in crowdfunding projects,

    O. V . Zakharova, I. N. Pupysheva, T. Y . Payusova, A. V . Zakharov, and L. Sulkarnaeva, “Green values in crowdfunding projects,” Glocalism, no. 1, 2021

    work page 2021

  6. [6]

    Detecting Mentions of Green Practices in Social Media Based on Text Classification,

    A. V . Glazkova, O. V . Zakharova, A. V . Zakharov, N. N. Moskvina, T. R. Enikeev, and A. N. Hodyrev, “Detecting Mentions of Green Practices in Social Media Based on Text Classification,” Modeling and Analysis of Information Systems , vol. 29, no. 4, pp. 316–332, 2022

    work page 2022

  7. [7]

    Green practices: Ways to investigation,

    O. V . Zakharova, T. I. Payusova, I. D. Akhmedova, and L. G. Suvorova, “Green practices: Ways to investigation,” Sotsiologicheskie issledovan- iya, no. 4, pp. 25–36, 2021

    work page 2021

  8. [8]

    EDA: Easy data augmentation techniques for boost- ing performance on text classification tasks,

    J. Wei and K. Zou, “EDA: Easy data augmentation techniques for boost- ing performance on text classification tasks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 6382–6388

    work page 2019

Show all 31 references
  1. [9]

    AEDA: An easier data augmentation technique for text classification,

    A. Karimi, L. Rossi, and A. Prati, “AEDA: An easier data augmentation technique for text classification,” in Findings of the Association for Computational Linguistics: EMNLP 2021 , 2021, pp. 2748–2754

    2021

  2. [10]

    Improving neural machine trans- lation models with monolingual data,

    R. Sennrich, B. Haddow, and A. Birch, “Improving neural machine trans- lation models with monolingual data,” arXiv preprint arXiv:1511.06709, 2015

    2015 arXiv

  3. [11]

    Data augmentation using pre- trained transformer models,

    V . Kumar, A. Choudhary, and E. Cho, “Data augmentation using pre- trained transformer models,” in Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems , 2020, pp. 18–26

    2020

  4. [12]

    Conditional BERT contextual augmentation,

    X. Wu, S. Lv, L. Zang, J. Han, and S. Hu, “Conditional BERT contextual augmentation,” in Computational Science–ICCS 2019: 19th Interna- tional Conference, Faro, Portugal, June 12–14, 2019, Proceedings, Part IV 19. Springer, 2019, pp. 84–95

    2019

  5. [13]

    AugGPT: Leveraging ChatGPT for text data augmentation,

    H. Dai, Z. Liu, W. Liao, X. Huang, Y . Cao, Z. Wu, L. Zhao, S. Xu, W. Liu, N. Liu et al. , “AugGPT: Leveraging ChatGPT for text data augmentation,” arXiv preprint arXiv:2302.13007 , 2023

    2023 arXiv

  6. [14]

    Is ChatGPT the ultimate Data Augment- ation Algorithm?

    F. Piedboeuf and P. Langlais, “Is ChatGPT the ultimate Data Augment- ation Algorithm?” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 15 606–15 615

    2023

  7. [15]

    Improving Text Classification with Large Language Model-Based Data Augmentation,

    H. Zhao, H. Chen, T. A. Ruggles, Y . Feng, D. Singh, and H.-J. Yoon, “Improving Text Classification with Large Language Model-Based Data Augmentation,” Electronics, vol. 13, no. 13, p. 2535, 2024

    2024

  8. [16]

    GPT3Mix: Leveraging large-scale language models for text augmentation,

    K. M. Yoo, D. Park, J. Kang, S.-W. Lee, and W. Park, “GPT3Mix: Leveraging large-scale language models for text augmentation,” in Findings of the Association for Computational Linguistics: EMNLP 2021, 2021, pp. 2225–2239

    2021

  9. [17]

    Medical data augmentation via ChatGPT: A case study on medication identification and medication event classification,

    S. Sarker, L. Qian, and X. Dong, “Medical data augmentation via ChatGPT: A case study on medication identification and medication event classification,” arXiv preprint arXiv:2306.07297 , 2023

    2023 arXiv

  10. [18]

    Enhancing social network hate detection using back translation and GPT-3 augmentations during training and test-time,

    S. Cohen, D. Presil, O. Katz, O. Arbili, S. Messica, and L. Rokach, “Enhancing social network hate detection using back translation and GPT-3 augmentations during training and test-time,”information Fusion, vol. 99, p. 101887, 2023

    2023

  11. [19]

    Improving multiclass classification of fake news using BERT-based models and ChatGPT- augmented data,

    E. Shushkevich, M. Alexandrov, and J. Cardiff, “Improving multiclass classification of fake news using BERT-based models and ChatGPT- augmented data,” Inventions, vol. 8, no. 5, p. 112, 2023

    2023

  12. [20]

    From Big to Small Without Losing It All: Text Augmentation with ChatGPT for Efficient Sentiment Analysis,

    S. Wo ´zniak and J. Koco ´n, “From Big to Small Without Losing It All: Text Augmentation with ChatGPT for Efficient Sentiment Analysis,” in 2023 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 2023, pp. 799–808

    2023

  13. [21]

    The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks,

    A. G. Møller, A. Pera, J. Dalsgaard, and L. Aiello, “The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers) , 202...

    2024

  14. [22]

    GreenRu: A Russian Dataset for Detecting Mentions of Green Practices in Social Media Posts,

    O. Zakharova and A. Glazkova, “GreenRu: A Russian Dataset for Detecting Mentions of Green Practices in Social Media Posts,” Applied Sciences, vol. 14, no. 11, p. 4466, 2024

    2024

  15. [23]

    The importance of green practices to reduce consumption,

    O. V . Zakharova, A. V . Glazkova, I. N. Pupysheva, and N. V . Kuznetsova, “The importance of green practices to reduce consumption,” Changing Societies & Personalities. 2022. Vol. 6. Iss. 4 , pp. 884–905, 2022

    2022

  16. [24]

    A family of pretrained transformer language models for Russian,

    D. Zmitrovich, A. Abramov, A. Kalmykov, V . Kadulin, M. Tikhonova, E. Taktasheva, D. Astafurov, M. Baushenko, A. Snegirev, T. Shavrina et al., “A family of pretrained transformer language models for Russian,” in Proceedings of the 2024 Joint International Conference on Computa...

    2024

  17. [25]

    ELECTRA: Pre- training text encoders as discriminators rather than generators,

    K. Clark, M.-T. Luong, Q. V . Le, and C. D. Manning, “ELECTRA: Pre- training text encoders as discriminators rather than generators,” in ICLR,

  18. [26]

    Adaptation of deep bidirectional multilin- gual transformers for Russian language,

    Y . Kuratov and M. Arkhipov, “Adaptation of deep bidirectional multilin- gual transformers for Russian language,” in Komp’juternaja Lingvistika i Intellektual’nye Tehnologii, 2019, pp. 333–339

    2019

  19. [27]

    BERT: Pre- training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” in 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long...

    2019

  20. [28]

    Simple transformers,

    T. C. Rajapakse, “Simple transformers,” https://github.com/ ThilinaRajapakse/simpletransformers, 2019

    2019

  21. [29]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013

    2004

  22. [30]

    BERTScore: Evaluating text generation with BERT,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “BERTScore: Evaluating text generation with BERT,” in International Conference on Learning Representations , 2020

    2020

  23. [2020]

    Available: https://openreview.net/pdf?id=r1xMH1BtvB

    [Online]. Available: https://openreview.net/pdf?id=r1xMH1BtvB

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.