Pith. sign in

REVIEW 4 major objections 7 minor 41 references

Small Language Models in the Real World: Insights from Industrial Text Classification

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For industrial text classification, a frozen 1-billion-parameter small language model with a trained classification head is the only portable, lightweight method the authors find viable.

desk verdict Practical, useful evaluation of small models for industrial text classification, but the headline 'FT is the only viable solution' stands on validation-set numbers and a narrow definition of fine-tuning; treat the ranking as provisional until a held-out test is added. read the letter →

arxiv 2505.16078 v3 pith:4KXJHXFU submitted 2025-05-21 cs.CL

Lujun Li , Lama Sleem , Niccolo' Gentile , Geoffrey Nichil , Radu State This is my paper
classification cs.CL
keywords smalllanguagemodelstextclassificationpromptengineeringsupervisedfine-tuningparameter-efficientindustrialdeploymentVRAMefficiencylongdocument
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to determine whether small language models can handle real industrial text classification, and which training or prompting method makes them usable. Across an EU-legislation corpus, a long academic-document corpus, and a proprietary multilingual insurance email corpus, the authors compare prompt engineering, soft-prompt tuning, prefix tuning, and supervised fine-tuning of the classification head on frozen weights. Their central finding is that the frozen-backbone fine-tuned head is the only approach they tested that is both portable and accurate: Llama-3.2-1B reaches 0.999 accuracy and F1 on the EU dataset, 0.892 on the long-document dataset, and 0.865 on the industrial email dataset, at 27.5 GPU RAM hours versus 86.8 for a prompted 70-billion-parameter model on the email corpus. The same experiments indicate that data volume, not model scale or head depth, is the main performance bottleneck on the harder datasets.

What carries the argument

The load-bearing mechanism is the classification head: a single linear layer appended to a frozen transformer, mapping the model's final hidden state to label scores and trained alone with BCEWithLogitsLoss, while all pretrained weights stay fixed. Because the backbone is never updated, training is cheap enough for local GPUs and the resulting model deploys at roughly the size of the 1B backbone plus one small layer, avoiding the slow token-by-token decoding that makes prompted generation costly. The paper contrasts this head with soft prompt tuning, a learnable input prompt, and prefix tuning, learnable tensors inside each attention layer, and shows that the head wins on both accuracy and efficiency.

What would settle it

Run the same frozen-weight classification-head recipe for Llama-3.2-1B on a held-out test set that is not used for checkpoint selection, repeated across several random seeds; if its accuracy on the email corpus falls to the prompted 70B level (about 0.80) or below, the claim that head fine-tuning is the only viable portable solution would be contradicted.

Watch

Extended reading notes

Core claim

The discovery is that decoder-only small language models are essentially unusable as prompted classifiers—base, few-shot, chain-of-thought, self-consistency, and chain-of-draft prompts on 1B and 3B Llama models hover near random accuracy, with few-shot prompting the only strategy that reliably helps—but the same models become strong classifiers once a single linear classification head is trained on frozen weights. On EURLEX57K the frozen Llama-3.2-1B hits 0.999 accuracy and F1; on the long-document dataset, 0.892 accuracy and 0.890 F1; and on the insurance email corpus, 0.865 accuracy and 0.863 F1, surpassing the locally run Llama-3.3-70B-Instruct prompt baseline (0.800 accuracy and 0.799 F1 on emails) at a fraction of the GPU RAM hours. Soft prompt tuning and prefix tuning sit between prompting and head fine-tuning, while encoder-only ModernBERT is competitive on academic long documents but poor on multilingual and legal-domain data. The paper concludes that for portable, lightweight industrial deployment, fine-tuning a classification head on a small frozen transformer remains the only viable solution among those tested.

Load-bearing premise

The paper's headline numbers come from validation splits with the lowest-validation-loss checkpoint, and the conclusions assume those validation results transfer to unseen deployment data without a held-out test set or repeated-seed averaging.

Editorial extensions

If this is right

  • A 1-billion-parameter decoder-only model with a trained head is a workable local replacement for prompted 70B models on this class of tasks, with materially lower GPU memory and time.
  • Few-shot prompting, not chain-of-thought or chain-of-draft, is the least bad prompt strategy on small models; the reasoning prompts sometimes lower accuracy.
  • On difficult datasets, adding labeled training data moves performance more than switching from a 1B to a 3B model or adding classification-head layers.
  • VRAM accounting should be a first-class selection criterion for industrial deployment, since the cheapest method here is also the most accurate.
  • Encoder-only models remain useful only when the deployment domain matches their pretraining distribution; the same fine-tuning recipe does not repair a domain mismatch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that its validation-split reporting means deployment should confirm these numbers on a held-out test set with repeated seeds.
  • The trained-head recipe turns a decoder-only generative model into an encoder-style discriminative classifier; a natural untested extension is pairing that head with a long-document retrieval or summarization step, since the email task's decisive information sits at the start of the thread.
  • Because few-shot prompting was the only prompt method that reliably helped small models, a hybrid system—trained head for fixed categories plus few-shot prompting as a fallback—is a plausible way to cover label spaces that change after deployment; the paper does not test it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper reports an empirical comparison of prompt-engineering families (base, few-shot, chain-of-thought, self-consistency, chain-of-draft) and training-based methods (supervised fine-tuning, soft prompt tuning, prefix tuning) for small language models on three text classification datasets: EURLEX57K (EU legislation), the Long Document Dataset (academic papers), and a proprietary multilingual insurance-email dataset (IE). The model set includes Llama-3.2-1B/3B and ModernBERT-base/large, with Llama-3.3-70B-Instruct and GPT-4o-mini as larger baselines. Table 3 shows that prompt-based SLM performance is often near random guessing, while the linear-head fine-tuning variant reaches very high accuracy on EUR and strong accuracy on LDD and IE. The authors conclude that fine-tuned transformer SLMs are the only viable portable and lightweight solution for industrial text classification and that the 1B model offers the best efficiency-accuracy trade-off.

Significance. If the reported results survive a proper held-out evaluation, the paper would provide practically useful guidance for industrial text classification: a frozen SLM backbone with a trained classification head is cheap to deploy and can outperform large-model prompting on these domains. The inclusion of a proprietary real-world email dataset, the systematic coverage of several prompt-engineering families, and the explicit reporting of GPU-hours and GPU-RAM-hours are genuine strengths of the study. The main contribution is empirical guidance rather than a new method, and its value depends heavily on the reliability of the measurement protocol.

major comments (4)
  1. [Section 3.4 and Table 3] The headline results are validation-set numbers obtained under checkpoint selection on the same validation data. Section 3.4 states that the checkpoint with the lowest evaluation loss is selected and that Table 3 reports validation performance; because the same validation split is used both for model selection and for the reported accuracy/F1, the numbers for the trained methods (e.g., 0.999/0.999 on EUR for Llama-3.2-1B FT and 0.865/0.863 on IE) are plausibly optimistic, and the paper provides no held-out test set and no repeated-seed variance. The central conclusion in Section 4.4 that fine-tuning is "the only viable solution" depends on these numbers, so a held-out test split (or cross-validation) and multiple seeds are required before that claim can be accepted.
  2. [Section 3.3 and Appendix A] The method called Supervised Fine-Tuning in this paper trains only a newly added linear classification head on top of completely frozen pre-trained weights; it does not update any transformer weights in the backbone. The conclusion in Section 4.4 that "FT transformers remains the only viable solution on SLMs" is therefore broader than what was tested: standard fine-tuning of the backbone, LoRA, or other parameter-efficient methods that update some transformer weights are not evaluated. The claim should be restricted to head-only training, or the missing configurations should be added; the ModernBERT-base result of 0.514 accuracy on IE also shows that the method is not universally viable.
  3. [Section 3.4 and Figure 2] The efficiency comparison is not apples-to-apples: for prompt-engineering methods the paper measures only the inference stage, while for fine-tuning the reported GH/GRH values include training the classification head. Figure 2 presents these values as a single efficiency frontier, so the conclusion that FT models are the most efficient depends on how training cost is amortized over deployment. The paper should either compare end-to-end costs under a stated deployment assumption or report training and inference costs separately for all methods.
  4. [Section 6] The Limitations section explicitly acknowledges that the number of virtual tokens for SPT and PT was not varied and that increasing that number could yield better results. Since Table 3 shows SPT and PT underperforming FT, the fixed choice of 128 virtual tokens could understate these methods; at minimum, the "only viable solution" claim should be tempered by this acknowledged hyperparameter uncertainty.
minor comments (7)
  1. [Section 4.2.3] The heading "Deeper Header?" should read "Deeper Head?", and the same misspelling appears in the body text where "header" is used instead of "head".
  2. [Appendix A] The phrase "the orignal weights of the pre-trained model" contains a typo and should read "the original weights".
  3. [Sections 3.4 and 4.1] The model names are inconsistent: Table 3 and the text refer to "GPT-4o-mini", while Section 4.1 refers to "ChatGPT-o1 mini"; please clarify which model was actually evaluated.
  4. [Section 3.3] The model list says "Llama-3.2-1B, Llama-3.2-1B and ModernBERT-base", listing Llama-3.2-1B twice; one of these entries should likely be Llama-3.2-3B-Instruct.
  5. [Section 4.1] The sentence "In the context of SFT, we observed that SPT outperformed prefix tuning" is confusing because SPT and PT are not SFT; the sentence should be rephrased to keep the method categories distinct.
  6. [Section 6] The limitation about virtual tokens refers to SPT/PT, not SFT; the sentence "the impact of the number of virtual tokens in SFT has not been thoroughly explored" should refer to soft prompt tuning and prefix tuning.
  7. [Table 2] The column header "Words / D" is undefined; please expand it to "Words/Document" or add a note in the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark comparison whose conclusions rest on external datasets and direct measurements, not on self-referential derivation.

full rationale

The paper's central claim that fine-tuning (FT) is the only viable portable and lightweight solution for SLMs in industrial text classification is an empirical conclusion drawn from direct comparisons across prompt engineering, soft prompt tuning, prefix tuning, and supervised fine-tuning on three datasets, including one proprietary industrial dataset. There is no derivation chain in which an output quantity is defined in terms of the claimed prediction, no fitted parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation chain. The methods compared are standard external techniques (base prompts, few-shot, COT, COD, SFT, SPT, PT), and the models and benchmarks are external artifacts. The only notable methodological concern is that Section 3.4 states the checkpoint with the lowest evaluation loss is selected and Table 3 reports validation performance, meaning the same validation split is used both for model selection and for headline reporting; this could make the reported FT accuracies optimistic and weaken the 'only viable solution' conclusion without a held-out test set or repeated-seed variance estimates. That is an experimental-design and measurement-validity concern, not circularity under the specified criteria. The paper also acknowledges possible pretraining contamination of public datasets, which is a data-contamination issue rather than a circular-reasoning issue. Therefore no circular step is present.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest entirely on empirical comparisons with fixed hyperparameters, expert labels, and a validation-based evaluation protocol. No new theory, model, or entity is proposed. The main free parameters are experimental constants fixed across models, and the main assumptions are that validation performance reflects deployment and that frozen-backbone linear probing represents supervised fine-tuning.

free parameters (6)
  • learning rate = 1e-6
    Fixed for all fine-tuning variants; no learning rate search is reported, so comparisons may be sensitive to this choice.
  • training epochs = 10
    Set to 10 with best-validation checkpoint selection; no epoch search or early stopping analysis is given.
  • batch size = 8
    Standardized across models for controlled comparison.
  • max context length = 4096
    Used for all models and methods; LDD documents average over 10,000 words, so truncation effects are not analyzed.
  • virtual tokens for SPT and PT = 128
    Set to 128 without exploration; the authors list this as an unexplored dimension in the limitations.
  • self-consistency paths = 3
    The number of reasoning chains for self-consistency CoT is fixed at 3; other values could change results.
assumptions (5)
  • domain assumption Validation performance is a reliable proxy for deployment performance
    The paper reports only validation metrics and selects checkpoints by validation loss, with no held-out test set.
  • ad hoc to paper Frozen pretrained weights plus a newly trained linear head constitutes supervised fine-tuning
    Section 3.3 defines SFT as training only the classification network on frozen weights, which is narrower than common usage and could differ from full fine-tuning or LoRA.
  • domain assumption Expert labels on the proprietary email dataset are correct and consistent
    IE labels were manually identified by industrial partner experts; no inter-annotator agreement or label noise analysis is provided.
  • domain assumption Public benchmark labels are not contaminated by pretraining in a way that inflates prompt evaluation
    The paper acknowledges possible pretraining bias in Section 2.2 but does not test it, and ChatGPT is excluded from the IE dataset only.
  • domain assumption BCEWithLogitsLoss is a suitable objective for the multi-class classification tasks
    Used for all classification tasks without justification; for single-label multi-class problems, the choice of sigmoid-based loss can affect calibration and F1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Small Language Models in the Real World: Insights from Industrial Text Classification." pith.science (2026). https://pith.science/paper/4KXJHXFU

@misc{pith2026250516078,
  author       = {Pith},
  title        = {Pith review of: Small Language Models in the Real World: Insights from Industrial Text Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4KXJHXFU}},
  note         = {Machine review of arXiv:2505.16078}
}
read the original abstract

With the emergence of ChatGPT, Transformer models have significantly advanced text classification and related tasks. Decoder-only models such as Llama exhibit strong performance and flexibility, yet they suffer from inefficiency on inference due to token-by-token generation, and their effectiveness in text classification tasks heavily depends on prompt quality. Moreover, their substantial GPU resource requirements often limit widespread adoption. Thus, the question of whether smaller language models are capable of effectively handling text classification tasks emerges as a topic of significant interest. However, the selection of appropriate models and methodologies remains largely underexplored. In this paper, we conduct a comprehensive evaluation of prompt engineering and supervised fine-tuning methods for transformer-based text classification. Specifically, we focus on practical industrial scenarios, including email classification, legal document categorization, and the classification of extremely long academic texts. We examine the strengths and limitations of smaller models, with particular attention to both their performance and their efficiency in Video Random-Access Memory (VRAM) utilization, thereby providing valuable insights for the local deployment and application of compact models in industrial settings.

Figures

Figures reproduced from arXiv: 2505.16078 by the authors.

Figure 1
Figure 1. Impact of Data Volume on Model Performance. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Reversed efficiency on LDD datasets [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 9 canonical work pages

  1. [1]

    Peters, and Arman Cohan

    Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. https://arxiv.org/abs/2004.05150 Longformer: The long-document transformer . CoRR, abs/2004.05150

    arXiv 2020

  2. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    arXiv 2020

  3. [4]

    Ilias Chalkidis, Emmanouil Fergadiotis, Prodromos Malakasiotis, and Ion Androutsopoulos. 2019. https://doi.org/10.18653/v1/P19-1636 Large-scale multi-label text classification on EU legislation . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6314--6322, Florence, Italy. Association for Computational Linguistics

    doi:10.18653/v1/p19-1636 2019

  4. [5]

    Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.261 LEGAL - BERT : The muppets straight out of law school . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898--2904, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.findings-emnlp.261 2020

  5. [6]

    Edward Collins, Nikolai Rozanov, and Bingbing Zhang. 2018. https://arxiv.org/abs/1811.01910 Evolutionary data measures: Understanding the difficulty of text classification tasks . CoRR, abs/1811.01910

    work page Pith review arXiv 2018

  6. [8]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://arxiv.org/abs/1810.04805 Bert: Pre-training of deep bidirectional transformers for language understanding . Preprint, arXiv:1810.04805

    arXiv 2019

  7. [9]

    Jinhua Du, Yan Huang, and Karo Moilanen. 2020. https://doi.org/10.18653/v1/2020.coling-main.544 Pointing to select: A fast pointer- LSTM for long text classification . In Proceedings of the 28th International Conference on Computational Linguistics, pages 6184--6193, Barcelona, Spain (Online). International Committee on Computational Linguistics

    work page doi:10.18653/v1/2020.coling-main.544 2020

  8. [10]

    Yingbo Gao, Christian Herold, Zijian Yang, and Hermann Ney. 2022. https://arxiv.org/abs/2210.11807 Is encoder-decoder redundant for neural machine translation? Preprint, arXiv:2210.11807

    work page Pith review arXiv 2022

Show all 41 references
  1. [11]

    Jun He, Liqun Wang, Liu Liu, Jiao Feng, and Hao Wu. 2019. https://doi.org/10.1109/ACCESS.2019.2907992 Long document classification from local word glimpses via recurrent attention learning . IEEE Access, 7:40707--40718

    2019

  2. [12]

    Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. https://arxiv.org/abs/2302.09210 How good are gpt models at machine translation? a comprehensive evaluation . Preprint, arX...

    2023 arXiv

  3. [13]

    Thorsten Joachims. 1998. https://doi.org/10.17877/DE290R-5097 Text categorization with support vector machines . Proc. European Conf. Machine Learning (ECML'98)

    1998 doi

  4. [14]

    Yoon Kim. 2014. https://arxiv.org/abs/1408.5882 Convolutional neural networks for sentence classification . CoRR, abs/1408.5882

    2014 arXiv

  5. [15]

    Rania Kora and Ammar Mohammed. 2023. https://doi.org/10.1109/MIUCC58832.2023.10278387 A comprehensive review on transformers models for text classification . In 2023 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pages 1--7

    2023

  6. [16]

    Pierre Lepagnol, Thomas Gerald, Sahar Ghannay, Christophe Servan, and Sophie Rosset. 2024. https://aclanthology.org/2024.lrec-main.1299/ Small language models are good too: An empirical study of zero-shot classification . In Proceedings of the 2024 Joint International Conferen...

    2024

  7. [17]

    Brian Lester, Rami Al - Rfou, and Noah Constant. 2021. https://arxiv.org/abs/2104.08691 The power of scale for parameter-efficient prompt tuning . CoRR, abs/2104.08691

    2021 arXiv

  8. [18]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. https://arxiv.org/abs/1910.13461 BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and compreh...

    2019 arXiv

  9. [19]

    Xiang Lisa Li and Percy Liang. 2021. https://arxiv.org/abs/2101.00190 Prefix-tuning: Optimizing continuous prompts for generation . CoRR, abs/2101.00190

    2021 arXiv

  10. [20]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

    2024 arXiv

  11. [21]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized BERT pretraining approach . CoRR, abs/1907.11692

    2019 arXiv

  12. [22]

    Zihan Liu, Feijun Jiang, Yuxiang Hu, Chen Shi, and Pascale Fung. 2021. https://arxiv.org/abs/2112.00405 NER-BERT: A pre-trained model for low-resource entity tagging . CoRR, abs/2112.00405

    2021 arXiv

  13. [23]

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2024. https://arxiv.org/abs/2307.06435 A comprehensive overview of large language models . Preprint, arXiv:2307.06435

    2024 arXiv

  14. [24]

    Made Nindyatama Nityasya, Haryo Akbarianto Wibowo, Rendi Chevi, Radityo Eko Prasojo, and Alham Fikri Aji. 2022. https://arxiv.org/abs/2201.00558 Which student is best? a comprehensive knowledge distillation exam for task-specific bert models . Preprint, arXiv:2201.00558

    2022 arXiv

  15. [25]

    Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, and Arsalan Shahid. 2024. https://arxiv.org/abs/2408.13296 The ultimate guide to fine-tuning llms from basics to breakthroughs: An exhaustive review of technologies, research, best practices, applied research chal...

    2024 arXiv

  16. [26]

    Sachin Pawar, Nitin Ramrakhiyani, Anubhav Sinha, Manoj Apte, and Girish Palshikar. 2024. https://aclanthology.org/2024.findings-eacl.74/ Why generate when you can discriminate? a novel technique for text classification using language models . In Findings of the Association for...

    2024

  17. [27]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. https://arxiv.org/abs/1910.10683 Exploring the limits of transfer learning with a unified text-to-text transformer . CoRR, abs/1910.10683

    2019 arXiv

  18. [28]

    Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2025. https://arxiv.org/abs/2402.07927 A systematic survey of prompt engineering in large language models: Techniques and applications . Preprint, arXiv:2402.07927

    2025 arXiv

  19. [29]

    Yoo, Chan Yeun, Dirar Homouz, and Aya Taha

    Kamal Taha, Paul D. Yoo, Chan Yeun, Dirar Homouz, and Aya Taha. 2024. https://doi.org/10.1016/j.cosrev.2024.100664 A comprehensive survey of text classification techniques and their research applications: Observational and experimental insights . Computer Science Review, 54:100664

    2024

  20. [30]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    2024 arXiv

  21. [31]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://arxiv.org/abs/2302.13971 Llama:...

    2023 arXiv

  22. [32]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 a . https://arxiv.org/abs/2203.11171 Self-consistency improves chain of thought reasoning in language models . Preprint, arXiv:2203.11171

    2023 arXiv

  23. [33]

    Yue Wang, Dan Qiao, Juntao Li, Jinxiong Chang, Qishen Zhang, Zhongyi Liu, Guannan Zhang, and Min Zhang. 2023 b . https://doi.org/10.18653/v1/2023.findings-acl.489 Towards better hierarchical text classification with data generation . In Findings of the Association for Computat...

    2023 doi

  24. [34]

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. https://arxiv.org/abs/2412.13663 Smarter, better, f...

    2024 arXiv

  25. [35]

    Chi, Quoc Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. 2022. https://arxiv.org/abs/2201.11903 Chain of thought prompting elicits reasoning in large language models . CoRR, abs/2201.11903

    2022 arXiv

  26. [36]

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025. https://arxiv.org/abs/2502.18600 Chain of draft: Thinking faster by writing less . Preprint, arXiv:2502.18600

    2025 arXiv

  27. [37]

    Qinyuan Ye, Maxamed Axmed, Reid Pryzant, and Fereshte Khani. 2024. https://arxiv.org/abs/2311.05661 Prompt engineering a prompt engineer . Preprint, arXiv:2311.05661

    2024 arXiv

  28. [38]

    Dani Yogatama, Chris Dyer, Wang Ling, and Phil Blunsom. 2017. https://arxiv.org/abs/1703.01898 Generative and discriminative text classification with recurrent neural networks . Preprint, arXiv:1703.01898

    2017 arXiv

  29. [39]

    Yang Zhao, Jiajun Zhang, and Chengqing Zong. 2023. Transformer: A general framework from machine translation to others. Machine Intelligence Research, 20(4):514--538

    2023

  30. [40]

    Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024. https://arxiv.org/abs/2402.18815 How do large language models handle multilingualism? Preprint, arXiv:2402.18815

    2024 arXiv

  31. [41]

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2024. https://doi.org/10.1162/tacl_a_00704 A survey on model compression for large language models . Transactions of the Association for Computational Linguistics, 12:1556--1577

    2024 doi

  32. [42]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  33. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.