Pith. sign in

REVIEW 4 major objections 6 minor 32 references

Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missions

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Decoder-only LLMs with few-shot examples detect team micro-behaviors from transcripts better than fine-tuned encoder-only models, reaching 44% and 68% macro F1.

desk verdict Same-team few-shot examples undermine the headline Llama comparison; the task and data are real and the paper is fixable. read the letter →

arxiv 2506.22679 v1 pith:74AYUEBE submitted 2025-06-27 cs.CL

Ankush Raut , Projna Paromita , Sydney Begerowski , Suzanne Bell , Theodora Chaspari This is my paper
classification cs.CL
keywords largelanguagemodelsmicro-behaviordetectionteamcommunicationin-contextlearningfew-shotclassificationspacemissionstranscriptanalysisLlama-3.1
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether large language models can read conversation transcripts from simulated space missions and pick out the small, momentary signs that crew members are uplifting or discouraging one another. It compares fine-tuned encoder-only models (RoBERTa for three-way classification, DistilBERT for binary classification) against a much larger decoder-only model (Llama-3.1) prompted with a handful of labeled examples. The central finding is that the decoder-only route is more feasible: its best configurations reach 44% on the three-way task and 68% on the binary task (macro F1, the average of per-class F1 scores), while the encoder-only models fail to detect the rare discouraging category almost entirely, with recall near 1% in the best configuration. If this result holds, subtle team dynamics can be monitored from text alone in settings where audio is unavailable and labeled data are too scarce to fine-tune large models.

What carries the argument

The central mechanism is few-shot in-context prompting: the target turn is embedded in a prompt that also contains its event type, three or four previous turns, the following response, and several labeled examples of each micro-behavior class, and an instruction-tuned causal language model is asked to generate a single label. The paper attributes the decoder-only advantage to autoregressive token-by-token prediction, which can assign high probability to the distinctive words that mark a rare class such as discouraging speech, whereas encoder-only models pool information across the whole sequence and tend to wash out those signals. Supporting devices include weighted cross-entropy loss and paraphrase-augmented fine-tuning for the encoders, and 8-bit quantization and truncated generation for the decoder-only model.

What would settle it

Hold out an entire crew from the few-shot examples as well as from the test set, and check whether Llama-3.1's macro F1 on that crew's turns stays near 44% (three-way) and 68% (binary).

Watch

Extended reading notes

Core claim

Using transcripts from five crews in a 45-day simulated space mission, the paper compares two routes to micro-behavior classification: fine-tuning encoder-only language models and few-shot prompting of an instruction-tuned decoder-only language model. Its central result is that the decoder-only model outperforms the fine-tuned encoders, improving macro F1 from 41% to 44% in the three-way task and from 57% to 68% in the binary task, with the gain concentrated in recall for the rare discouraging class (28% versus 1% in the best configurations). The paper interprets this as evidence that decoder-only models can detect subtle, underrepresented micro-behaviors from text transcripts alone, and that in-context learning with a handful of examples is a viable alternative to fine-tuning on small, imbalanced datasets.

Load-bearing premise

The evaluation assumes that drawing the few-shot examples from the same team as the turn being classified gives a fair estimate of how the model would perform on a team it has never seen.

Editorial extensions

If this is right

  • Transcript-only analysis of team communication can identify valenced micro-behaviors with moderate macro F1 without needing audio, which matters for space missions and other high-stakes settings where text is the only accessible data.
  • The rare discouraging class, which fine-tuned encoders almost never detect, is recoverable by a decoder-only model with as few as seven in-context examples.
  • For small, imbalanced datasets, few-shot prompting of a large instruction-tuned model can beat fine-tuning of smaller encoder-only models, avoiding the catastrophic forgetting observed when the 8-billion-parameter decoder was fine-tuned.
  • Three previous conversational turns appear sufficient context; larger context windows and more in-context examples give diminishing returns or degrade performance.
  • Paraphrase-based data augmentation is not reliably beneficial and can hurt performance on very small datasets, as the DistilBERT binary results show.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported decoder-only advantage may be partly inflated because the few-shot examples are drawn from the same team as the turn being classified; a test with examples taken only from other crews would give a fairer estimate of cross-team generalization.
  • The token-level explanation suggests that deliberately selecting in-context examples rich in rare-class vocabulary, or balancing examples across classes, could push recall for discouraging speech higher still.
  • The binary result of 68% is close to a usable screening signal, but operational use would require setting a cost for confusing uplifting and discouraging turns, since the two errors have very different consequences for team support.
  • With only five crews in the data, team-level variance is a serious unknown; leave-multiple-teams-out evaluation or additional analog-mission transcripts would show how stable the 44% and 68% figures are.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper reports a feasibility study of large language models (LLMs) for detecting micro-behaviors (uplifting, discouraging, neither) in conversational turns from transcripts of five teams in NASA's Human Exploration Research Analog (HERA) missions. It compares zero-shot, fine-tuned, and paraphrase-augmented fine-tuned encoder-only models (RoBERTa, DistilBERT) with few-shot prompting of Llama-3.1-8B-Instruct. The authors report best macro F1-scores of 44% (Llama) versus 41% (RoBERTa) for 3-way classification and 68% (Llama) versus 57% (DistilBERT) for binary classification, concluding that decoder-only LLMs with few-shot examples are more feasible for this task. The paper also reports that encoder-only models fail to detect discouraging turns and that paraphrase augmentation is not consistently beneficial.

Significance. If the reported comparison were valid, the paper would provide a useful empirical data point for the team-process and spoken-language-understanding communities: an 8B instruction-tuned model with a few in-context examples outperforming fine-tuned encoders on a real, high-stakes collaborative task. The dataset from the HERA analog mission is a valuable resource, and the task of detecting both positive and negative micro-behaviors from text alone is relatively unexplored. The paper's principal strength is its exploration of several modeling paradigms on a practically relevant corpus. However, the central claim is undermined by an evaluation asymmetry between the decoder and encoder pipelines, and by the absence of any variance or significance analysis. As presented, the paper does not establish that decoder-only LLMs are more feasible for micro-behavior detection; the observed advantage may be an artifact of the experimental protocol. With a corrected protocol and proper statistical reporting, the study could still be a meaningful contribution.

major comments (4)
  1. [Section 4.2 (few-shot sampling) and Section 4.1 (cross-validation)] The few-shot evaluation for Llama draws in-context examples from the same team as the test turn (Section 4.2: 'The few-shot examples ... are randomly sampled from the conversations of the same team that the turn belongs to'), whereas the encoder-only models are evaluated with leave-one-team-out cross-validation (Section 4.1: 'holding out 1 team and randomly splitting the input contexts from the remaining 4 teams'). This is an asymmetric comparison: Llama sees team-specific lexical and conversational cues in its prompt that are not available to the encoder baselines, which must generalize to an entirely unseen team. The reported gaps (44% vs 41%; 68% vs 57%) could be explained by this protocol mismatch rather than by model capability. To support the abstract's central claim, the authors must re-evaluate Llama under a matched held-out-team protocol, e.g., sampling few-shot examples only from the training teams and testing on the held-out team, or otherwise give the encoder models access to the same within-team information.
  2. [Section 5 and Figure 4] No confidence intervals, per-fold variance, or significance tests are reported anywhere in the results. The dataset is small (five teams), and the headline differences are modest (3 points in 3-way macro F1). Moreover, the best configurations are selected post hoc from a grid over context length k, number of examples m, and whether coding definitions are included; Figure 4 shows many configurations, and Tables 1-3 report only selected maxima. This selection without correction or variance reporting makes the reported superiority of Llama unsubstantiated. The authors should provide per-fold results, a bootstrap or permutation test over teams (or at minimum over turns), and either pre-specify the configuration selection rule or report all configurations with appropriate multiplicity corrections.
  3. [Section 4.2 (binary task)] It is not stated whether the binary classification for Llama excludes 'neither' turns, as is done for DistilBERT (Section 4.1: 'binary classification on conversational turns that are valenced, i.e., labeled as either discouraging or uplifting'). If the two binary tasks use different data subsets, the comparison in Figure 4(b) versus Table 2 is invalid. The authors should state explicitly which turns are included in the binary evaluation for each model and ensure the evaluation sets are identical.
  4. [Table 3] Table 3 compares precision and recall between 'the best RoBERTa configuration' and 'the best Llama-3.1 configuration', but the specific configurations are not identified in the table or the text. Without knowing which k, m, weighting, and cross-validation fold produced these numbers, and without any uncertainty estimate, the claim of 'significantly better recall' (Section 5) is unsupported. The authors should identify the exact configurations and provide error bars or significance tests for these per-class metrics.
minor comments (6)
  1. [Section 4.1, 4.2] The paper states that k=2 previous turns and m>7 few-shot examples were excluded because of 'poor results' or 'diminishing returns', but the corresponding results are not shown. Reporting these in an appendix would make the claim verifiable.
  2. [Section 4.1] The use of off-the-shelf sentiment models as zero-shot sequence classifiers maps sentiment labels to micro-behavior labels; this is a strong assumption, because sentiment is not equivalent to the context-dependent micro-behavior construct. A sentence acknowledging this limitation would help calibrate reader expectations.
  3. [Section 5] The word 'significantly' is used without any statistical test ('significantly better recall'); please replace with a quantitative statement or add a supporting test.
  4. [Figure 3] The prompt example in Figure 3 would be easier to follow if the location where the coding definition (when used) is inserted were marked clearly in the figure, since the paper reports that including definitions changes performance.
  5. [References] Reference [8] is missing publication venue information; please complete it. Also, reference [3] appears in a non-standard format; please use the venue's citation style.
  6. [Abstract / Conclusion] The conclusion states that 'LLMs are feasible for this task' without noting that the results are based on a single analog mission corpus with five teams; adding a caveat about generalizability would be more accurate.

Circularity Check

0 steps flagged · score 0.0 of 10

No definitional circularity; the same-team few-shot protocol is an evaluation confound, not a circular step.

full rationale

The paper is an empirical feasibility study rather than a derivation. The decoder-only Llama-3.1 outputs are produced by prompting with in-context examples, and the target turn's own label is never included among those examples, so the prediction is not equivalent to its inputs by construction. The same-team sampling of few-shot examples (Section 4.2) is a genuine validity threat: it makes the Llama evaluation transductive with respect to team, whereas the encoder-only baselines are evaluated with leave-one-team-out cross-validation (Section 4.1), so the reported comparisons (44% vs 41% for 3-way; 68% vs 57% for binary) are not team-independent. This is a methodological confound and a correctness risk, but it is not circularity under the definition used here: no quantity is defined in terms of another, no fitted parameter is reported as a prediction on the data that determined it, and no load-bearing claim is justified solely by self-citation. The authors' prior work [11] is cited only for the motivational claim about repeated micro-behaviors and is not load-bearing for the central result. The lack of variance or significance reporting further weakens the numeric comparisons, but that too is a statistical validity issue, not a circularity issue.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical comparison depends on hand-chosen hyperparameters (context length, number of in-context examples, loss weights, augmentation count) and on assumptions about label reliability, text-only detectability, and the validity of same-team few-shot examples. The same-team sampling is the most consequential, because it makes the Llama evaluation non-independent of the test teams and threatens the main comparison.

free parameters (4)
  • In-context examples per class (m) = 7 (best)
    Llama-3.1 performance improved as m increased from 3 to 7; the headline results use the best configuration.
  • Context window size (k) = 3 or 4 previous turns
    k=2 gave poor results and k>4 gave diminishing returns; results are reported separately for k=3 and k=4.
  • Weighted cross-entropy loss weights = 0.4/0.4/0.2 for valenced/neither
    Hand-chosen weights for RoBERTa 3-way training to counter class imbalance; the weighted and unweighted results differ.
  • Paraphrase augmentation count = 2 per turn
    Training data tripled using Pegasus paraphrases; augmentation helped RoBERTa 3-way but hurt DistilBERT binary, so its effect is config-dependent.
assumptions (4)
  • domain assumption Micro-behaviors (Violation dimension) are sufficiently expressed in the lexical content of conversational turns that text-only classification is meaningful.
    The entire pipeline uses transcripts only; the feasibility claim depends on this assumption (Sections 3 and 4).
  • domain assumption The ground-truth labels produced by the adapted Smith and Griffins coding framework are reliable and consistent enough to train and evaluate classifiers.
    Section 3 describes the labels but reports no inter-rater reliability or annotation procedure.
  • domain assumption The sentiment labels from Twitter RoBERTa and SST-2 DistilBERT map cleanly onto the micro-behavior classes (negative to discouraging, positive to uplifting, neutral to neither).
    Section 4.1 uses this mapping for zero-shot evaluation.
  • ad hoc to paper Few-shot examples drawn from the same team as the test turn are representative and do not leak team-specific information into the evaluation.
    Section 4.2 chooses this sampling design; the Llama results depend on it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missions." pith.science (2026). https://pith.science/paper/74AYUEBE

@misc{pith2026250622679,
  author       = {Pith},
  title        = {Pith review of: Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/74AYUEBE}},
  note         = {Machine review of arXiv:2506.22679}
}
read the original abstract

We explore the feasibility of large language models (LLMs) in detecting subtle expressions of micro-behaviors in team conversations using transcripts collected during simulated space missions. Specifically, we examine zero-shot classification, fine-tuning, and paraphrase-augmented fine-tuning with encoder-only sequence classification LLMs, as well as few-shot text generation with decoder-only causal language modeling LLMs, to predict the micro-behavior associated with each conversational turn (i.e., dialogue). Our findings indicate that encoder-only LLMs, such as RoBERTa and DistilBERT, struggled to detect underrepresented micro-behaviors, particularly discouraging speech, even with weighted fine-tuning. In contrast, the instruction fine-tuned version of Llama-3.1, a decoder-only LLM, demonstrated superior performance, with the best models achieving macro F1-scores of 44% for 3-way classification and 68% for binary classification. These results have implications for the development of speech technologies aimed at analyzing team communication dynamics and enhancing training interventions in high-stakes environments such as space missions, particularly in scenarios where text is the only accessible data.

Figures

Figures reproduced from arXiv: 2506.22679 by the authors.

Figure 1
Figure 1. Illustration of the pre-processing steps for encoder￾only sequence classification [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the instruction prompt creation pipeline for decoder-only causal text generation. classification with pre-trained word embeddings [15, 16, 17], while decoder-only models that leverage autoregressive genera￾tion and in-context learning have not been explored despite their potential advantages in handling nuanced language and contex￾tual dependencies. Additionally, our work focuses on micro￾behavior de… view at source ↗
Figure 3
Figure 3. An example of a 3-way classification few-shot learning prompt without the coding definitions for the decoder-only models. The yellow block contains the instructions, followed by few-shot examples in turquoise blocks, and concludes with the test example in the blue block. If applicable, the coding definition is included in the instruction block. nuanced nature of micro-behaviors, augmented turns were eval￾uated by su… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Macro F1-scores for decoder-only Llama-3.1 on 3- way and 2-way micro-behavior classification, respectively, us￾ing input with k = 3, 4 previous turns, including or excluding the micro-behavior coding definition. Llama-3.1 RoBERTa Uplift Discourage Uplift Discourage Rec…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 27 canonical work pages

  1. [1]

    Introduction Given their effectiveness in spoken language understanding (SLU) [1, 2, 3], competence as few-shot learners [4], and rea- soning abilities [5], large language models (LLMs) can poten- tially infer obscure meanings and semantics from speech in interpersonal communication settings, such as dyadic or mul- tiparty interactions. This capability en...

    work page

  2. [2]

    Assessing the feasibility of Large Language Models for detecting micro-behaviors in team interactions during space missions

    Prior Work Recent research has explored automated methods for detect- ing micro-aggressions, a term used for commonplace verbal, behavioral, or environmental slights that communicate hostile, derogatory, or negative attitudes toward members of marginal- ized groups [12]. These methods include handcrafted lexi- cons [13], unigrams/bi-grams, and topic model...

    work page Pith review arXiv 2025

  3. [3]

    National Aeronautics and Space Administration (NASA)

    Data Description Our data came from five teams that participated in a 45-day mis- sion at the Human Exploration Research Analog (HERA) of the U.S. National Aeronautics and Space Administration (NASA). Each team took part in a simulated space expedition, perform- ing a geological exploration scenario of the journey to and from Mars’s moon, Phobos. In this ...

    work page 2022

  4. [4]

    Methodology In this section, we describe the encoder-only sequence classifi- cation (Section 4.1) and the decoder-only causal text generation (Section 4.2) methods which are used in the 3-way (i.e., uplift- ing, discouraging, and neither) and binary (i.e., uplifting and discouraging) micro-behavior classification tasks. 4.1. Encoder-only Sequence Classifi...

    work page

  5. [5]

    Fine-tuning on in-domain data improves performance

    Results The zero-shot macro F1-score with RoBERTa for 3-way micro- behavior classification was 37% for k = 3, 4. Fine-tuning on in-domain data improves performance. Fine-tuning after adding paraphrases to the training data results in further performance improvement with k = 3, yielding a 41% macro F1-score. All configurations of the RoBERTa pipeline faile...

    work page

  6. [6]

    Conclusion We explored the feasibility of LLMs in detecting micro- behaviors in team conversations during space missions, using conversation transcripts alone. While fine-tuning encoder-only models improved the detection of uplifting and neutral conver- sations as compared to zero-shot classification, they failed to identify discouraging conversations, po...

    work page

  7. [7]

    Are llms robust for spoken dialogues?

    S. M. Mousavi, G. Roccabruna, S. Alghisi, M. Rizzoli, M. Ra- vanelli, and G. Riccardi, “Are llms robust for spoken dialogues?” in HCI International 2024 Posters, 2024

    work page 2024

  8. [8]

    Zero-shot spoken language understanding via large language models: A preliminary study,

    Z. Zhu, X. Cheng, H. An, Z. Wang, D. Chen, and Z. Huang, “Zero-shot spoken language understanding via large language models: A preliminary study,” inProceedings of the 2024 Joint In- ternational Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , 2024, pp. 17 877–17 883

    work page 2024

Show all 32 references
  1. [9]

    Can ChatGPT detect intent? evaluat- ing large language models for spoken language understanding,

    M. He and P. N. Garner, “Can ChatGPT detect intent? evaluat- ing large language models for spoken language understanding,” in Interspeech, 2023, pp. 2163–2167

    2023

  2. [10]

    Language models are few-shot learners,

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B....

    2020 arXiv

  3. [11]

    Chain-of-thought prompting elic- its reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-thought prompting elic- its reasoning in large language models,” inNIPS’22: Proceedings of the 36th International Conference on Neural Information Pro- cessing Systems, 2022, pp....

    2022

  4. [12]

    Can generative artificial intelli- gence productivity tools support workplace learning? a qualitative study on employee perceptions in a multinational corporation,

    T. C. Callari and L. Puppione, “Can generative artificial intelli- gence productivity tools support workplace learning? a qualitative study on employee perceptions in a multinational corporation,” Journal of Workplace Learning, 2025

    2025

  5. [13]

    Bastola, H

    A. Bastola, H. Wang, J. Hembree, P. Yadav, N. McNeese, and A. Razi, “Llm-based smart reply (lsr): Enhancing collab- orative performance with chatgpt-mediated smart reply system (acm)(draft) llm-based smart reply (lsr): Enhancing collaborative performance with chatgpt-mediated ...

    2023 arXiv

  6. [14]

    Conversational ai as the new employee liaison: Llm-powered chatbots in enhancing workplace collaboration and inclusion,

    E. Oluwagbade, “Conversational ai as the new employee liaison: Llm-powered chatbots in enhancing workplace collaboration and inclusion,” 2024

    2024

  7. [15]

    Selective incivility as modern discrimination in organi- zations: Evidence and impact,

    L. M. Cortina, D. Kabat-Farr, E. A. Leskinen, M. Huerta, and V . J. Magley, “Selective incivility as modern discrimination in organi- zations: Evidence and impact,” Journal of management, vol. 39, no. 6, pp. 1579–1605, 2013

    2013

  8. [16]

    Microaggressions, everyday dis- crimination, workplace incivilities, and other subtle slights at work: A meta-synthesis,

    I. A. Smith and A. Griffiths, “Microaggressions, everyday dis- crimination, workplace incivilities, and other subtle slights at work: A meta-synthesis,” Human Resource Development Review, vol. 21, no. 3, pp. 275–299, 2022

    2022

  9. [17]

    What’s that supposed to mean? capturing micro- behaviors in teams,

    S. R. Begerowski, A. M. Khader, P. Paromita, T. Chaspari, and S. T. Bell, “What’s that supposed to mean? capturing micro- behaviors in teams,” in 38th Annual Society for Industrial and Organizational Psychology (SIOP) Conference, 2023

    2023

  10. [18]

    D. W. Sue, Microaggressions in Everyday Life: Race, Gender, and Sexual Orientation. Hoboken, NJ: Wiley, 2010

    2010

  11. [19]

    Automated detection of racial microaggressions using machine learning,

    O. Ali, N. Scheidt, A. Gegov, E. Haig, M. Adda, and B. Aziz, “Automated detection of racial microaggressions using machine learning,” in 2020 IEEE symposium series on computational in- telligence (SSCI). IEEE, 2020, pp. 2477–2484

    2020

  12. [20]

    Finding mi- croaggressions in the wild: A case for locating elusive phenom- ena in social media posts,

    L. Breitfeller, E. Ahn, D. Jurgens, and Y . Tsvetkov, “Finding mi- croaggressions in the wild: A case for locating elusive phenom- ena in social media posts,” in Proceedings of the 2019 confer- ence on empirical methods in natural language processing and the 9th international ...

    2019

  13. [21]

    Leveraging bias in pre- trained word embeddings for unsupervised microaggression de- tection,

    N. Sabri, V . Basile, T. Caselli et al. , “Leveraging bias in pre- trained word embeddings for unsupervised microaggression de- tection,” in Italian Conference on Computational Linguistics 2021: CLiC-it 2021. CEUR Workshop Proceedings (CEUR-WS. org), 2021

    2021

  14. [22]

    Overview of machine learning algorithms for detect- ing microaggression in written text,

    A. Tareque, H. H. Siddegowda, D. J. Frank, N. Lee, and R. Moieni, “Overview of machine learning algorithms for detect- ing microaggression in written text,” Open Journal of Social Sci- ences, vol. 12, no. 7, pp. 347–358, 2024

    2024

  15. [23]

    Towards identification of microaggressions in real-life and scripted conversations, using context-aware machine learning techniques,

    M. K. Ngueajio, I. Hernandez, K. Cornett, G. Washington et al. , “Towards identification of microaggressions in real-life and scripted conversations, using context-aware machine learning techniques,” 2023

    2023

  16. [24]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” 2019. [Online]. Available: https://arxiv.org/abs/1907.11692

    2019 arXiv

  17. [25]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,

    V . Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,”

  18. [27]

    Twitter-roberta-base for sentiment analysis - up- dated (2022),

    CardiffNLP, “Twitter-roberta-base for sentiment analysis - up- dated (2022),” https://huggingface.co/cardiffnlp/twitter-roberta- base-sentiment-latest, 2022

    2022

  19. [28]

    Distilbert base uncased finetuned sst-2,

    DistilBERT-community, “Distilbert base uncased finetuned sst-2,” https://huggingface.co/distilbert/distilbert-base-uncased- finetuned-sst-2-english

  20. [29]

    Pegasus: Pre-training with extracted gap-sentences for abstractive summarization,

    J. Zhang, Y . Zhao, M. Saleh, and P. J. Liu, “Pegasus: Pre-training with extracted gap-sentences for abstractive summarization,”

  21. [30]

    Available: https://arxiv.org/abs/1912.08777

    [Online]. Available: https://arxiv.org/abs/1912.08777

    1912 arXiv

  22. [31]

    A survey on in-context learning,

    Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, B. Chang et al., “A survey on in-context learning,” inPro- ceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 1107–1128

    2024

  23. [32]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, A. Zhang, A. Rodriguez, A. Gregerson, A. Spataru,...

    2024 arXiv

  24. [2020]

    Available: https://arxiv.org/abs/1910.01108

    [Online]. Available: https://arxiv.org/abs/1910.01108

    1910 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.