Pith. sign in

REVIEW 5 major objections 4 minor 40 references

Fine-tuned 0.5B language model classifies leader-follower roles at 86.66% accuracy with 22 ms latency, and the paper introduces the first public dataset for leader-follower HRI communication.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:23 UTC pith:CCTYZ6U5

load-bearing objection Solid applied SLM/HRI paper with a plausible zero-shot result, but the headline accuracy rests on 100 self-labeled test questions and the conclusion contradicts its own length analysis. the 5 major comments →

arxiv 2602.23312 v3 pith:CCTYZ6U5 submitted 2026-02-26 cs.HC cs.AIcs.LGcs.ROcs.SYeess.SY

Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction

Rafael R. Baptista , Andr\'e de Lima Salgado , Ricardo V. Godoy , Marcelo Becker , Thiago Boaventura , Gustavo J. G. Lahr This is my paper
classification cs.HC cs.AIcs.LGcs.ROcs.SYeess.SY
keywords leader-follower interactionsmall language modelshuman-robot interactionzero-shot fine-tuningone-shot interactionprompt engineeringedge computingrole classification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that a small language model, Qwen2.5-0.5B, can be adapted to assign leader-follower roles in human-robot interaction. The authors build a new dataset from DailyDialog, augment it with synthetic paraphrases, and compare fine-tuning against prompt engineering and an untrained baseline. Their zero-shot fine-tuned model achieves 86.66% accuracy with 22.2 ms latency, suggesting edge-deployable, real-time role assignment is feasible. However, adding a clarifying turn (one-shot) degrades performance to near chance, indicating that multi-turn dialogue exceeds the capacity of sub-1B models. If the results hold, they point to simple, single-turn interactions as the most reliable way to use small models for initiative detection on resource-constrained robots.

Core claim

The central claim is that zero-shot fine-tuning of Qwen2.5-0.5B on a synthetic-augmented leader-follower dataset yields reliable role classification: 86.66% accuracy, 84.81% precision, 90.05% recall, with 22.2 ms per sample and 432 tokens/s. This decisively outperforms prompt engineering (53.87%) and the untrained baseline (55.00%) under the zero-shot interaction mode. The same model collapses in one-shot mode (51.65% accuracy, 8.90% recall) when a clarifying question and a simulated ambiguous response are added to the context, suggesting the 0.5B model lacks the capacity to maintain semantic fidelity over longer multi-turn inputs. The authors introduce the first public dataset for leader-fo

What carries the argument

The mechanism is fine-tuning a small language model with a lightweight linear classification head: Qwen2.5-0.5B is adapted for binary leader-follower classification using 5,400 synthetic-augmented questions (315 originals plus LLM paraphrases). The dataset construction pairs DailyDialog-derived questions with role labels (LEADER if the dialogue participant requests guidance, FOLLOWER if they initiate alone or suggest accompaniment), and the one-shot mode additionally inserts a clarifying question and an intentionally ambiguous LLM-generated response. The fine-tuned zero-shot model directly maps a user utterance to the role, while the one-shot setup supplies extra context that, per the paper,

Load-bearing premise

The paper assumes its author-assigned LEADER/FOLLOWER labels on the 100 held-out DailyDialog questions are correct and representative, with no inter-annotator agreement or external validation; if the labels are idiosyncratic, the reported accuracy could reflect the labeling scheme rather than the model's true capability.

What would settle it

Run the fine-tuned model on a new set of leader-follower utterances collected from real human-robot interactions or independently annotated by multiple judges. If accuracy falls far below 86.66%, or if annotators disagree on the labels, the central claim is undermined. A simpler test: check whether the model's classifications agree with a human user's own role assessment in a live interaction.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A fine-tuned 0.5B SLM can perform leader-follower role assignment in about 22 ms per sample, enabling real-time on-device initiative detection for mobile and assistive robots without cloud access.
  • Prompt engineering is insufficient for sub-1B models on this task; fine-tuning is required to beat chance-level performance.
  • Multi-turn clarification is counterproductive for tiny models: longer context progressively degrades accuracy, so concise one-turn interaction design is preferable on the edge.
  • Fine-tuning also improves efficiency: lower latency and higher throughput than baseline and prompt engineering, even when generating more tokens in one-shot mode.
  • The released dataset and pipeline allow reproducible comparison of SLM adaptation strategies for dyadic HRI.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the labels are validated with human annotation, this approach might transfer to other role-negotiation tasks, such as assistance requests versus commands in hospital navigation.
  • The one-shot collapse hints that adding a clarifying question without context pruning could increase coordination breakdowns in real robots; a retrieval-based or question-by-question pipeline might rescue multi-turn capability.
  • The synthetic augmentation recipe (three LLMs producing paraphrases) is a reusable template for building HRI-specific datasets, but its semantic fidelity metrics (cosine similarity) show variability across generators; human judgment would strengthen it.
  • The reported 86.66% accuracy is based on a 100-sample test set; a larger multi-annotator benchmark would likely narrow the confidence interval and test generalizability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces a novel leader-follower HRI dataset derived from DailyDialog and augmented with LLM-generated paraphrases, and evaluates Qwen2.5-0.5B under prompt engineering versus fine-tuning in zero-shot and one-shot interaction modes. The central empirical claim is that zero-shot fine-tuning achieves robust classification (86.66% accuracy, 22.2 ms latency), significantly outperforming baseline and prompt-engineered approaches, while one-shot fine-tuning degrades to near chance (51.65%), which the authors attribute to increased context length exceeding the 0.5B model's capacity. The paper also reports a sentence-length analysis and positions the dataset and efficiency measurements as contributions to edge-deployed HRI.

Significance. If the results hold, the paper provides a useful benchmark artifact and shows that a sub-1B model can perform single-turn role classification efficiently, which is of practical relevance for resource-constrained robots. The public dataset, 30-run Monte Carlo cross-validation, and reported efficiency metrics are concrete strengths that support reproducibility. However, the central accuracy claim is only as strong as the author-assigned labels on the 100-sample held-out set, and the one-shot degradation explanation is confounded by uncontrolled differences between interaction modes. The paper's significance therefore depends on whether the label validity and controlled-experiment concerns can be resolved.

major comments (5)
  1. [§II-A, Table I] The central 86.66% accuracy is computed against a 100-sample held-out set whose labels were assigned by the authors using a rule: 'requested guidance or directions' => LEADER; 'initiated a task alone or suggested being accompanied' => FOLLOWER. No inter-annotator agreement, external validation, or human evaluation is reported, and the SBERT semantic-fidelity check only measures embedding similarity between synthetic and real samples within the same class; it does not validate label correctness. Because the training set is labeled by the same rule, the fine-tuned model may simply reproduce the authors' operationalization rather than learn a generalizable HRI role concept. The rule itself is non-obvious: labeling the requester as LEADER and the self-initiating participant as FOLLOWER requires justification against established HRI definitions. Please add an independent annotation study with
  2. [§III-A, Table I] The paper attributes the one-shot accuracy drop (86.66% to 51.65%) to 'increased context length' and 'limited parameter capacity,' but the one-shot condition differs from zero-shot in several confounded ways: it uses a different dataset (with LLM-generated clarification and 'scarecrow' responses), a two-model pipeline, and longer inputs. No ablation isolates context length, such as truncating one-shot inputs, padding zero-shot inputs to similar length, or using human clarification responses. The sentence-length analysis in Fig. 4 also compares non-overlapping bin ranges (15–55 vs 130–180 characters), so it cannot support a length-driven explanation. Provide controlled experiments or soften the causal claim to a measured correlation.
  3. [§III-A] The z-test is applied to 30 MCCV iterations, but these accuracy estimates are not independent because random train/test splits overlap across iterations. The text reports 'p < 0.001' and 'p > 0.001' as thresholds without stating whether the test is paired, what the null hypothesis is, or how multiple comparisons were handled. Please specify the exact statistical test, report effect sizes and confidence intervals, and justify the use of overlapping samples. Without this, the claim that fine-tuning significantly outperforms baseline and prompt engineering is not fully established.
  4. [§IV] The conclusion states that 'one-shot configurations are less sensitive to input complexity and length than zero-shot models,' which is the opposite of the results in §III-B and the abstract (zero-shot is robust; one-shot shows a progressive decline). As written, this sentence directly contradicts the paper's central finding and must be corrected to avoid misleading readers.
  5. [§I, §III-A] The paper motivates the work by edge deployment and reports latency/throughput (Table I), but no inference hardware, software stack, batch size, or quantization is specified. Fine-tuning was performed on NVIDIA L40S GPUs; if inference also ran on server-class hardware, the 22.2 ms latency does not demonstrate edge viability. Specify the evaluation platform or temper the edge-deployment claim.
minor comments (4)
  1. [References, [17]] Reference [17] contains an editorial annotation: 'peer-reviewed? Note: This is a technical report. You might want to check...' This appears to be a leftover review note in the manuscript. Please clean it and verify whether the citation should be a formal publication or an arXiv technical report.
  2. [§III-B, Fig. 4] The figure x-axis labels (15, 23, 31, 39, 47, 55 and 130, 140, 150, 160, 170, 180) do not clearly correspond to the stated 'step size of 10 characters.' Please specify bin edges or bin centers explicitly.
  3. [§III-A, Table I] The 'Baseline' condition is described only as 'untrained.' It is unclear whether the baseline is a randomly initialized model, a majority-class classifier, or the pretrained model with no prompt. Define the baseline concretely, since all comparative claims reference it.
  4. [§III-A] The phrase 'z-test, p > 0.001' is unconventional; consider using a standard significance level (e.g., p < 0.05) and reporting exact p-values or confidence intervals.

Circularity Check

0 steps flagged

No significant circularity; the central evaluation uses a held-out human test set and no prediction reduces to a fitted input or self-citation.

full rationale

The paper is an empirical benchmark rather than a derivation, and its main claim—zero-shot fine-tuning reaches 86.66% accuracy—is measured against 100 original DailyDialog sentences reserved before synthetic augmentation, so the test set is not part of the training data and the metric is not fitted. The role labels are assigned by the authors' stated rule ('requested guidance or directions' => LEADER; 'initiated a task alone or suggested being accompanied' => FOLLOWER), and the same rule is used for both training and test labels; this is a standard supervised setup, not a case where the predicted quantity is defined in terms of the fitted model. No parameter is fitted to the reported accuracy, no 'prediction' is a renamed fit, and no load-bearing conclusion depends on a self-citation chain: the cited Qwen2.5 technical report supports model choice but is not used to derive the accuracy result. The paper's own limitation statement—'our dataset, while novel, is synthetic and limited in scale; extending it with larger, multimodal, and human-annotated samples will strengthen the ecological validity'—flagges a validity/generalizability concern about the unvalidated labels, not circularity. The one-shot scarecrow responses are LLM-generated in both training and evaluation, which may limit realism, but the paper reports one-shot degradation as a capacity effect and does not claim external human-validation for it. Overall, the held-out evaluation is independent of the training inputs, so there is no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The paper's central claims rest on subjectively labeled data, synthetic augmentation, and a single model. The benchmark is useful, but these unvalidated assumptions limit the strength of the conclusions.

free parameters (5)
  • Synthetic paraphrase count per original question = 6
    Chosen ad hoc; no ablation shows 6 is optimal or sufficient.
  • Fine-tuning epochs = 10
    Set in Autotrain; no sensitivity analysis reported.
  • Validation split = 20%
    Used for best-model selection; may affect reported accuracy.
  • MCCV iterations = 30
    Chosen for statistical de-biasing; no justification for the number.
  • Sentence-length bin width = 10 characters
    Arbitrary bin width shapes the sentence-length analysis.
axioms (6)
  • ad hoc to paper DailyDialog questions can be mapped to leader-follower roles by the authors' labeling rule
    Labels are assigned by the authors without inter-annotator agreement or validation against HRI ground truth.
  • domain assumption LLM-generated paraphrases preserve leader-follower semantics
    Only SBERT cosine similarity is used; no human evaluation of synthetic labels.
  • domain assumption The scarecrow LLM responses are realistic ambiguous user replies
    One-shot test data depends on this assumption; no human validation.
  • domain assumption Qwen2.5-0.5B is representative of sub-1B edge SLMs
    Single model evaluated; no comparison with other SLMs.
  • domain assumption Accuracy on 100 held-out phrases is a reliable estimate of deployment performance
    Small test set; no real-robot evaluation.
  • domain assumption SBERT cosine similarity is a valid semantic fidelity metric
    Used to validate synthetic data quality; not a ground-truth measure.

pith-pipeline@v1.3.0-alltime-deepseek · 10219 in / 11972 out tokens · 98466 ms · 2026-08-02T20:23:07.226489+00:00 · methodology

0 comments
read the original abstract

Leader-follower interaction is an important paradigm in human-robot interaction (HRI). Yet, assigning roles in real time remains challenging for resource-constrained mobile and assistive robots. While large language models (LLMs) have shown promise for natural communication, their size and latency limit on-device deployment. Small language models (SLMs) offer a potential alternative, but their effectiveness for role classification in HRI has not been systematically evaluated. In this paper, we present a benchmark of SLMs for leader-follower communication, introducing a novel dataset derived from a published database and augmented with synthetic samples to capture interaction-specific dynamics. We investigate two adaptation strategies: prompt engineering and fine-tuning, studied under zero-shot and one-shot interaction modes, compared with an untrained baseline. Experiments with Qwen2.5-0.5B reveal that zero-shot fine-tuning achieves robust classification performance (86.66% accuracy) while maintaining low latency (22.2 ms per sample), significantly outperforming baseline and prompt-engineered approaches. However, results also indicate a performance degradation in one-shot modes, where increased context length challenges the model's architectural capacity. These findings demonstrate that fine-tuned SLMs provide an effective solution for direct role assignment, while highlighting critical trade-offs between dialogue complexity and classification reliability on the edge.

Figures

Figures reproduced from arXiv: 2602.23312 by Andr\'e de Lima Salgado, Gustavo J. G. Lahr, Marcelo Becker, Rafael R. Baptista, Ricardo V. Godoy, Thiago Boaventura.

Figure 1
Figure 1. Figure 1: Edge-deployed leader–follower classification for as [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the dataset construction and model evaluation pipeline. The top section illustrates the creation of zero [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Accuracy dispersion across 30 runs for each in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Accuracy as a function of sentence length for the fine-tuned models for a) Zero-shot and b) One-shot. Error bars [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 1 canonical work pages

  1. [1]

    Redraw- ing boundaries: Systemic impacts of rehabilitation robots in clinical care settings,

    A. Asadi, D. Herath, G. Shaw, G. Caldwell, and E. Williams, “Redraw- ing boundaries: Systemic impacts of rehabilitation robots in clinical care settings,” in2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 2025, pp. 1216–1220

    2025

  2. [2]

    Adebayo, J

    S. Adebayo, J. C. Dessing, and S. McLoone,Dyadic Human-Robot Interaction: Emerging Technologies, Challenges, and Opportunities. Singapore: Springer Nature Singapore, 2026, pp. 1–51. [Online]. Available: https://doi.org/10.1007/978-981-96-9471-6 1

    work page doi:10.1007/978-981-96-9471-6 2026

  3. [3]

    Exploring social robots for healthy older adults: Aging with companionship,

    J. Tschida, K. Michael, and T. McDaniel, “Exploring social robots for healthy older adults: Aging with companionship,”IEEE Transactions on Technology and Society, 2025

    2025

  4. [4]

    Role spe- cialization enables superior task performance by human dyads than individuals,

    A. Takai, Q. Fu, Y . Doibata, G. Lisi, T. Tsuchiya, K. Mojtahedi, T. Yoshioka, M. Kawato, J. Morimoto, and M. Santello, “Role spe- cialization enables superior task performance by human dyads than individuals,”The International Journal of Robotics Research, 8 2025

    2025

  5. [5]

    Human leading or following preferences: Effects on human percep- tion of the robot and the human–robot collaboration,

    A. Noormohammadi-Asl, K. Fan, S. L. Smith, and K. Dautenhahn, “Human leading or following preferences: Effects on human percep- tion of the robot and the human–robot collaboration,”Robotics and Autonomous Systems, vol. 183, p. 104821, 2025

    2025

  6. [6]

    Influencing leading and following in human-robot teams

    M. Kwon, M. Li, A. Bucquet, and D. Sadigh, “Influencing leading and following in human-robot teams.” inRobotics: Science and Systems, 2019

    2019

  7. [7]

    Communication in human-robot interaction,

    A. Bonarini, “Communication in human-robot interaction,”Current Robotics Reports, vol. 1, no. 4, pp. 279–285, 2020

    2020

  8. [8]

    Natural language processing in the era of large language models,

    A. Zubiaga, “Natural language processing in the era of large language models,” p. 1350306, 2024

    2024

  9. [9]

    FurChat: An embodied conversational agent using LLMs, combining open and closed- domain dialogue with facial expressions,

    N. Cherakara, F. Varghese, S. Shabana, N. Nelson, A. Karukayil, R. Kulothungan, M. Afil Farhan, B. Nesset, M. Moujahid, T. Dinkar, V . Rieser, and O. Lemon, “FurChat: An embodied conversational agent using LLMs, combining open and closed- domain dialogue with facial expressions,” inProceedings of the 24th Annual Meeting of the Special Interest Group on Di...

    2023

  10. [10]

    Lami: Large language models for multi-modal human-robot interaction,

    C. Wang, S. Hasler, D. Tanneberg, F. Ocker, F. Joublin, A. Ceravola, J. Deigmoeller, and M. Gienger, “Lami: Large language models for multi-modal human-robot interaction,” inExtended Abstracts of the CHI Conference on Human Factors in Computing Systems, 2024, pp. 1–10

    2024

  11. [11]

    Enhancing human-robot collaborative assembly in manufacturing sys- tems using large language models,

    J. Lim, S. Patel, A. Evans, J. Pimley, Y . Li, and I. Kovalenko, “Enhancing human-robot collaborative assembly in manufacturing sys- tems using large language models,” in2024 IEEE 20th International Conference on Automation Science and Engineering (CASE). IEEE, 2024, pp. 2581–2587

    2024

  12. [12]

    An empirical analysis and resource footprint study of deploying large language models on edge devices,

    N. Dhar, B. Deng, D. Lo, X. Wu, L. Zhao, and K. Suo, “An empirical analysis and resource footprint study of deploying large language models on edge devices,” inProceedings of the 2024 ACM southeast conference, 2024, pp. 69–76

    2024

  13. [13]

    Small language models learn enhanced reasoning skills from medical textbooks,

    H. Kim, H. Hwang, J. Lee, S. Park, D. Kim, T. Lee, C. Yoon, J. Sohn, J. Park, O. Reykhart, T. Fetherston, D. Choi, S. H. Kwak, Q. Chen, and J. Kang, “Small language models learn enhanced reasoning skills from medical textbooks,”npj Digital Medicine, vol. 8, no. 1, p. 240, May 2025. [Online]. Available: https://doi.org/10.1038/s41746-025-01653-8

    doi:10.1038/s41746-025-01653-8 2025

  14. [14]

    Are we there yet? a measurement study of efficiency for llm applications on mobile devices,

    X. Yan and Y . Ding, “Are we there yet? a measurement study of efficiency for llm applications on mobile devices,” inProceedings of the 2nd International Workshop on Foundation Models for Cyber- Physical Systems & Internet of Things, 2025, pp. 19–24

    2025

  15. [15]

    Edgellm: Fast on-device LLM inference with speculative decoding,

    D. Xu, W. Yin, H. Zhang, X. Jin, Y . Zhang, S. Wei, M. Xu, and X. Liu, “Edgellm: Fast on-device LLM inference with speculative decoding,”IEEE Transactions on Mobile Computing, vol. 24, no. 4, pp. 3256–3273, 2025. [Online]. Available: https: //ieeexplore.ieee.org/document/10812936

    arXiv 2025

  16. [16]

    Edgemoe: Empowering sparse large language models on mobile devices,

    R. Yi, L. Guo, S. Wei, A. Zhou, S. Wang, and M. Xu, “Edgemoe: Empowering sparse large language models on mobile devices,” IEEE Transactions on Mobile Computing, 2025, early Access / in press. [Online]. Available: https://www.computer.org/csdl/journal/tm/ 2025/08/10906629/24GiOh3U4pi

    2025

  17. [17]

    Qwen2.5 technical report,

    Qwen Team (An Yang et al.), “Qwen2.5 technical report,” arXiv preprint arXiv:2412.15115, 2024, peer-reviewed? Note: This is a tech- nical report. You might want to check if Qwen2.5 formal evaluation appears in any conference or journal paper; if not, include with caution

    Pith/arXiv arXiv 2024

  18. [18]

    Tiny large language models in embedded NVIDIA portable hardware: Comparative analysis,

    M. A. J. G ´omez, R. Jim ´enez-Moreno, and A. A. Espitia-Cubillos, “Tiny large language models in embedded NVIDIA portable hardware: Comparative analysis,”Journal of Hunan University (Natural Sciences), vol. 52, no. 4, pp. 12–24, 2025. [Online]. Available: https://jonuns.com/index.php/journal/article/view/1722/1714

    2025

  19. [19]

    State of the art and future directions of small language models: A systematic review,

    F. Corradini, M. Leonesi, and M. Piangerelli, “State of the art and future directions of small language models: A systematic review,” Big Data and Cognitive Computing, vol. 9, no. 7, p. 189, 2025. [Online]. Available: https://www.mdpi.com/2504-2289/9/7/189

    2025

  20. [20]

    An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing,

    S. Sivarajkumar, M. Kelley, A. Samolyk-Mazzanti, S. Visweswaran, and Y . Wang, “An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing,”JMIR Medical Informatics, vol. 12, no. 1, 2024. [Online]. Available: https://medinform.jmir.org/2024/1/e55318

    2024

  21. [21]

    Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,

    P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,”ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023

    2023

  22. [22]

    Multilingual prompts in llm-based recommenders: Per- formance across languages,

    M. G. Ozsoy, “Multilingual prompts in llm-based recommenders: Per- formance across languages,”arXiv preprint arXiv:2409.07604, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Simple llm prompting is state-of-the-art for robust and multilingual dialogue evaluation,

    J. Mendonc ¸a, P. Pereira, H. Moniz, J. P. Carvalho, A. Lavie, and I. Trancoso, “Simple llm prompting is state-of-the-art for robust and multilingual dialogue evaluation,”arXiv preprint arXiv:2308.16797, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Synthetic Data Generation Using Large Language Models: Advances in Text and Code,

    M. Nadas, L. Diosan, and A. Tomescu, “Synthetic Data Generation Using Large Language Models: Advances in Text and Code,”IEEE Access, vol. 13, pp. 134 615–134 633, 2025, arXiv:2503.14023 [cs]. [Online]. Available: http://arxiv.org/abs/2503.14023

    arXiv 2025

  25. [25]

    Few-shot LLM Synthetic Data with Distribution Matching,

    J. Ren, Z. Du, Z. Wen, Q. Jia, S. Dai, C. Wu, and Z. Dong, “Few-shot LLM Synthetic Data with Distribution Matching,” Feb. 2025, arXiv:2502.08661 [cs]. [Online]. Available: http://arxiv.org/abs/ 2502.08661

    Pith/arXiv arXiv 2025

  26. [26]

    Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction,

    M. Josifoski, M. Sakota, M. Peyrard, and R. West, “Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction,” Oct. 2023, arXiv:2303.04132 [cs]. [Online]. Available: http://arxiv.org/abs/2303.04132

    Pith/arXiv arXiv 2023

  27. [27]

    Comparison of Prompt Engineering and Fine-Tuning Strategies in Large Language Models in the Classification of Clinical Notes,

    X. Zhang, N. Talukdar, S. Vemulapalli, S. Ahn, J. Wang, H. Meng, S. M. B. Murtaza, D. Leshchiner, A. A. Dave, D. F. Joseph, M. Witteveen-Lane, D. Chesla, J. Zhou, and B. Chen, “Comparison of Prompt Engineering and Fine-Tuning Strategies in Large Language Models in the Classification of Clinical Notes,” medRxiv, p. 2024.02.07.24302444, Feb. 2024. [Online]....

    2024

  28. [28]

    Dailydialog: A manually labelled multi-turn dialogue dataset,

    Y . Li, H. Su, X. Shen, W. Li, Z. Cao, and S. Niu, “Dailydialog: A manually labelled multi-turn dialogue dataset,”arXiv preprint arXiv:1710.03957, 2017

    Pith/arXiv arXiv 2017

  29. [29]

    On llms-driven synthetic data generation, curation, and evaluation: A survey,

    L. Long, R. Wang, R. Xiao, J. Zhao, X. Ding, G. Chen, and H. Wang, “On llms-driven synthetic data generation, curation, and evaluation: A survey,”arXiv preprint arXiv:2406.15126, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Data augmentation using llms: Data perspectives, learning paradigms and challenges,

    B. Ding, C. Qin, R. Zhao, T. Luo, X. Li, G. Chen, W. Xia, J. Hu, L. A. Tuan, and S. Joty, “Data augmentation using llms: Data perspectives, learning paradigms and challenges,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 1679–1705

    2024

  31. [31]

    Fine-tuning bert for low- resource natural language understanding via active learning,

    D. Grießhaber, J. Maucher, and N. T. Vu, “Fine-tuning bert for low- resource natural language understanding via active learning,”arXiv preprint arXiv:2012.02462, 2020

    Pith/arXiv arXiv 2012

  32. [32]

    Raft: A real-world few-shot text classification benchmark,

    N. Alex, E. Lifland, L. Tunstall, A. Thakur, P. Maham, C. J. Riedel, E. Hine, C. Ashurst, P. Sedille, A. Carlieret al., “Raft: A real-world few-shot text classification benchmark,”arXiv preprint arXiv:2109.14076, 2021

    Pith/arXiv arXiv 2021

  33. [33]

    Scarecrows in oz: the use of large language models in hri,

    T. Williams, C. Matuszek, R. Mead, and N. Depalma, “Scarecrows in oz: the use of large language models in hri,” pp. 1–11, 2024

    2024

  34. [34]

    Gpt-4 as a moral reasoner for robot command rejection,

    R. Wen, F. Ferraro, and C. Matuszek, “Gpt-4 as a moral reasoner for robot command rejection,” inProceedings of the 12th International Conference on Human-Agent Interaction, 2024, pp. 54–63

    2024

  35. [35]

    A survey on evaluation of large language models,

    Y . Chang, X. Wang, J. Wang, Y . Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y . Wanget al., “A survey on evaluation of large language models,”ACM transactions on intelligent systems and technology, vol. 15, no. 3, pp. 1–45, 2024

    2024

  36. [36]

    Making pre-trained language models better few-shot learners,

    T. Gao, A. Fisch, and D. Chen, “Making pre-trained language models better few-shot learners,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), 2021, pp. 3816–3830

    2021

  37. [37]

    Llm-based nlg evaluation: Current status and challenges,

    M. Gao, X. Hu, X. Yin, J. Ruan, X. Pu, and X. Wan, “Llm-based nlg evaluation: Current status and challenges,”Computational Linguistics, pp. 1–27, 2025

    2025

  38. [38]

    Scaling instruction- finetuned language models,

    H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y . Tay, W. Fedus, Y . Li, X. Wang, M. Dehghani, S. Brahmaet al., “Scaling instruction- finetuned language models,”Journal of Machine Learning Research, vol. 25, no. 70, pp. 1–53, 2024

    2024

  39. [39]

    The ultimate guide to fine-tuning llms from basics to breakthroughs: An exhaustive review of technologies, research, best practices, applied research challenges and opportunities,

    V . B. Parthasarathy, A. Zafar, A. Khan, and A. Shahid, “The ultimate guide to fine-tuning llms from basics to breakthroughs: An exhaustive review of technologies, research, best practices, applied research challenges and opportunities,”arXiv preprint arXiv:2408.13296, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Hallucination detection in foundation models for decision-making: A flexible defi- nition and review of the state of the art,

    N. Chakraborty, M. Ornik, and K. Driggs-Campbell, “Hallucination detection in foundation models for decision-making: A flexible defi- nition and review of the state of the art,”ACM Computing Surveys, vol. 57, no. 7, pp. 1–35, 2025

    2025