Pith. sign in

REVIEW 4 major objections 6 minor 4 cited by

RIRO: Reshaping Inputs, Refining Outputs Unlocking the Potential of Large Language Models in Data-Scarce Contexts

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Reformulating inputs and reshaping outputs lifts fine-tuned LLM scores

desk verdict A plausible pipeline undermined by an unspecified output reshaper that makes Table 1 circular; reject in current form, but the ablation design hints at a fixable idea. read the letter →

arxiv 2412.15254 v1 pith:KNT6KGNO submitted 2024-12-15 cs.CL

Ali Hamdi , Hozaifa Kassab , Mohamed Bahaa , Marwa Mohamed This is my paper · ORCID
classification cs.CL
keywords largelanguagemodelsdatascarcityfine-tuningQLoRAinputreformulationoutputreshapingtestcasegenerationuserstories
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that wrapping a fine-tuned language model in two extra LLM layers—one that normalizes the input and one that reshapes the output—makes the model produce better results when training data is scarce. The test bed is generating software test cases from user stories, with only a small domain-specific dataset for fine-tuning. The paper reports that the full two-layer pipeline outperforms fine-tuning alone on every metric it measures, including a BLEU increase from 0.55 to 0.72 and a cosine similarity increase from 0.816 to 0.891. A sympathetic reader would care because data scarcity is common in specialized fields, and this is a practical recipe for squeezing more accuracy out of a small fine-tuning set without retraining the base model.

What carries the argument

The machinery is a two-layer architecture around a QLoRA-fine-tuned base LLM (Phi-2 in the main experiments). The first layer is a reformulation function $x' = f_r(x)$ that rewrites a raw user story into the 'Action, Condition, Result' structure used during fine-tuning. The second layer is a reshaping function $y' = \text{Reshape}(y)$ that edits the model's raw output into a clean, test-case format. QLoRA keeps the fine-tune affordable by quantizing the base weights to 4-bit and updating only a low-rank adaptation $W_q + UV^T$. The ablation compares the full stack (Reformulation–Fine-tuning–Reshaping) with each one-layer variant to attribute the gains.

What would settle it

Give the reshaping layer the fine-tuned model's raw outputs but withhold all reference test cases, then compare its edits to the references; if BLEU and ROUGE still jump, the gain is genuine repair, whereas if the layer's edits match reference phrasing only when it has seen the reference distribution, the improvement is an artifact of the evaluation setup.

Watch

Extended reading notes

Core claim

The central claim is that the RIRO pipeline—reformulate input, fine-tune with QLoRA, reshape output—demonstrates superior performance across all metrics compared with fine-tuning alone. On the user-story benchmark, the full Stacked LLM version raises BLEU from 0.55 to 0.72, ROUGE-1 from 0.265 to 0.402, ROUGE-2 from 0.128 to 0.149, ROUGE-L from 0.172 to 0.257, and cosine similarity from 0.816 to 0.891, while reducing Levenshtein distance from 1157.620 to 1000.880. The paper interprets these numbers as evidence that input normalization aligns the test input with the training distribution and that output reshaping fixes residual inconsistencies, so the two operations are complementary rather than redundant. It also reports that the full pipeline beats either layer applied alone in the ablation study.

Load-bearing premise

The whole improvement rests on the assumption that a user story can be rewritten into the training format without changing its requirements, and that the reshaping layer can edit the output without adding or dropping content.

Editorial extensions

If this is right

  • If the reported gains hold, practitioners with small domain datasets can improve generation quality by adding an input-normalization prompt before fine-tuning, without changing the base model.
  • The ablation implies that each layer contributes: skipping either reformulation or reshaping lowers at least some of the metrics, so the two operations are not interchangeable.
  • Because QLoRA is used for the fine-tune, the added layers do not require full retraining of the backbone, making the recipe affordable for modest compute budgets.
  • The same wrapper could transfer to other structured-output tasks where a small reference corpus defines the expected format, such as report generation or requirement-to-specification conversion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension, not explored in the paper, is to measure whether the reformulation layer preserves meaning under paraphrased inputs; if it does, the same wrapper could serve as a drop-in robustness module for out-of-distribution inputs.
  • The architecture invites a fair-comparison experiment against deterministic template normalization: if a rule-based normalizer matches the LLM reformulation's gains, the improvement is about format alignment rather than LLM understanding.
  • Testing the pipeline on a second data-scarce domain, such as legal clause summarization, with the same metrics would show whether the gains generalize beyond user stories.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes RIRO, a pipeline intended to improve LLM fine-tuning in data-scarce settings, applied to generating test cases from user stories. The pipeline consists of an LLM-based input reformulation step (x' = fr(x)), QLoRA fine-tuning, and an LLM-based output reshaping step (y' = Reshape(y)). The paper claims that the full pipeline (RIRO) outperforms its own variants and the fine-tuned baseline across BLEU, ROUGE-1/2/L, Levenshtein distance, and cosine similarity, based on Table 1. The manuscript also introduces a benchmark built from a subset of the 'user story neodataset.' No code, data, or detailed experimental protocol is provided.

Significance. If the claimed gains were established, the idea of wrapping a fine-tuned LLM with input normalization and output reshaping would be a pragmatic contribution for low-resource applications, and the introduced benchmark could be useful to the community. However, the manuscript as written does not establish the central claim: the output-reshaping step is undefined, the ablation variants are not reported, the evaluation lacks any protocol details, and the table's component labels contradict the architecture diagram. The paper provides no derivations, machine-checked artifacts, or reproducible code; its only falsifiable output is Table 1, which cannot be checked or interpreted. The significance of the work is therefore currently unassessable.

major comments (4)
  1. [§4.4, Table 1] The output-reshaping function y' = Reshape(y) is never specified: the paper does not state whether it is an LLM call, what prompt or model it uses, whether it is fine-tuned, or whether it has access to the reference test cases. This is load-bearing because RIRO's Levenshtein distance drops from 1157.080 (Reshaping) and 1420.500 (Refining) to 1000.880, which is exactly the pattern expected if a post-processor rewrites outputs toward reference-like strings. A reference-blind evaluation (or at minimum a generic post-processing baseline without references) is required to rule out circularity in the reported BLEU, ROUGE, and Levenshtein gains.
  2. [§5, Table 1] The evaluation is not reproducible: no dataset size, train/validation/test split, number of test examples, QLoRA rank, learning rate, batch size, epochs, or decoding parameters are reported, and there are no error bars or statistical significance tests. The differences between some configurations are very small (e.g., ROUGE-2: 0.147 vs 0.149), so the claim that RIRO 'demonstrates superior performance across all metrics' is unsupported. The abstract also promises results for Falcon 7B and Falcon 1B, but only Phi-2 results appear in Table 1.
  3. [§4.5, §5, Figure 1] The ablation study is described in Section 4.5 with variants LLM-RFR, LLM-RF, and LLM-FR, but none of these variants appears in Table 1. Instead, Table 1 lists 'Reshaping,' 'Refining,' and 'RIRO,' and Section 5 labels Reshaping as input-focused and Refining as output-focused, which is the opposite of Figure 1, where (a) Refining LLM performs input normalization and (b) Reshaping LLM performs output reshaping. This contradiction makes it impossible to determine which component drives the improvement.
  4. [§3, §4.2] The dataset description is incomplete: 'a subset of the user story neodataset' with no size, split, filtering criteria, or an explanation of how the issue fields (title, description, story points) are converted into user stories and reference test cases. Similarly, the reformulation function fr is defined only symbolically, with no implementation, prompt, target format, or example. Without these details, the benchmark is ill-specified and the experiment cannot be replicated or independently evaluated.
minor comments (6)
  1. [Abstract] The sentence 'Through fine-tuning models like Phi-2, Falcon 7B, and Falcon 1B, with Phi-2 outperforming the others' is a sentence fragment and should be revised.
  2. [§1] The phrase 'working in Adjective way' appears to be a typo; likely 'additive' or 'complementary' was intended.
  3. [§1, §4.5] The introduction says 'RIRO is composed of three model architectures' but the abstract and methodology describe a two-layer architecture; the terminology should be harmonized across sections.
  4. [§5] The table caption and surrounding text should use the same names as Figure 1 for the three architectures, or explicitly state why different names are used.
  5. [§5] The paper claims superiority over 'existing state-of-the-art methods' but provides no external baseline; at least one non-LLM or standard LLM baseline from the related work should be included for comparison.
  6. [References] Reference 9 (QLoRA) contains a URL with a line break ('URLhttps://arxiv.org/abs/2305.14314'); the formatting should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No demonstrated circularity: RIRO's claims rest on an empirical comparison; the unspecified Reshape layer raises validity concerns but no quoted equation or self-citation reduces the result to its inputs.

full rationale

The paper does not contain a derivation chain whose conclusions are equivalent to its premises. The central claim (Table 1) is an empirical performance comparison. The reformulation and reshaping layers are defined symbolically as x' = fr(x) and y' = Reshape(y) (Sections 4.2 and 4.4), with no implementation details; one cannot exhibit from the text any mechanism by which Reshape uses the reference strings used in BLEU/ROUGE/Levenshtein computation. The suggestive Levenshtein drop is consistent with reference-informed post-processing but does not by itself demonstrate it. While this missing specification is a serious reproducibility limitation, and the absent LLM-RF/LLM-FR ablation numbers (Section 4.5) weaken the ablation claim, absence of evidence is not circular equivalence. The self-citations in Related Work are survey/positioning references and are not load-bearing for the main result. No fitted constant is renamed as a prediction, and no uniqueness theorem or ansatz is imported from prior work by the same authors. Under the required standard of quoting a specific reduction, no circular step can be identified.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper contributes a pipeline but no derivation. Its claims rest on a small, unnamed dataset, unstated hyperparameters, and evaluation metrics used without validation. The reformulation and reshaping layers are the largest unexamined components.

free parameters (3)
  • QLoRA rank r = not reported
    The paper defines NQLoRA = r times k but never gives the rank or number of fine-tuned parameters used in experiments.
  • Reformulation target format = Action, Condition, Result template
    The reformulation layer aligns inputs to a hand-picked template; the choice is not justified or varied in an ablation.
  • Fine-tuning hyperparameters (epochs, learning rate, batch size) = not reported
    None of the training settings are given, so results cannot be reproduced.
assumptions (4)
  • domain assumption Reformulation via an extra LLM call preserves the task-relevant content of user stories.
    Section 4.2 defines x' = fr(x) but gives no evidence that fr is lossless or that errors in reformulation are negligible.
  • domain assumption The small user story subset is large enough for QLoRA fine-tuning to learn the mapping.
    Section 3 describes the dataset only as a subset with no size or statistics.
  • domain assumption BLEU, ROUGE, Levenshtein distance, and cosine similarity are valid proxies for test case quality.
    Section 4.6 lists these metrics but does not validate them against human judgment or execution of generated test cases.
  • domain assumption The output reshaping layer does not degrade semantic correctness.
    Section 4.4 defines y' = Reshape(y) but offers no analysis of what Reshape does or when it fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RIRO: Reshaping Inputs, Refining Outputs Unlocking the Potential of Large Language Models in Data-Scarce Contexts." pith.science (2026). https://pith.science/paper/KNT6KGNO

@misc{pith2026241215254,
  author       = {Pith},
  title        = {Pith review of: RIRO: Reshaping Inputs, Refining Outputs Unlocking the Potential of Large Language Models in Data-Scarce Contexts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KNT6KGNO}},
  note         = {Machine review of arXiv:2412.15254}
}
read the original abstract

Large language models (LLMs) have significantly advanced natural language processing, excelling in areas like text generation, summarization, and question-answering. Despite their capabilities, these models face challenges when fine-tuned on small, domain-specific datasets, often struggling to generalize and deliver accurate results with unfamiliar inputs. To tackle this issue, we introduce RIRO, a novel two-layer architecture designed to improve performance in data-scarce environments. The first layer leverages advanced prompt engineering to reformulate inputs, ensuring better alignment with training data, while the second layer focuses on refining outputs to minimize inconsistencies. Through fine-tuning models like Phi-2, Falcon 7B, and Falcon 1B, with Phi-2 outperforming the others. Additionally, we introduce a benchmark using evaluation metrics such as cosine similarity, Levenshtein distance, BLEU score, ROUGE-1, ROUGE-2, and ROUGE-L. While these advancements improve performance, challenges like computational demands and overfitting persist, limiting the potential of LLMs in data-scarce, high-stakes environments such as healthcare, legal documentation, and software testing.

Figures

Figures reproduced from arXiv: 2412.15254 by the authors.

Figure 1
Figure 1. The proposed model architectures for RIRO Versions. (a) Refining LLM: This architecture focuses on input normalization. It aligns the input user stories with the training data distribution. (b) Reshaping LLM: Here, the output reshaping layer to ensure coherent test cases. This method adjusts the final output to maintain consistency and accuracy. (c) Stacked LLM: A combined approach that first normalizes the input, p… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Arabic Large Language Models for Medical Text Generation

    cs.CL 2025-09 reject novelty 4.0 of 10

    Fine-tuning Mistral-7B on a new Arabic medical Q&A dataset yields BERTScore F1 of 68.50 percent, higher than LLaMA-2, GPT-2 Medium, AraGPT2, and BLOOM in this study.

  2. A Multi-Layered Large Language Model Framework for Disease Prediction

    cs.CL 2025-01 conditional novelty 4.0 of 10

    LLAMA3-based NER preprocessing plus LoRA fine-tuning of CAMeL-BERT gives 83% disease type and 69% severity accuracy on Arabic telehealth posts, though without significance testing.

  3. Retrieval Augmented Generation Based LLM Evaluation For Protocol State Machine Inference With Chain-of-Thought Reasoning

    cs.NI 2025-01 conditional novelty 4.0 of 10

    A RAG-plus-chain-of-thought agent improves BLEU, ROUGE, and WER scores for RTSP packet seed generation over Gemma-2-9B and Llama-3-8B baselines, though the evaluation does not test actual fuzzing effectiveness.

  4. Scaling Arabic Medical Chatbots Using Synthetic Data: Enhancing Generative AI with Synthetic Patient Records

    cs.CL 2025-09 conditional novelty 3.0 of 10

    Synthetic patient-doctor dialogues generated by ChatGPT-4o and Gemini and mixed with 20,000 real Arabic records improved fine-tuned LLM BERTScore F1 scores, with ChatGPT-4o data giving larger gains than Gemini data.

Reference graph

Works this paper leans on

40 extracted references · 35 canonical work pages · cited by 4 Pith papers

  1. [1]

    In: Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering, pp

    Alshahwan, N., Chheda, J., Finogenova, A., Gokkaya, B., Harman, M., Harper, I., Marginean, A., Sengupta, S., Wang, E.: Automated unit test improvement using large language models at meta. In: Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering, pp. 185–196 (2024)

    work page 2024

  2. [2]

    ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)18(3), 1–52 (2019)

    Badaro, G., Baly, R., Hajj, H., El-Hajj, W., Shaban, K.B., Habash, N., Al-Sallab, A., Hamdi, A.: A survey of opinion mining in arabic: A comprehensive system perspective covering challenges and advances in tools, resources, models, applica- tions, and visualizations. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)18(3),...

    work page 2019

  3. [3]

    In: Proceedings of the 11th international workshop on semantic evaluation (SEMEVAL-2017), pp

    Baly, R., Badaro, G., Hamdi, A., Moukalled, R., Aoun, R., El-Khoury, G., Al Sal- lab, A., Hajj, H., Habash, N., Shaban, K., et al.: Omam at semeval-2017 task 4: Evaluation of english state-of-the-art sentiment analysis models for arabic and a new topic-based model. In: Proceedings of the 11th international workshop on semantic evaluation (SEMEVAL-2017), p...

    work page 2017

  4. [4]

    test! Morgan Kaufmann (2020)

    Barnum, C.M.: Usability testing essentials: Ready, set... test! Morgan Kaufmann (2020)

    work page 2020

  5. [5]

    In: Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval), pp

    Bhatt, S., Jain, R., Dandapat, S., Sitaram, S.: A case study of efficacy and chal- lenges in practical human-in-loop evaluation of nlp systems using checklist. In: Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval), pp. 120–130 (2021)

    work page 2021

  6. [6]

    MIT press (2003)

    Carroll, J.M.: Making use: scenario-based design of human-computer interactions. MIT press (2003)

    work page 2003

  7. [7]

    In: International Conference on Computational Intelligence in Data Science, pp

    Chinnaswamy, A., Sabarish, B., Deepak Menan, R.: User story based automated test case generation using nlp. In: International Conference on Computational Intelligence in Data Science, pp. 156–166. Springer (2024)

    work page 2024

  8. [8]

    In: International Conference on Knowledge Science, Engineering and Management, pp

    Chuor, P., Ittoo, A., Heng, S.: User story classification with machine learning and llms. In: International Conference on Knowledge Science, Engineering and Management, pp. 161–175. Springer (2024)

    work page 2024

Show all 40 references
  1. [9]

    arXiv preprint arXiv:2305.14314 (2023)

    Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: Qlora: Efficient fine- tuning of quantized llms. arXiv preprint arXiv:2305.14314 (2023). URLhttps: //arxiv.org/abs/2305.14314

    2023 arXiv

  2. [10]

    Applied Linguistics Review14(5), 1451–1473 (2023)

    Di Zhang, E., Yu, S.: Investigating the relationship between linguistic changes in l2 writers’ paraphrasing, paraphrasing performance and l2 proficiency. Applied Linguistics Review14(5), 1451–1473 (2023)

    2023

  3. [11]

    Journal of Artificial Intelligence Research61, 623–698 (2018)

    Fioretto, F., Pontelli, E., Yeoh, W.: Distributed constraint optimization problems and applications: A survey. Journal of Artificial Intelligence Research61, 623–698 (2018)

    2018

  4. [12]

    Journal of Systems and Software 165, 110,570 (2020)

    Garousi, V., Rainer, A., Lauvås Jr, P., Arcuri, A.: Software-testing education: A systematic literature mapping. Journal of Systems and Software 165, 110,570 (2020)

    2020

  5. [13]

    arXiv preprint arXiv:2408.03095 (2024)

    Gu, S., Fang, C., Zhang, Q., Tian, F., Chen, Z.: Testart: Improving llm-based unit test via co-evolution of automated generation and repair iteration. arXiv preprint arXiv:2408.03095 (2024)

    2024 arXiv

  6. [14]

    Big Data and Cognitive Computing6(3), 88 (2022)

    Hamad, O., Hamdi, A., Hamdi, S., Shaban, K.: Steducov: an explored and bench- marked dataset on stance detection in tweets towards online education during covid-19 pandemic. Big Data and Cognitive Computing6(3), 88 (2022)

    2022

  7. [15]

    In: International Conference on Text, Speech, and Dialogue, pp

    Hamad, O., Hamdi, A., Shaban, K.: Attention-based model for accurate stance detection. In: International Conference on Text, Speech, and Dialogue, pp. 212–

  8. [16]

    arabic chatbots: A survey and future directions

    Hamad, O., Hamdi, A., Shaban, K.: Empathy and persona of english vs. arabic chatbots: A survey and future directions. In: International Conference on Text, Speech, and Dialogue, pp. 525–537. Springer (2022)

    2022

  9. [17]

    In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp

    Hamad, O., Shaban, K., Hamdi, A.: Asem: Enhancing empathy in chatbot through attention-based sentiment and emotion modeling. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 1588–...

    2024

  10. [18]

    Journal of Computer Science (2016)

    Hamdi, A., Shaban, K., Zainal, A.: A review on challenging issues in arabic senti- ment analysis. Journal of Computer Science (2016)

    2016

  11. [19]

    ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)17(4), 1–28 (2018)

    Hamdi, A., Shaban, K., Zainal, A.: Clasenti: a class-specific sentiment analysis framework. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)17(4), 1–28 (2018)

    2018

  12. [20]

    Springer Nature (2022)

    Hertzum, M.: Usability testing: A practitioner’s guide to evaluating the user expe- rience. Springer Nature (2022)

    2022

  13. [21]

    Hurani, M., Idris, H.: Investigating the use of llms for automated test generation: challenges, benefits, and suitability (2024)

    2024

  14. [22]

    In: 2012 19th Working Conference on Reverse Engineering, pp

    Ichii, M., Myojin, T., Nakagawa, Y., Chikahisa, M., Ogawa, H.: A rule-based au- tomated approach for extracting models from source code. In: 2012 19th Working Conference on Reverse Engineering, pp. 308–317. IEEE (2012)

    2012

  15. [23]

    In: International Conference on Evaluation of Novel Software Approaches to Software Engineering, vol

    Kamalakar, S., Edwards, S.H., Dao, T.M.: Automatically generating tests from natural language descriptions of software behavior. In: International Conference on Evaluation of Novel Software Approaches to Software Engineering, vol. 2, pp. 238–245. SCITEPRESS (2013)

    2013

  16. [24]

    In: 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp

    Karmarkar, H., Agrawal, S., Chauhan, A., Shete, P.: Navigating confidentiality in test automation: A case study in llm driven test data generation. In: 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 337–348. IEEE (2024)

    2024

  17. [25]

    In: 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), pp

    Klammer, C., Ramler, R.: A journey from manual testing to automated test gener- ation in an industry project. In: 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), pp. 591–592. IEEE (2017)

    2017

  18. [26]

    O’Reilly Media, Inc

    Kleppmann, M.: Designing data-intensive applications: The big ideas behind reli- able, scalable, and maintainable systems. " O’Reilly Media, Inc." (2017)

    2017

  19. [27]

    Natural Language Engineering9(4), 381–420 (2003)

    McRoy, S.W., Channarukul, S., Ali, S.S.: An augmented template-based approach to text realization. Natural Language Engineering9(4), 381–420 (2003)

    2003

  20. [28]

    Ieee Access 12, 25,553–25,579 (2024)

    Mohamed, Y.A., Khanan, A., Bashir, M., Mohamed, A.H.H., Adiel, M.A., Elsadig, M.A.: The impact of artificial intelligence on language translation: a review. Ieee Access 12, 25,553–25,579 (2024)

    2024

  21. [29]

    Applied Soft Computing132, 109,803 (2023)

    Pellicer, L.F.A.O., Ferreira, T.M., Costa, A.H.R.: Data augmentation techniques in natural language processing. Applied Soft Computing132, 109,803 (2023)

    2023

  22. [30]

    IEEE Access (2024)

    Raiaan, M.A.K., Mukta, M.S.H., Fatema, K., Fahad, N.M., Sakib, S., Mim, M.M.J., Ahmad, J., Ali, M.E., Azam, S.: A review on large language models: Architectures, applications, taxonomies, open issues and challenges. IEEE Access (2024)

    2024

  23. [31]

    Proceedings of the ACM on Software Engineering1(FSE), 951– 971 (2024)

    Ryan, G., Jain, S., Shang, M., Wang, S., Ma, X., Ramanathan, M.K., Ray, B.: Code-aware prompting: A study of coverage-guided test generation in regression setting using llm. Proceedings of the ACM on Software Engineering1(FSE), 951– 971 (2024)

    2024

  24. [32]

    IEEE Access (2024)

    Saha, D., Tarek, S., Yahyaei, K., Saha, S.K., Zhou, J., Tehranipoor, M., Farah- mandi, F.: Llm for soc security: A paradigm shift. IEEE Access (2024)

    2024

  25. [33]

    IEEE Transactions on Soft- ware Engineering (2023)

    Schäfer, M., Nadi, S., Eghbali, A., Tip, F.: An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Soft- ware Engineering (2023)

    2023

  26. [34]

    Software quality journal30(2), 455–481 (2022)

    Stray, V., Florea, R., Paruch, L.: Exploring human factors of the agile software tester. Software quality journal30(2), 455–481 (2022)

    2022

  27. [35]

    In: International conference on intelligent systems design and applications, pp

    Tyagi, A.K., Fernandez, T.F., Mishra, S., Kumari, S.: Intelligent automation sys- tems at the core of industry 4.0. In: International conference on intelligent systems design and applications, pp. 1–18. Springer (2020)

    2020

  28. [36]

    In: Advances in Neural Information Processing Systems (NeurIPS), vol

    Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 30 (2017). URL https://arxiv.org/abs/ 1706.03762

    2017 arXiv

  29. [37]

    In: International conference on artificial intelligence in education, pp

    Wang, Z., Valdez, J., Basu Mallick, D., Baraniuk, R.G.: Towards human-like educa- tional question generation with large language models. In: International conference on artificial intelligence in education, pp. 153–166. Springer (2022)

    2022

  30. [38]

    Advances in Neural Information Processing Systems 36 (2024)

    Xue, F., Fu, Y., Zhou, W., Zheng, Z., You, Y.: To repeat or not to repeat: Insights from scaling llm under token-crisis. Advances in Neural Information Processing Systems 36 (2024)

    2024

  31. [39]

    In: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, pp

    Xue, Z., Li, L., Tian, S., Chen, X., Li, P., Chen, L., Jiang, T., Zhang, M.: Domain knowledge is all you need: A field deployment of llm-powered test case generation in fintech domain. In: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: ...

    2024

  32. [40]

    Advances in Neural Information Processing Systems36, 58,478– 58,507 (2023)

    Yuan, L., Chen, Y., Cui, G., Gao, H., Zou, F., Cheng, X., Ji, H., Liu, Z., Sun, M.: Revisiting out-of-distribution robustness in nlp: Benchmarks, analysis, and llms evaluations. Advances in Neural Information Processing Systems36, 58,478– 58,507 (2023)

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.