Pith. sign in

REVIEW 3 major objections 6 minor 38 references

Can LLMs Write CI? A Study on Automatic Generation of GitHub Actions Configurations

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that large language models can mimic CI configuration syntax but rarely produce correct GitHub Actions workflows, with the best model matching ground truth only 3% of the time.

desk verdict Useful new dataset and honest preliminary evaluation, but the headline code-vs-general comparison is confounded and the similarity metrics are unvalidated. read the letter →

arxiv 2507.17165 v1 pith:6ISWDFF6 submitted 2025-07-23 cs.SE

Taher A. Ghaleb , Dulina Rathnayake This is my paper · ORCID
classification cs.SE
keywords continuousintegrationGitHubActionsYAMLconfigurationgenerationlargelanguagemodelszero-shotpromptingcodesimilaritymetricsLLMevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can turn plain-English descriptions of CI tasks into working GitHub Actions YAML configurations. It builds a labeled dataset of 1,318 description-and-configuration pairs from official GitHub Actions documentation and tests six models with zero-shot prompting, meaning the models receive no example configurations. The best model, GPT-4o, reaches 69% lexical similarity with the ground truth, yet only 3% of its outputs are exact matches. Code-pretrained models finish at the bottom, underperforming general-purpose models, which the paper reads as evidence that code-centric pretraining is not enough for YAML generation. A manual analysis of 90 outputs shows the failures are mostly semantic, such as missing steps, hallucinated defaults, and misread descriptions, not YAML syntax errors.

What carries the argument

The load-bearing mechanism is the evaluation pipeline the paper constructs: a dataset of 1,318 Description–YAML pairs extracted from GitHub Actions documentation, a fixed zero-shot prompt that asks each model to output only YAML with no comments or extra formatting, and five similarity metrics (Cosine Similarity, Euclidean Distance, Tree Edit Distance, ROUGE-L, and chrF) that combine lexical overlap with structural distance. The manual card-sorting analysis of 90 GPT-4o outputs supplies the qualitative evidence that similarity scores cannot: it attributes failures to intent misreading, omitted essential keys, and hallucinated additions. The dataset and prompt standardization make the six models comparable, while the metric battery turns similarity into numbers that support the ranking.

What would settle it

Run the 1,318 generated configurations through a GitHub Actions schema validator or workflow linter and compare validation failure rates across models; if the validator ranks models differently from the five similarity metrics, then the reported similarity does not measure what the paper claims. A more direct version is to execute the ground-truth and best-generated workflows in a sandbox and count how many generated workflows actually run as described.

Watch

Extended reading notes

Core claim

The central claim is that LLM-generated CI configurations are structurally plausible but rarely correct: zero-shot GPT-4o achieves up to 0.69 cosine similarity and 0.66 tree-edit similarity with the reference configurations, yet only 3% of outputs match exactly. The paper further claims this is not a syntax problem but an intent problem: models omit directive fields like caching, rename or reorder steps, and invent defaults such as example_input or extra checkout steps. It also claims that code-pretraining does not help, because Code Llama and CodeGemma score significantly below their general-purpose counterparts, so YAML generation appears to depend more on broad instruction-following than on code-specific training. The paper treats these results as revealing a gap between generation quality and the precision that executable CI configuration requires.

Load-bearing premise

The study treats the five similarity scores as stand-ins for whether a generated configuration is correct and usable, and it never executes or schema-validates any output; if those scores are poor proxies for executability, the central ranking and the claim about LLM limitations do not follow.

Editorial extensions

If this is right

  • If the 3% perfect-match result generalizes, developers should treat LLM-written CI configurations as drafts requiring human review and validation rather than as drop-in workflows.
  • The underperformance of code-pretrained models implies that scaling code-focused training alone will not fix configuration generation; instruction following and CI-specific knowledge matter more.
  • The observed missing and hallucinated directives suggest that CI automation tools should add schema validation and workflow linting on top of raw LLM output.
  • Because longer descriptions correlate weakly but significantly with better output, improving documentation clarity and prompt specificity could directly improve generated configurations.
  • The new dataset provides a shared benchmark for future research on natural-language-to-CI generation, allowing direct comparison with these results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: executing or schema-validating the generated YAML would likely rank models differently than the five similarity metrics do, since a single indentation error can make a workflow unusable while registering as a small structural distance.
  • Editorial inference: the failure patterns point to a testable fix, namely fine-tuning on CI-specific constraints or retrieving relevant documentation at generation time, which should reduce hallucinated defaults and omitted mandatory keys.
  • Editorial inference: because only GPT-4o received manual analysis, the other five models' error patterns are unknown; applying the same card-sorting protocol to them could reveal whether the semantic failures are model-specific or shared across architectures.
  • Editorial inference: a stricter version of this benchmark would validate the ground-truth configurations by running them or checking them against GitHub Actions' schema, which the paper did not do.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper evaluates six LLMs on generating GitHub Actions YAML configurations from 1,318 natural-language descriptions curated from official GitHub Actions documentation. Using a zero-shot prompt and five similarity metrics (Cosine, Euclidean, Tree Edit, ROUGE-L, chrF), the authors report that GPT-4o achieves the highest average similarity (up to 0.69 Cosine, 3% perfect matches), and that code-specialized models (Code Llama, CodeGemma) rank lowest. A manual analysis of 90 GPT-4o outputs identifies missing steps, hallucinated additions, and renaming as recurring inaccuracies. The authors position the dataset and the evaluation as a first step toward CI-configuration-specific LLM alignment.

Significance. If the findings hold, the dataset alone is a useful community resource, and the replication package (data, scripts, raw results) supports reproducibility, which is a concrete strength. The observation that even the best model matches the reference in only 3% of cases is a potentially valuable baseline. However, the paper's central comparative claims currently outrun the evidence: the similarity metrics are not validated proxies for correctness, and the code-versus-general model comparison is confounded by model scale and vintage. The qualitative error analysis is careful but covers only one model. With reframing and additional validation, the study would be a solid preliminary contribution.

major comments (3)
  1. [IV-C / V] The five similarity metrics (Cosine, Euclidean, Tree Edit, ROUGE-L, chrF) are used as the quantitative basis for all RQ1 conclusions, but the paper validates none of them against the actual property of interest: whether a generated YAML is correct and executable. The Construct Validity paragraph in Section V concedes that these metrics "may not fully reflect practical executability or correctness," and the External Validity paragraph states that no generated configuration was executed or schema-validated. Therefore the headline numbers (e.g., 69% Cosine, 3% perfect matches) are only statements of lexical or structural resemblance to one particular reference string, not of configuration quality. The abstract's conclusion that these results "reveal LLM limitations for CI configuration generation" requires at least a schema validation of outputs (e.g., parsing the generated YAML against GitHub Actions syntax) or a manual executability check; without this, the central quantitative claim is not supported.
  2. [IV-A / IV-D1] The comparative claim that code-pretrained models underperform general-purpose ones is confounded by model scale, architecture vintage, and API access. GPT-4o and GPT-4.1 are closed models of unknown size, while Llama3.1-8B, Gemma3-12B, Code Llama-7B, and CodeGemma-7B are open models with 7B to 12B parameters; in particular, Code Llama-7B is built on the older Llama 2 architecture, and CodeGemma-7B is older and smaller than Gemma3-12B. A rank ordering in which all large/API models outscore all small open models cannot be attributed to pretraining objective. Section IV-D3 defers size-controlled comparisons to future work, yet the abstract and conclusion present the code-versus-general comparison as a finding. The paper should either add a matched-size comparison (e.g., comparing Code Llama with a similarly sized general-purpose Llama variant, or confining the comparison to open models) or explicitly reframe the result as "among the six models tested, the two code-specialized open models ranked lowest," without a causal interpretation. Additionally, GPT-4.1 is classified as code-pretrained solely because of "improved coding capabilities," which is not the same as code-specialized pretraining; this category boundary should be justified or renamed.
  3. [IV-D1] The significance testing is not fully specified: the text says a Wilcoxon signed-rank test was used to compare "the five models," but six models are evaluated, and the reported p-values (e.g., p < 0.001 for Code Llama vs. GPT-4o) do not state how many pairwise comparisons were performed, whether the test was applied per metric, and how Holm–Bonferroni correction was applied across that family of tests. Without this information, the reader cannot assess whether the reported differences survive multiple-comparison correction. Please report the exact pairwise comparison matrix, the number of tests, and the corrected thresholds.
minor comments (6)
  1. [VI] The conclusion states "We created the first labeled dataset of CI configuration errors," but the dataset described in Section III-A contains description–YAML pairs from documentation, not configuration errors; reword to match the dataset description.
  2. [IV-A] The citation for Llama3.1-8B is [22] (Touvron et al., "LLaMA: Open and efficient foundation language models"), which describes the original Llama, not Llama 3.1; update the reference to the appropriate Llama 3 herd paper.
  3. [IV-D1] The number of models in the Wilcoxon test ("the five models") appears inconsistent with the six models listed in Section IV-A; clarify which comparisons were performed.
  4. [Figure 2] The metric labels are run together (e.g., "CosineSimilarity," "TreeEditDistance"); add spaces, and consider a table of means and medians with confidence intervals, since 1,318 points per boxplot can obscure distributional detail.
  5. [IV-D2] For the manual analysis sample, state the random seed or the exact procedure used to draw the 90 configurations so that the sample can be reproduced.
  6. [III-A4] The dataset is said to contain 1,318 examples; please clarify how many are unique workflows versus partial snippets, and whether any examples overlap across documentation pages (which would affect the independence of the paired comparisons).

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the evaluation uses an external documentation-derived benchmark, zero-shot prompting with no fitted parameters, and standard similarity metrics; self-citations are background-only and not load-bearing.

full rationale

The paper's central derivation chain is an empirical evaluation: construct a Description–YAML dataset from the official GitHub Actions documentation (Section III-A), prompt six LLMs zero-shot with a fixed template (Listing 1, temperature 0), and score outputs against the ground truth with five standard metrics (Section IV-C). No parameter is fitted to the test set, no model is fine-tuned on the benchmark, and no result is defined in terms of another result: the similarity scores are computed between independently generated YAML and documentation-derived references. The only self-citations (refs [15]–[18] and the replication package [12]) are used as related work or artifact pointers, not as premises that force any conclusion. The acknowledged data-leakage threat (Section V) is a contamination concern, not circularity: even if the documentation appeared in training data, the paper does not exploit that to manufacture results. The construct-validity caveat that similarity 'may not fully reflect practical executability or correctness' and the model-size confound in the code-versus-general comparison are validity and experimental-design risks, which belong in a correctness assessment rather than a circularity finding. Under the hard rule that circularity requires a quoted reduction of a claim to its own inputs, no such reduction exists in this paper.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper makes no derivational claims, so there are no fitted free parameters or invented entities. The load-bearing assumptions are about the validity of the documentation-based ground truth, the similarity metrics, and the representativeness of the model comparison; these are listed above.

assumptions (4)
  • domain assumption Ground truth YAML configurations extracted from official GitHub Actions documentation are authoritative and correct.
    Section V External Validity states 'we did not execute or validate the correctness of the ground truth YAML configurations, as they were directly extracted from the official GitHub Actions documentation.' The entire evaluation treats these as correct references.
  • domain assumption The five similarity metrics are a valid proxy for the quality and correctness of generated CI configurations.
    Section V Construct Validity acknowledges the metrics 'may not fully reflect practical executability or correctness.' All RQ1 conclusions depend on this proxy.
  • domain assumption The six selected models are representative of general-purpose and code-pretrained LLM capabilities.
    Section IV-A selects models of different sizes and providers; the comparison of code-pretrained versus general-purpose is confounded by model scale and release date, an assumption the paper does not fully control.
  • domain assumption Zero-shot prompting without examples reflects realistic developer usage.
    Section III-B uses a single zero-shot template; the paper assumes this maps to typical usage, though developers may use few-shot or agentic workflows.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can LLMs Write CI? A Study on Automatic Generation of GitHub Actions Configurations." pith.science (2026). https://pith.science/paper/6ISWDFF6

@misc{pith2026250717165,
  author       = {Pith},
  title        = {Pith review of: Can LLMs Write CI? A Study on Automatic Generation of GitHub Actions Configurations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ISWDFF6}},
  note         = {Machine review of arXiv:2507.17165}
}
read the original abstract

Continuous Integration (CI) services, such as GitHub Actions, require developers to write YAML-based configurations, which can be tedious and error-prone. Despite the increasing use of Large Language Models (LLMs) to automate software engineering tasks, their ability to generate CI configurations remains underexplored. This paper presents a preliminary study evaluating six LLMs for generating GitHub Actions configurations from natural language descriptions. We assess three general-purpose foundation models (GPT-4o, Llama, and Gemma) and three code-pretrained models (GPT-4.1, Code Llama, and CodeGemma). We also introduce the first labeled dataset of its kind, constructed from GitHub Actions documentation, pairing descriptions with corresponding best-practice YAML configurations. Zero-shot prompting achieves up to 69% similarity with the ground truth, with only 3% perfect matches. Code-pretrained models slightly underperform compared to general-purpose ones in YAML-based CI tasks, revealing LLM limitations for CI configuration generation. Analyzing GPT-4o outputs reveals issues like missing or renamed steps, misinterpreted descriptions, and unnecessary additions that may affect structural and contextual correctness, indicating a gap between generation quality and the precision required for executable CI configurations. Our research offers insights for improving LLM alignment with configuration languages and guiding future efforts on CI automation and tooling support.

Figures

Figures reproduced from arXiv: 2507.17165 by the authors.

Figure 1
Figure 1. Overview of our study [Results] [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Boxplots showing the distribution of similarity scores across six LLMs using five evaluation metrics in generating YAML configurations [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 20 canonical work pages

  1. [1]

    Continuous Integration,

    M. Fowler, “Continuous Integration,” https://martinfowler.com/articles/ originalContinuousIntegration.html, accessed: 2021-09-23

    work page 2021

  2. [2]

    Workflow syntax for GitHub Actions,

    “Workflow syntax for GitHub Actions,” https://docs.github.com/en/ actions/writing-workflows, Accessed May 2025

    2025

  3. [3]

    GitHub Actions best practices,

    “GitHub Actions best practices,” https://docs.github.com/en/actions/ use-cases-and-examples, Accessed May 2025

    work page 2025

  4. [4]

    Automated code generation for information technology tasks in Y AML through large language models,

    S. Pujar, L. Buratti, X. Guo, N. Dupuis, B. Lewis, S. Suneja, A. Sood, G. Nalawade, M. Jones, A. Morari, and R. Puri, “Automated code generation for information technology tasks in Y AML through large language models,” arXiv preprint arXiv:2305.02783 , 2023

    arXiv 2023

  5. [5]

    Usage, costs, and benefits of continuous integration in open-source projects,

    M. Hilton, T. Tunnell, K. Huang, D. Marinov, and D. Dig, “Usage, costs, and benefits of continuous integration in open-source projects,” in Proceedings of the 31st IEEE/ACM international conference on automated software engineering , 2016, pp. 426–437

    work page 2016

  6. [6]

    A conceptual replication of continuous integration pain points in the context of Travis CI,

    D. G. Widder, M. Hilton, C. K ¨astner, and B. Vasilescu, “A conceptual replication of continuous integration pain points in the context of Travis CI,” in Proceedings of the 2019 27th acm joint meeting on european software engineering conference and symposium on the foundations of software engineering, 2019, pp. 647–658

    work page 2019

  7. [7]

    An empirical characterization of bad practices in contin- uous integration,

    F. Zampetti, C. Vassallo, S. Panichella, G. Canfora, H. Gall, and M. Di Penta, “An empirical characterization of bad practices in contin- uous integration,” Empirical Software Engineering , vol. 25, pp. 1095– 1135, 2020

    2020

  8. [8]

    Configuration smells in continuous delivery pipelines: a linter and a six- month study on GitLab,

    C. Vassallo, S. Proksch, A. Jancso, H. C. Gall, and M. Di Penta, “Configuration smells in continuous delivery pipelines: a linter and a six- month study on GitLab,” in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 2020, pp. 327–337

    work page 2020

Show all 38 references
  1. [9]

    Toward automatically completing GitHub workflows,

    A. Mastropaolo, F. Zampetti, G. Bavota, and M. Di Penta, “Toward automatically completing GitHub workflows,” inProceedings of the 46th IEEE/ACM International Conference on Software Engineering, 2024, pp. 1–12

    2024

  2. [10]

    On the effectiveness of large language models for GitHub workflows,

    X. Zhang, S. Muralee, S. Cherupattamoolayil, and A. Machiry, “On the effectiveness of large language models for GitHub workflows,” in Proceedings of the 19th International Conference on Availability, Reliability and Security , 2024, pp. 1–14

    2024

  3. [11]

    Large lan- guage models are zero-shot reasoners,

    T. Kojima, S. S. Gu, M. Reid, Y . Matsuo, and Y . Iwasawa, “Large lan- guage models are zero-shot reasoners,” Advances in neural information processing systems, vol. 35, pp. 22 199–22 213, 2022

    2022

  4. [12]

    Can LLMs write CI? a study on automatic generation of github actions configurations (replication package),

    T. A. Ghaleb and D. Rathnayake, “Can LLMs write CI? a study on automatic generation of github actions configurations (replication package),” https://github.com/Taher-Ghaleb/ICSME25-LLM4CI, 2025

    2025

  5. [13]

    Automatically generating Dockerfiles via deep learning: Challenges and promises,

    G. Rosa, A. Mastropaolo, S. Scalabrino, G. Bavota, and R. Oliveto, “Automatically generating Dockerfiles via deep learning: Challenges and promises,” arXiv preprint arXiv:2303.15990 , 2023

    2023 arXiv

  6. [14]

    Automated DevOps pipeline generation for code repositories using large language models,

    D. Mehta, K. Rawool, S. Gujar, and B. Xu, “Automated DevOps pipeline generation for code repositories using large language models,” arXiv preprint arXiv:2312.13225, 2023

    2023 arXiv

  7. [15]

    CI/CD configuration practices in open-source Android apps: An empirical study,

    T. Ghaleb, O. Abduljalil, and S. Hassan, “CI/CD configuration practices in open-source Android apps: An empirical study,” ACM Transactions on Software Engineering and Methodology , 2025

    2025

  8. [16]

    An empirical study of the long duration of continuous integration builds,

    T. A. Ghaleb, D. A. Da Costa, and Y . Zou, “An empirical study of the long duration of continuous integration builds,” Empirical Software Engineering, vol. 24, no. 4, pp. 2102–2139, 2019

    2019

  9. [17]

    Studying the interplay between the durations and breakages of continuous integration builds,

    T. A. Ghaleb, S. Hassan, and Y . Zou, “Studying the interplay between the durations and breakages of continuous integration builds,” IEEE Transactions on Software Engineering , vol. 49, no. 4, pp. 2476–2497, 2022

    2022

  10. [18]

    Studying the impact of noises in build breakage data,

    T. A. Ghaleb, D. A. Da Costa, Y . Zou, and A. E. Hassan, “Studying the impact of noises in build breakage data,” IEEE Transactions on Software Engineering, vol. 47, no. 9, pp. 1998–2011, 2019

    1998

  11. [19]

    On the usage, co-usage and migration of CI/CD tools: A qualitative analysis,

    P. Rostami Mazrae, T. Mens, M. Golzadeh, and A. Decan, “On the usage, co-usage and migration of CI/CD tools: A qualitative analysis,” Empirical Software Engineering , vol. 28, no. 2, p. 52, 2023

    2023

  12. [21]

    Introducing GPT-4.1 in the API,

    OpenAI, “Introducing GPT-4.1 in the API,” 2025, accessed: 2025-06-04. [Online]. Available: https://openai.com/index/gpt-4-1/

    2025

  13. [22]

    LLaMA: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023. [Online]. A...

    2023 arXiv

  14. [23]

    Code llama: Open foundation models for code,

    B. Rozi `ere, J. Gehring, S. Bekman, M. Ott, T. Scialom, S. Edunov, and G. Lample, “Code llama: Open foundation models for code,” arXiv preprint arXiv:2308.12950 , 2023. [Online]. Available: https: //arxiv.org/abs/2308.12950

    2023 arXiv

  15. [24]

    Gemma 3 technical report,

    R. Anil et al. , “Gemma 3 technical report,” arXiv preprint arXiv:2503.19786, 2024. [Online]. Available: https://arxiv.org/abs/2503. 19786

    2024 arXiv

  16. [25]

    Codegemma: Open code models based on gemma,

    H. Zhao, J. Hui, J. Howland, N. Nguyen, S. Zuo, A. Hu, C. A. Choquette-Choo, J. Shen, J. Kelley, K. Bansal, L. Vilnis, M. Wirth, P. Michel, P. Choy, P. Joshi, R. Kumar, S. Hashmi, S. Agrawal, Z. Gong, J. Fine, T. Warkentin, A. J. Hartman, B. Ni, K. Korevec, K. Schaefer, and S....

    2024 arXiv

  17. [26]

    Salton and M

    G. Salton and M. J. McGill, Introduction to modern information re- trieval. McGraw-Hill, 1983

    1983

  18. [27]

    Simple fast algorithms for the editing distance between trees and related problems,

    K. Zhang and D. Shasha, “Simple fast algorithms for the editing distance between trees and related problems,” SIAM Journal on Computing , vol. 18, no. 6, pp. 1245–1262, 1989

    1989

  19. [28]

    C. D. Manning, P. Raghavan, and H. Sch ¨utze, Introduction to Informa- tion Retrieval. Cambridge University Press, 2008

    2008

  20. [29]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81

    2004

  21. [30]

    chrF: character n-gram F-score for automatic MT eval- uation,

    M. Popovi ´c, “chrF: character n-gram F-score for automatic MT eval- uation,” in Proceedings of the tenth workshop on statistical machine translation, 2015, pp. 392–395

    2015

  22. [31]

    Language models are few-shot learners,

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B...

    2020

  23. [32]

    Individual comparisons by ranking methods,

    F. Wilcoxon, “Individual comparisons by ranking methods,” Biometrics Bulletin, vol. 1, no. 6, pp. 80–83, 1945

    1945

  24. [33]

    The proof and measurement of association between two things,

    C. Spearman, “The proof and measurement of association between two things,” The American Journal of Psychology, vol. 15, no. 1, pp. 72–101, 1904

    1904

  25. [34]

    Cohen, Statistical Power Analysis for the Behavioral Sciences

    J. Cohen, Statistical Power Analysis for the Behavioral Sciences. Rout- ledge, 1988

    1988

  26. [35]

    A simple sequentially rejective multiple test procedure,

    S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics , vol. 6, no. 2, pp. 65–70, 1979

    1979

  27. [36]

    Spencer, Card sorting: Designing usable categories

    D. Spencer, Card sorting: Designing usable categories . Rosenfeld Media, 2009

    2009

  28. [37]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  29. [38]

    Retrieval-augmented generation for knowledge-intensive NLP tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, I. Ku- likov, A. Fan, V . Chaudhary, F. Guzman et al. , “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Advances in Neural Information Processing Systems , vol. 33, 2020, pp. 9459–9474

    2020

  30. [2023]

    Available: https://arxiv.org/abs/2303.08774

    [Online]. Available: https://arxiv.org/abs/2303.08774

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.