Pith. sign in

REVIEW 3 major objections 5 minor 61 references

Bridging Behavior and Implementation: Automated Java Glue Code Generation for Behavior-Driven Development

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A behavior-first multi-agent pipeline can turn underspecified BDD steps into Java glue code, with 46.1% directly usable.

desk verdict A genuinely new task and a clean agent design—but the 46.1% 'directly usable' headline is an LLM judge's opinion, not a compilation or execution check, so read it with that grain of salt. read the letter →

arxiv 2607.19703 v1 pith:RX4RMEYQ submitted 2026-07-22 cs.SE

Xinyu Shi , Zhou Yang , An Ran Chen This is my paper
classification cs.SE
keywords behavior-drivendevelopmentgluecodegenerationLLMagentsmulti-agentsystemsJavaBDDLLM-as-a-judgespecification-to-code
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the most labor-intensive part of Behavior-Driven Development—the glue code that wires natural-language steps to project APIs—can be automated with a hierarchical multi-agent LLM system. AutoGlue first interprets a step's intent from its scenario context, then retrieves similar BDD artifacts and relevant project code, and only then generates the Java glue code. On 1,307 real-world steps from eight open-source Java projects, AutoGlue outperforms few-shot prompting by 58.7% in API F1 and 43.7% in CodeBLEU, and its output is judged directly usable in 46.1% of cases. If correct, this reduces a major BDD maintenance bottleneck and suggests a broader specification-to-code paradigm.

What carries the argument

The central object is AutoGlue, a hierarchical multi-agent pipeline with three roles: a Behavior Interpreter that converts an underspecified step plus its scenario/feature context into an explicit behavior summary; two retrieval sub-agents that independently gather BDD references (similar steps and existing glue code) and project code context; and a Developer agent that fuses these into final Java glue code. The design embodies a 'behavior-first' workflow, separating what the step means from how to implement it, which the paper credits for avoiding mixed reasoning and context noise.

What would settle it

Take a random sample of the 603 Exact Match outputs, compile each in its original project, and run the corresponding BDD scenario; if substantially fewer than the claimed proportion compile and pass, the usability judgment is overly optimistic. A cheaper check: have a human with project access review a random sample of the LLM-judged Exact Match cases and look specifically for missing or wrong API calls the judge missed.

Watch

Extended reading notes

Core claim

AutoGlue establishes that LLMs can bridge natural-language behavior specifications and project code. The framework decomposes glue-code generation into three sequential stages: a Behavior Interpreter derives the intended behavior of a step from its feature and scenario context; a Developer agent coordinates a BDD Context Retriever that finds semantically similar steps and existing glue code, and a Project Context Retriever that explores the codebase for relevant classes and methods; and finally the Developer agent generates the annotation, signature, and body of the glue code. The central empirical claim is that this hierarchy produces directly usable glue code for 603 of 1,307 steps (46.1%)

Load-bearing premise

The evaluation assumes that a reference-aware LLM judge, without compiling or executing the generated code, can reliably tell when glue code is directly usable; if the judge overestimates semantic equivalence to the human reference, the 46.1% Exact Match figure is inflated.

Editorial extensions

If this is right

  • If the results hold, a developer can run AutoGlue on a large BDD suite and get nearly half of the glue code written without modification, with the rest often needing only small edits.
  • Behavior interpretation before code generation becomes a design principle: making intent explicit helps LLMs on other specification-to-code tasks, such as user stories, acceptance criteria, and ticket-to-code.
  • The error profile (missing actions and parameter mismatches dominate) points to a natural next step: feedback-guided repair that compiles, runs, or statically checks the output and iterates.
  • Since the framework is not Java-specific in its design, it could transfer to other BDD ecosystems (JavaScript, Ruby, Python, PHP) with framework-specific retrieval.
  • BDD-context retrieval and project-context retrieval are each load-bearing: removing the former drops similarity substantially, removing the latter lowers the directly-usable rate by over 21%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the LLM judge is biased toward the reference, the true directly-usable rate could be lower; the only way to know is to compile and run a sample of the 603 Exact Match cases in their projects.
  • The 'conservative generation strategy' finding suggests AutoGlue under-specifies rather than hallucinating, meaning a post-generation step that adds missing actions from scenario-level plans could recover many partial cases without risking over-implementation.
  • The behavior-first separation could generalize to a 'requirement-aware code generation' paradigm, where an explicit intent layer sits between natural language and code—a testable extension would be applying the same three-stage pipeline to issue-ticket-driven code changes.
  • Because all components share one LLM, the framework's ceilings are tied to that model's retrieval and code abilities; swapping in stronger or weaker models would be a natural stress test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces AutoGlue, a hierarchical multi-agent framework for generating Java BDD glue code from natural-language Gherkin steps. The pipeline consists of a Behavior Interpreter, a Developer agent, and two retrieval sub-agents (BDD context and project context). The authors construct a dataset of 1,307 step–glue-code pairs from eight open-source Java projects and compare AutoGlue against few-shot and plain LLM prompting. They report API F1 of 0.660 vs. 0.416 for few-shot prompting, CodeBLEU of 0.641 vs. 0.446, and an LLM-judge-assessed 'directly usable' rate of 46.1% (603/1,307), with 35.0% partial and 18.9% mismatch. Ablations show that behavior interpretation, BDD-context retrieval, and project-context retrieval each contribute to performance. The paper's central claim is that AutoGlue can effectively bridge underspecified behavior specifications and project code, substantially reducing manual glue-code effort.

Significance. If the results hold, AutoGlue would be the first end-to-end automated approach for BDD glue code generation, addressing a real and overlooked bottleneck. The strongest assets are the realistic 1,307-case dataset, the clear behavior-first architecture, the ablations that isolate each design component, and the explicit leakage-control measures. The reference-aware LLM judge is validated against 120 human-annotated cases, which is a useful starting point. However, the headline usability claim—'46.1% directly usable'—is a judge-assessed semantic label rather than a compilation or execution result, and the human validation does not calibrate the absolute category boundaries across all 1,307 cases. This validity gap is load-bearing for the paper's practical-value conclusion and needs to be addressed before the contribution can be fully accepted.

major comments (3)
  1. [Section 7.2, Table 4] The headline claim that 46.1% of AutoGlue outputs are 'directly usable' is an LLM-judge semantic label, not a compilation or execution result. The judge never compiles or runs the generated Java code. The 120-case human validation (ρ=0.84, binary κ=0.71) shows rank-order and binary Exact/non-Exact agreement, but does not calibrate the absolute Exact/Partial/Mismatch thresholds for the remaining 1,187 cases. The validation counts (human 66/26/28 vs. judge 61/39/20) show systematic boundary drift: the judge labels more cases as Partial and fewer as Mismatch. The Section 9 stability checks (repeated runs, alternate models) only show self-consistency, not ground-truth correctness. Please provide a three-category confusion matrix on the human sample, or a larger stratified human calibration of the Exact decision, or execution-based checks on a sample, and report adjusted rates.
  2. [Section 7.2, Figure 4; Section 6.2 judge prompt] The argument that most partially correct outputs 'require only minor fixes' rests on subcategory labels (Missing Action, Parameter Mismatch, etc.) that are assigned by the same LLM judge. No human agreement is reported for these subcategory labels, and the 120-case validation does not include them. The boundary between Partial and Mismatch is especially important because the paper treats Partial as 'minor fixes.' Please validate the subcategory labels against human annotators, or temper the 'minor revisions only' conclusion.
  3. [Section 7.1, Tables 2–3; Section 7.3, Table 5] The main comparisons (API F1 +58.7%, CodeBLEU +43.7%) are aggregate means without confidence intervals or significance tests. Per-project variation is large (e.g., Table 3: CxFlow CodeBLEU 0.530 vs. 0.427 for plain prompting; metasfresh 0.585 vs. 0.348). I request paired significance tests and effect sizes for both the main comparisons and the ablations. Also, the retrieval top-k and similarity thresholds in Section 4.2 are not reported; these are free parameters that should be specified for reproducibility.
minor comments (5)
  1. [Section 6.2] The mapping from continuous score to categories uses hand-picked thresholds ([4.0,5.0], [2.0,4.0), [1.0,2.0)). Please justify these thresholds or report sensitivity to them.
  2. [Section 6.3] The 'GPT-5 mini' model is referenced via a web page URL [37]; provide a stable citation or model version for reproducibility.
  3. [Section 5] The alignment of steps to glue code via pattern matching is described in one sentence; details on handling parameterized patterns and capturing groups would improve reproducibility.
  4. [Section 7.1] The statement that 'no pattern-matching errors are observed' needs a defined criterion; please clarify how binding correctness was verified.
  5. [Section 9] The external-validity discussion dismisses Java-specificity quickly; the strict type system and annotation-regex binding may not transfer to dynamically typed BDD frameworks such as Ruby or JavaScript.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: central results are measured against independently collected human-written ground truth; only minor non-load-bearing self-citations and a shared-model judge overlap are present.

full rationale

AutoGlue's central comparisons are not defined in terms of quantities the paper fits. API F1, CodeBLEU, METEOR, and ROUGE-L are computed by deterministic AST/token/structural matching against human-written glue code collected from eight open-source Java projects (Sections 5 and 7.1, Tables 2-3). The headline usability figure (603/1,307 = 46.1% Exact Match, Section 7.2/Table 4) is produced by a reference-aware LLM judge, and that judge is anchored by a 120-case human validation (Spearman rho=0.84, Cohen's kappa=0.71 for binary Exact vs Non-Exact; Section 7.2) plus stability checks across judge models and runs (Section 9). Using the same LLM family for generation and judging is a validity threat—the judge may share generation biases—but it is not a definitional reduction: the judge's labels are not a function of any fitted parameter, and the category thresholds were fixed before judging. The self-citations in the references ([44] for BDD maintenance cost, [45]/[50] for context-aware retrieval and test generation context) are contextual and do not carry the central derivation. No equation, fitted parameter, or uniqueness theorem is imported to force the results. Hence the paper's derivation chain is self-contained; at most there is minor self-reference that does not rise to circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

AutoGlue introduces no new physical entities and no fitted statistical model. The central claims depend on hand-set retrieval and judge thresholds, on the assumption that human-written glue code is the right reference, and on the assumption that an LLM judge can approximate developer judgment without execution.

free parameters (2)
  • Retrieval similarity threshold / top-k for BDD and Project context retrievers = not reported
    Embedding-based retrieval with cosine similarity is described in Section 4.2, but the similarity cutoff, top-k, and context budgets are not given. These hand-set values determine what context the Developer sees and therefore influence generation quality.
  • LLM-as-a-Judge score-to-category thresholds = [4.0,5.0] Exact, [2.0,4.0) Partial, [1.0,2.0) Mismatch
    The usability categories in Section 6.2 are defined by author-chosen score intervals. The 46.1% Exact Match headline is sensitive to the [4.0,5.0] boundary and to the prompt instruction to 'be conservative.'
assumptions (4)
  • domain assumption Human-written ground-truth glue code is a valid oracle for generation quality.
    Sections 5 and 7 compare generated code against human-written glue code using exact symbol matching and similarity metrics. If the human code is not the only acceptable implementation, the metrics understate valid alternative solutions.
  • domain assumption LLM-as-a-Judge judgments approximate developer usability judgments.
    Section 6.2 justifies a reference-aware LLM judge and validates it on 120 cases (rho=0.84, kappa=0.71). The remaining 1,187 cases are judged without human confirmation, so the full-set usability numbers depend on this assumption.
  • domain assumption Step-to-glue alignment via annotation pattern matching captures the intended executable relationship.
    Section 5 aligns steps to glue code by pattern matching and excludes steps with no glue code. Misalignments or incomplete BDD suites are filtered out, biasing the dataset toward well-specified, maintainable steps.
  • standard math Standard code-similarity metrics (CodeBLEU, METEOR, ROUGE-L) are meaningful for Java glue code.
    These metrics are inherited from prior code-generation evaluation work and used in Section 6.2 to measure lexical and structural similarity. They are standard tools, not derived in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging Behavior and Implementation: Automated Java Glue Code Generation for Behavior-Driven Development." pith.science (2026). https://pith.science/paper/RX4RMEYQ

@misc{pith2026260719703,
  author       = {Pith},
  title        = {Pith review of: Bridging Behavior and Implementation: Automated Java Glue Code Generation for Behavior-Driven Development},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RX4RMEYQ}},
  note         = {Machine review of arXiv:2607.19703}
}
read the original abstract

Behavior-Driven Development (BDD) helps technical and non-technical stakeholders share a common understanding of software requirements through natural-language scenarios. Glue code makes these scenarios executable by mapping each step to the corresponding project code. However, developing and maintaining glue code requires knowledge of both the intended behavior and the underlying codebase, making it a labor-intensive part of BDD as requirements evolve. Although large language models (LLMs) have shown strong code generation capabilities, their use for automated glue code generation remains unexplored. This task requires reasoning over underspecified behavior, related BDD artifacts, and large project codebases. We present AutoGlue, a hierarchical multi-agent framework for automated Java glue code generation. AutoGlue follows a behavior-first workflow that separates behavior interpretation, context retrieval, and code generation. A Behavior Interpreter derives the intent of a step from its scenario context, while a Developer agent retrieves relevant BDD artifacts and project code before generating the final glue code. We evaluate AutoGlue on 1,307 steps from eight open-source Java projects. Compared with few-shot prompting, AutoGlue improves API F1 by 58.7% and CodeBLEU by 43.7%. It produces directly usable glue code for 46.1% of the evaluated steps, while most partially correct outputs require only minor revisions, such as adding missing actions or refining parameters. Ablation results show that behavior interpretation and project-aware context retrieval both contribute substantially to generation quality. These findings demonstrate that LLMs can effectively connect natural-language behavior specifications with project code and support specification-driven software development.

Figures

Figures reproduced from arXiv: 2607.19703 by the authors.

Figure 1
Figure 1. BDD describes expected system behaviors as feature files, scenarios, and natural-language steps. Glue [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Challenges of glue code generation in BDD. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Overview of the AutoGlue framework. The Behavior Interpreter agent first clarifies the intended behavior of the target step. The Developer agent then supervises the remaining stages. It delegates context retrieval to two sub-agents, the BDD Context Retriever and the Project Context Retriever, and uses the retrieved context to generate the final glue code. The BDD context helps AutoGlue follow existing glue-code patt… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Usability evaluation results and breakdown of partial match cases [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 10 linked inside Pith

  1. [47]

    Xinyu Shi. 2026. Behavior-Driven Development Meets LLMs: Do We Need Them, and Do They Work?. InProceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE-Companion ’26). Association for Computing Machinery, Rio de Janeiro, Brazil. doi:10.1145/3774748.3787745 , Vol. 1, No. 1, Article . Publication date: July 2018. Bridging...

    arXiv 2026

  2. [1]

    Hisham M Abushama, Hanaa Altigani Alassam, and Fatin A Elhaj. 2021. The effect of test-driven development and behavior-driven development on project success factors: A systematic literature review based study. In2020 International Conference on Computer, Control, Electrical, and Electronics Engineering (ICCCEEE). IEEE, 1–9

    2021

  3. [2]

    Jehad Al Dallal. 2015. Identifying refactoring opportunities in object-oriented code: A systematic literature review. Information and software Technology58 (2015), 231–249

    2015

  4. [3]

    2025.Claude Sonnet 4.5

    Anthropic. 2025.Claude Sonnet 4.5. https://www.anthropic.com/claude/sonnet Accessed 30 January 2026

    2025

  5. [4]

    Víctor Manuel Arredondo-Reyes, Saúl Domínguez-Isidro, Ángel J Sánchez-García, and Jorge Octavio Ocharán- Hernández. 2023. Benefits and challenges of the behavior-driven development: A systematic literature review. In2023 11th International Conference in Software Engineering Research and Innovation (CONISOFT). IEEE Computer Society, 45–54

    2023

  6. [5]

    Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. InProceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72

    2005

  7. [6]

    Leonard Peter Binamungu, Suzanne M Embury, and Nikolaos Konstantinou. 2018. Maintaining behaviour driven development specifications: Challenges and opportunities. In2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 175–184

    2018

  8. [7]

    Elizabeth Bjarnason, Per Runeson, Markus Borg, Michael Unterkalmsteiner, Emelie Engström, Björn Regnell, Giedre Sabaliauskaite, Annabella Loconsole, Tony Gorschek, and Robert Feldt. 2014. Challenges and practices in aligning requirements with verification and validation: a case study of six companies.Empirical software engineering19, 6 (2014), 1809–1855

    2014

Show all 61 references
  1. [8]

    Elizabeth Bjarnason, Krzysztof Wnuk, and Björn Regnell. 2011. Requirements are slipping through the gaps — A case study on causes & effects of communication gaps in large-scale software development. In2011 IEEE 19th International Requirements Engineering Conference. 37–46. doi...

    2011

  2. [9]

    Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. 2024. Repairagent: An autonomous, llm-based agent for program repair.arXiv preprint arXiv:2403.17134(2024)

    2024 arXiv

  3. [10]

    Adwait Chandorkar, Nitish Patkar, Andrea Di Sorbo, and Oscar Nierstrasz. 2022. An exploratory study on the usage of gherkin features in open-source projects. In2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 1159–1166

    2022

  4. [11]

    Thiciane Couto, Sabrina Marczak, and Fabio Gomes. 2020. On the understanding of how to measure the benefits of behavior-driven development adoption: Preliminary literature results from a grey literature study. InProceedings of the XIX Brazilian Symposium on Software Quality. 1–7

    2020

  5. [12]

    Giuseppe Crupi, Rosalia Tufano, Alejandro Velasco, Antonio Mastropaolo, Denys Poshyvanyk, and Gabriele Bavota

  6. [13]

    Ora Nova Fandina, Eitan Farchi, Shmulik Froimovich, Rami Katan, Alice Podolsky, Orna Raz, and Avi Ziv. 2025. Automated Validation of LLM-based Evaluators for Software Engineering Artifacts.arXiv preprint arXiv:2508.02827 (2025)

    2025 arXiv

  7. [14]

    Muhammad Shoaib Farooq, Uzma Omer, Amna Ramzan, Mansoor Ahmad Rasheed, and Zabihullah Atal. 2023. Behavior driven development: A systematic literature review.IEEE access11 (2023), 88008–88024

    2023

  8. [15]

    Cuiyun Gao, Xing Hu, Shan Gao, Xin Xia, and Zhi Jin. 2025. The current challenges of software engineering in the era of large language models.ACM Transactions on Software Engineering and Methodology34, 5 (2025), 1–30

    2025

  9. [16]

    Jie Gao, Simret Araya Gebreegziabher, Kenny Tsu Wei Choo, Toby Jia-Jun Li, Simon Tangi Perrault, and Thomas W Malone. 2024. A taxonomy for human-llm interaction modes: An initial exploration. InExtended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–11

    2024

  10. [17]

    Ning Gao and Zhi Li. 2016. Generating testing codes for behavior-driven development from problem diagrams: A tool-based approach. In2016 IEEE 24th International Requirements Engineering Conference (RE). IEEE, 399–400

    2016

  11. [18]

    2025.Gemini 2.5 Flash

    Google. 2025.Gemini 2.5 Flash. https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/2-5-flash Accessed 30 January 2026

    2025

  12. [19]

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang

  13. [20]

    Junda He, Jieke Shi, Terry Yue Zhuo, Christoph Treude, Jiamou Sun, Zhenchang Xing, Xiaoning Du, and David Lo

  14. [21]

    Junda He, Christoph Treude, and David Lo. 2025. LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead.ACM Transactions on Software Engineering and Methodology34, 5 (2025), 1–30

    2025

  15. [22]

    Hubert F Hofmann and Franz Lehner. 2001. Requirements engineering as a success factor in software projects.IEEE software18, 4 (2001), 58

    2001

  16. [23]

    From code to courtroom: Llms as the new software judges.arXiv preprint arXiv:2503.02246(2025)

    2025 arXiv

  17. [24]

    2003.JBehave

    JBehave Organization. 2003.JBehave. https://jbehave.org/index.html

    2003

  18. [25]

    Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, and Huaming Chen. 2024. From llms to llm-based agents for software engineering: A survey of current, challenges and future.arXiv preprint arXiv:2408.02479(2024)

    2024 arXiv

  19. [26]

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review.ACM Transactions on Software Engineering and Methodology33, 8 (2024), 1–79

    2024

  20. [27]

    Sunil Kamalakar, Stephen H Edwards, and Tung M Dao. 2013. Automatically generating tests from natural language descriptions of software behavior. InInternational Conference on Evaluation of Novel Software Approaches to Software Engineering, Vol. 2. SCITEPRESS, 238–245

    2013

  21. [28]

    Shanthi Karpurapu, Sravanthy Myneni, Unnati Nettur, Likhit Sagar Gajja, Dave Burke, Tom Stiehm, and Jeffery Payne. 2024. Comprehensive evaluation and insights into the use of large language models in the automation of behavior-driven development acceptance test formulation.IEE...

    2024

  22. [29]

    Kailun Jin, Chung-Yu Wang, Hung Viet Pham, and Hadi Hemmati. 2024. Can chatgpt support developers? an empirical evaluation of large language models for code generation. InProceedings of the 21st International Conference on Mining Software Repositories. 167–171

    2024

  23. [30]

    Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges.Vicinagearth1, 1 (2024), 9

    2024

  24. [31]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. InText summarization branches out. 74–81

    2004

  25. [32]

    Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. Llms-as-judges: a comprehensive survey on llm-based evaluation methods.arXiv preprint arXiv:2412.05579(2024)

    2024 arXiv

  26. [33]

    Feng Lin, Dong Jae Kim, Zhenhao Li, Jinqiu Yang, et al. 2025. Robunfr: Evaluating the robustness of large language models on non-functional requirements aware code generation.arXiv preprint arXiv:2503.22851(2025)

    2025 arXiv

  27. [34]

    Ggaliwango Marvin, Nakayiza Hellen, Daudi Jjingo, and Joyce Nakatumba-Nabende. 2023. Prompt engineering in large language models. InInternational conference on data intelligence and cognitive informatics. Springer, 387–402

    2023

  28. [35]

    Feng Lin, Dong Jae Kim, et al. 2024. Soen-101: Code generation by emulating software process models using large language model agents.arXiv preprint arXiv:2403.15852(2024)

    2024 arXiv

  29. [36]

    2006.Introducing BDD

    Dan North. 2006.Introducing BDD. https://dannorth.net/blog/introducing-bdd/

    2006

  30. [37]

    OpenAI. [n. d.].GPT-5 mini Model | OpenAI API — platform.openai.com. Accessed 30-01-2026

    2026

  31. [38]

    Nicolas Nascimento, Alan R Santos, Afonso Sales, and Rafael Chanin. 2020. Behavior-driven development: A case study on its impacts on agile development teams. InProceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. 109–116

    2020

  32. [39]

    Ciprian Paduraru, Miruna Zavelca, and Alin Stefanescu. 2025. Agentic AI for Behavior-Driven Development Testing Using Large Language Models. InProceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART. 805–815

    2025

  33. [40]

    Debalina Ghosh Paul, Hong Zhu, and Ian Bayley. 2024. Benchmarks and metrics for evaluations of code generation: A critical review. In2024 IEEE International Conference on Artificial Intelligence Testing (AITest). IEEE, 87–94

    2024

  34. [41]

    Wendkûuni C Ouédraogo, Yinghua Li, Xueqi Dang, Xin Zhou, Anil Koyuncu, Jacques Klein, David Lo, and Tegawendé F Bissyandé. 2025. Beyond Surface Similarity: Evaluating LLM-Based Test Refactorings with Structural and Semantic Awareness.arXiv preprint arXiv:2506.06767(2025)

    2025

  35. [42]

    Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020. Codebleu: a method for automatic evaluation of code synthesis.arXiv preprint arXiv:2009.10297 (2020)

    2020 arXiv

  36. [43]

    Hossein Saiedian and Robert Dale. 2000. Requirements engineering: making the connection between the software developer and customer.Information and software technology42, 6 (2000), 419–428

    2000

  37. [44]

    Indra Kharisma Raharjana, Fadel Harris, and Army Justitia. 2020. Tool for generating behavior-driven development test-cases.Journal of Information Systems Engineering and Business Intelligence6, 1 (2020), 27

    2020

  38. [45]

    Xinyu Shi, Zhenhao Li, and An Ran Chen. 2025. Enhancing LLM-based Fault Localization with a Functionality-Aware Retrieval-Augmented Generation Framework.arXiv preprint arXiv:2509.20552(2025)

    2025

  39. [46]

    2023.BDD in Action: Behavior-driven development for the whole software lifecycle

    John Ferguson Smart and Jan Molak. 2023.BDD in Action: Behavior-driven development for the whole software lifecycle. Simon and Schuster

    2023

  40. [48]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural information processing systems30 (2017)

    2017

  41. [49]

    Ruiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan, Chun Yong Chong, and Xin Xia. 2025. Can llms replace human evaluators? an empirical study of llm-as-a-judge in software engineering.Proceedings of the ACM on Software Engineering2, ISSTA (2025), 1955–1977

    2025

  42. [50]

    Tim Storer and Ruxandra Bob. 2019. Behave nicely! automatic generation of code for behaviour driven development test suites. In2019 19th International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 228–237

    2019

  43. [51]

    Xinru Wang, Hannah Kim, Sajjadur Rahman, Kushan Mitra, and Zhengjie Miao. 2024. Human-llm collaborative annotation through effective verification of llm labels. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–21

    2024

  44. [52]

    Aidan ZH Yang, Daniel Alencar Da Costa, and Ying Zou. 2019. Predicting co-changes between functionality specifica- tions and source code in behavior driven development. In2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). IEEE, 534–544

    2019

  45. [53]

    Wenhan Wang, Chenyuan Yang, Zhijie Wang, Yuheng Huang, Zhaoyang Chu, Da Song, Lingming Zhang, An Ran Chen, and Lei Ma. 2025. Testeval: Benchmarking large language models for test case generation. InFindings of the Association for Computational Linguistics: NAACL 2025. 3547–3562

    2025

  46. [54]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629(2022)

    2022 arXiv

  47. [55]

    Tannaz Zameni, Petra van Den Bos, Jan Tretmans, Johan Foederer, and Arend Rensink. 2023. From BDD scenarios to test case generation. In2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 36–44

    2023

  48. [56]

    Zezhou Yang, Sirong Chen, Cuiyun Gao, Zhenhao Li, Xing Hu, Kui Liu, and Xin Xia. 2025. An empirical study of retrieval-augmented code generation: Challenges and opportunities.ACM Transactions on Software Engineering and Methodology(2025)

    2025

  49. [57]

    Quanjun Zhang, Chunrong Fang, Yang Xie, Yaxin Zhang, Shengcheng Yu, Weisong Sun, Yun Yang, and Zhenyu Chen

  50. [59]

    Fiorella Zampetti, Andrea Di Sorbo, Corrado Aaron Visaggio, Gerardo Canfora, and Massimiliano Di Penta. 2020. Demystifying the adoption of behavior-driven development in open source projects.Information and Software Technology123 (2020), 106311

    2020

  51. [2024]

    Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680 (2024). , Vol. 1, No. 1, Article . Publication date: July 2018. 24 Shi et al

    2024 arXiv

  52. [2025]

    On the Effectiveness of LLM-as-a-judge for Code Generation and Summarization.IEEE Transactions on Software Engineering(2025)

    2025

  53. [2026]

    A survey on large language models for software engineering.Science China Information Sciences69, 4 (2026), 141102. , Vol. 1, No. 1, Article . Publication date: July 2018

    2026

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.