Pith. sign in

REVIEW 3 major objections 5 minor 44 references

This paper argues that enterprise software built with AI code generators should accept generated code only when it passes a deterministic verification contract, not merely when it looks right in casual observation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:13 UTC pith:NETYLNBA

load-bearing objection A credible, well-organized synthesis of spec-driven development ideas with honest evidence weighting; the main risks are the idealized deterministic validator and an abstract that oversells unreplicated case numbers. the 3 major comments →

arxiv 2607.16680 v1 pith:NETYLNBA submitted 2026-07-18 cs.SE

Specification-Driven Development as the Foundation of AI-Native Enterprise Software Engineering

Mamdouh Alenezi This is my paper
classification cs.SE
keywords specification-driven developmentvibe codingAI-native software engineeringspecification governancelarge language modelssoftware qualityISO/IEC 25010verification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's central claim: enterprise software built with AI code generators should accept generated code only when it passes a deterministic verification contract, not when it merely appears to work in casual testing. It identifies four documented failure modes of ungoverned conversational generation, then proposes the Specification Governance Reference Model (SGRM), a tool-independent architecture built on a four-component specification contract and a deterministic validator. The author argues that this converts probabilistic AI generation into deterministic, auditable engineering, and evaluates the model against standard quality characteristics, citing a reported 73% security-defect reduction and halved time-to-market as provisional supporting evidence. A sympathetic reader would care because the paper offers a concrete, falsifiable mechanism for where to place human discipline when LLMs generate code at scale.

Core claim

The paper's central discovery is the Specification Governance Reference Model (SGRM). A specification is a machine-readable tuple S=(F,Q,K,Σ) of functional obligations, quality constraints, constitutional security rules, and architectural structure; a stochastic generator samples candidate implementations; and a deterministic validator V=Vstat∧Vtest∧Vcontract∧Varch admits only implementations that satisfy the contract. Specification governance holds two invariants: every integrated artifact belongs to the acceptance set A(S), and every intentional change to system behavior is initiated by a change to S. In this model, vibe coding's acceptance-by-observation is a strictly weaker criterion tha

What carries the argument

The central mechanism is the four-component specification contract S=(F,Q,K,Σ) paired with the deterministic validator V composed of static/constitutional checks, specification-derived test execution, contract and proof obligations, and architectural conformance checking. The validator defines the acceptance set A(S), which forms a deterministic boundary around the stochastic generator. The operational core is a closed-loop regeneration algorithm: rejection sampling in which validator diagnostics feed back as corrective context, with budget exhaustion escalating to human governance. Three rigor tiers (spec-first, spec-anchored, spec-as-source) formalize progressively stronger process invaria

Load-bearing premise

The load-bearing premise is that a deterministic, decidable validator can automatically enforce all four specification components; for real enterprise systems, contract checking and architectural conformance are in general undecidable or expensive, and if the validator is incomplete the guarantee that A(S) membership is strictly stronger than observational sampling weakens.

What would settle it

A controlled study where matched enterprise teams build the same feature under vibe coding and under SGRM, measuring defect density, maintenance effort, and time-to-market; the central claim would be falsified if specification-governed teams show no fewer defects or higher maintenance costs after the same development period.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If SGRM is correct, organizations can adopt specification governance incrementally along the rigor tiers, moving from one-shot spec-first validation to continuous spec-anchored validation without changing their underlying generators.
  • Strengthening any validator component shrinks A(S) without touching the generator, so quality improvements compose through the deterministic boundary rather than through model retraining.
  • Under spec-anchored or spec-as-source maintenance, drift between intent and implementation is detected at edit time, and regeneration bounded by the validator acts as continuous refactoring at low marginal human cost.
  • Traceability becomes a byproduct of the pipeline, reducing compliance demonstration to two checks: that the specification captures applicable obligations, and that the pipeline enforces the validator.
  • The enterprise case evidence, if replicated, implies that the binding constraint on AI-augmented delivery shifts from model capability to specification clarity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The strongest pro-SDD quantifications rest on single case studies, so a natural next experiment is a preregistered comparison of matched teams building the same feature under vibe coding versus SGRM, manipulating specification discipline directly to isolate whether governance, not AI presence, is the moderating variable.
  • If the model's cost relocation is right, specification authoring will become the scarce engineering skill; a testable corollary is that teams with stronger specification authors will realize larger quality gains from AI generation.
  • The paper leaves open how to measure specification authoring cost; building such a measure and comparing maintenance effort across the three rigor tiers is a concrete extension that would make the adoption decision surface in its boundary conditions operational.
  • The validator assumption is the load-bearing point: measuring how incomplete or expensive Vstat, Vcontract, and Varch are on realistic enterprise codebases would directly test whether A(S) membership remains strictly stronger than observational sampling in practice.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that enterprise-grade software engineering with AI requires Specification-Driven Development (SDD) rather than 'vibe coding' (observational acceptance of AI-generated artifacts). It identifies four failure modes of ungoverned generation (productivity–reliability paradox, architectural erosion, security exposure, technical debt) from a verified literature corpus, then introduces the Specification Governance Reference Model (SGRM). SGRM formalizes specifications as four-component contracts S=(F,Q,K,Σ), places a deterministic validator V around the stochastic generator, defines three rigor tiers, a closed-loop regeneration algorithm, and six falsifiable design propositions. The model is evaluated analytically against ISO/IEC 25010, with evidence from the literature. The paper explicitly weights evidence maturity and includes boundary conditions and threats to validity.

Significance. If the central thesis holds, SGRM would provide a unifying, tool-independent vocabulary for specification governance in AI-native development, with transferable mechanisms (contracts, constitutional constraints, architectural conformance checking) and falsifiable propositions for future empirical work. The paper's strengths include its systematic corpus verification, explicit design-science method, clear formal definitions, honest evidence-maturity weighting (distinguishing replicated findings from unreplicated case studies), and the deliberate statement of boundary conditions rather than an overreaching claim. The two headline quantitative results (73% security-defect reduction, 50% time-to-market reduction) are appropriately acknowledged as single-case-study evidence in the body, though the abstract presents them without qualification. The major weakness is the idealized assumption that the validator is a deterministic, decidable, and complete procedure for the full four-component specification.

major comments (3)
  1. [§5.1, Definition 3] The definition of the validator V as a 'deterministic, decidable procedure' that checks obligations quantified over F, Q, K, and Σ is load-bearing for the central claim that membership in A(S) is strictly stronger than observational sampling. For general programs, contract/proof obligations and architectural conformance (Vcontract, Varch) are undecidable or require interactive proof effort; the parenthetical 'where F admits formal semantics' does not define a decidable fragment nor show that the enterprise exemplars (Marri's ten-CWE banking constitution, Vilas Boas's brownfield microservices) fall inside it. The paper's own list of open problems in §7.3 (specification languages, verification-native generators) confirms the mechanism is partly aspirational. Additionally, §5.3 includes 'N-version differential assessment' in the verification layer (L3), which is stochastic, not deterministi
  2. [Abstract and §6.2] The two headline quantitative claims — '73% reduction in security defects' and '50% reduction in time-to-market' — are presented in the abstract as if they were established outcomes of the approach. The body correctly labels them as unreplicated single-case-study evidence (§6.4) and states the argument does not rest on them, but the abstract includes no such caveat. This mismatch matters for a journal readership: the abstract will be read in isolation and may overstate the evidentiary base. The abstract should either add a qualifier (e.g., 'in single unreplicated case studies') or de-emphasize these figures in favor of the replicated failure-mode evidence that the paper says carries the weight.
  3. [§5.1, 'Observational sampling is strictly weaker'] The formal contrast between vibe coding and SDD hinges on the claim that acceptance by A(S) 'checks obligations quantified over F,Q,K,Σ' whereas observational sampling checks finitely many behaviors. If the validator is incomplete (as any practical static analyzer or test suite must be), then A(S) membership is not equivalent to satisfying all obligations; it only certifies passage of the implemented checks. In that case the gap between the two acceptance criteria is quantitative (number and kind of checks) rather than the strict qualitative superiority asserted in the text. The paper should weaken the wording to 'can be strictly stronger' and spell out the conditions under which the strengthening holds (e.g., for the decidable subsets of F, K, and Σ).
minor comments (5)
  1. [§3, Table 1] The corpus includes several preprints and self-citations ([1], [4], [15], [16]) alongside peer-reviewed sources. The verification discipline is described, but a statement about the proportion of preprints or a reproducibility package for the corpus would strengthen the 'verified' claim.
  2. [§5.3, Figure 2] The arrows in Figure 2 are not fully explained in the caption; specifically, the difference between solid and dashed arrows (generation/verification flow vs. feedback) is clear only from the main text, not the figure itself. The caption should include a legend.
  3. [§2.3] The sentence 'The framework maps onto real tooling: behavior-driven development frameworks, API-contract ecosystems, and AI-assisted toolkits such as GitHub Spec Kit' lists concrete tools without citations (other than [2]); if these are discussed only in [2], the sentence could be phrased as 'as described in [2]' to avoid implying independent verification.
  4. [§7.3, Research agenda] The open question about specification languages is well placed, but the paper's formal model could benefit from a paragraph in §5 acknowledging that the current definitions assume a yet-to-be-designed language, so readers are not left to infer this from the open problems.
  5. [§9, Conclusion] The conclusion repeats 'deterministic, auditable engineering' without the caveats about validator incompleteness that the earlier sections properly include. One sentence acknowledging that the deterministic boundary is as strong as the implemented checks would avoid overstatement.

Circularity Check

0 steps flagged

No load-bearing circularity; minor self-citations in background and an acknowledged in-sample evaluation component do not undermine the central derivation.

full rationale

The paper's derivation chain is largely self-contained. The failure-mode analysis (F1-F4) is supported by externally replicated evidence (Peng et al., Pearce et al., Perry et al., Fu et al., Fawzy et al., Liu et al.), and SGRM is explicitly a formalization of external methodological work by Piskala [2], Marri [15], and Vilas Boas et al. [16]. The central contrast between observational sampling and acceptance-by-verification is definitional (Definitions 1-4): the paper states 'This vocabulary makes the contrast with vibe coding precise,' so the 'strictly weaker' claim is presented as a formal framing, not as an empirical prediction derived from the model. The strongest pro-SDD quantifications (73% security-defect reduction, 50% time-to-market) are explicitly flagged by the paper itself as unreplicated case evidence (Section 6.4: 'the argument of this article does not rest its weight on them'), so they are not concealed inputs masquerading as independent confirmation. The author's own prior works [1,4] are cited only for background and competency-model framing, not for the SGRM construction or the core empirical claim, so they are not load-bearing self-citations. The weakest point is Definition 3's assumption of a 'deterministic, decidable procedure' for contract, architectural, and constitutional checks; however, Section 7.3 concedes that 'designing such languages... is an open design problem,' making this an acknowledged idealization and boundary condition rather than a hidden circular step. Score 2 reflects the minor self-citations and the mild in-sample texture of using SDD case studies both as sources for formalization and as supporting evidence, without treating these as circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced; SGRM is a conceptual artifact. Its six design propositions are falsifiable in principle but are not tested in this paper, so they do not constitute independent empirical evidence. The free-parameter count is zero because the paper performs no fits; the load-bearing costs are the four stated axioms above.

axioms (4)
  • domain assumption The verified 44-source corpus is representative of the empirical evidence on AI-assisted development.
    Stage 1 corpus assembly (Section 3) uses judgment-based inclusion and excludes unverifiable citations; representativeness is not independently demonstrated.
  • domain assumption The validator V is deterministic, decidable, and can automatically enforce all four specification components.
    Definition 3 states V is a deterministic, decidable procedure, but contract checking and architectural conformance are in general undecidable or expensive; the paper only notes 'where F admits formal semantics'.
  • domain assumption Governance, not the presence of AI, is the moderating variable for software quality outcomes.
    Section 4.5 and Section 7.3 explicitly mark the moderating-variable inference as a hypothesis consistent with the data, not a tested finding; SGRM is designed around it.
  • domain assumption ISO/IEC 25010 is the correct external standard for 'enterprise-grade' quality.
    Section 6 uses ISO/IEC 25010 as the evaluation benchmark; this is a reasonable but contestable choice, since other quality frameworks could yield different mappings.

pith-pipeline@v1.3.0-alltime-deepseek · 16913 in / 10986 out tokens · 116062 ms · 2026-08-01T20:13:57.046587+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) and agentic AI are shifting software engineering from manual coding toward intent specification, architecture, and governance. Two paradigms have emerged: vibe coding, an intuition-driven approach accepting AI artifacts via observed behavior, and Specification-Driven Development (SDD), which uses structured specifications as the authoritative source of truth. This article makes three contributions. First, based on a verified literature corpus, it identifies failure modes of ungoverned conversational generation: the productivity-reliability paradox, architectural erosion from limited context, security exposure, and technical debt. Second, it introduces the Specification Governance Reference Model (SGRM). This tool-independent framework defines four-component specification contracts, constrains stochastic generation via deterministic validation, formalizes three rigor levels, and integrates generation, verification, and governance into a closed-loop architecture. Third, it evaluates SGRM against ISO/IEC 25010, mapping quality characteristics to governance mechanisms. Empirical evidence supports this, reporting a 73% reduction in security defects under constitutional constraints and a 50% reduction in time-to-market through specification-governed agentic delivery. The analysis concludes that while vibe coding is valuable for ideation and rapid prototyping, enterprise software requires specification governance to transform probabilistic AI generation into deterministic, auditable engineering. Boundary conditions, threats to validity, and future research directions are discussed.

Figures

Figures reproduced from arXiv: 2607.16680 by Mamdouh Alenezi.

Figure 1
Figure 1. Figure 1: The rigor continuum from vibe coding to spec-as-source development. Moving right, the specification gains [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The Specification Governance Reference Model (SGRM). Solid arrows denote the generation-and [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Selected quantified findings from the verified corpus [9–11,15,16]. Metrics are heterogeneous across studies [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 7 linked inside Pith

  1. [1]

    The rise of AI-native software engineering: Implications for practice, education, and the future workforce.arXiv preprint arXiv:2606.12986, 2026

    Mamdouh Alenezi. The rise of AI-native software engineering: Implications for practice, education, and the future workforce.arXiv preprint arXiv:2606.12986, 2026

    Pith/arXiv arXiv 2026

  2. [2]

    Spec-driven development: From code to contract in the age of AI coding assistants.arXiv preprint arXiv:2602.00180, 2026

    Deepak Babu Piskala. Spec-driven development: From code to contract in the age of AI coding assistants.arXiv preprint arXiv:2602.00180, 2026

    arXiv 2026

  3. [3]

    Hassan, Dayi Lin, Gopi Krishnan Rajbahadur, Keheliya Gallaba, Filipe R

    Ahmed E. Hassan, Dayi Lin, Gopi Krishnan Rajbahadur, Keheliya Gallaba, Filipe R. C ˆogo, Boyuan Chen, et al. Towards AI-native software engineering (SE 3.0): A vision and a challenge roadmap.arXiv preprint arXiv:2410.06107, 2024

    arXiv 2024

  4. [4]

    Rethinking software engineering for agentic AI systems.arXiv preprint arXiv:2604.10599, 2026

    Mamdouh Alenezi. Rethinking software engineering for agentic AI systems.arXiv preprint arXiv:2604.10599, 2026

    Pith/arXiv arXiv 2026

  5. [5]

    Application of large language models to software engineering tasks: Opportunities, risks, and implications.IEEE Software, 40(3):4–8, 2023

    Ipek Ozkaya. Application of large language models to software engineering tasks: Opportunities, risks, and implications.IEEE Software, 40(3):4–8, 2023

    2023

  6. [6]

    Vibe coding in practice: Motivations, challenges, and a future outlook—a grey literature review

    Ahmed Fawzy, Amjed Tahir, and Kelly Blincoe. Vibe coding in practice: Motivations, challenges, and a future outlook—a grey literature review. InProceedings of the 48th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP ’26). ACM, 2026. Preprint: arXiv:2510.00328

    arXiv 2026

  7. [7]

    vibe coding

    Andrej Karpathy. There’s a new kind of coding i call “vibe coding”. X (formerly Twitter) post,https://x. com/karpathy/status/1886192184808149383, February 2025

    arXiv 2025

  8. [8]

    Vibe coding as a reconfiguration of intent mediation in software development: Definition, implications, and research agenda

    Christian Meske, Tobias Hermanns, Esther von der Weiden, Kai-Uwe Loser, and Thorsten Berger. Vibe coding as a reconfiguration of intent mediation in software development: Definition, implications, and research agenda. IEEE Access, 13:213242–213259, 2025. DOI: 10.1109/ACCESS.2025.3645466. 15 APREPRINT- JULY21, 2026

    arXiv 2025

  9. [9]

    The impact of AI on developer productivity: Evidence from GitHub Copilot.arXiv preprint arXiv:2302.06590, 2023

    Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. The impact of AI on developer productivity: Evidence from GitHub Copilot.arXiv preprint arXiv:2302.06590, 2023

    Pith/arXiv arXiv 2023

  10. [10]

    Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability

    Markus Borg, Dave Hewett, Nadim Hagatulah, No´emie Couderc, Emma S¨oderberg, Donald Graham, Uttam Kini, and Dave Farley. Echoes of AI: Investigating the downstream effects of AI assistants on software maintainability. Empirical Software Engineering, 31(6), 2026. Preprint: arXiv:2507.00788

    arXiv 2026

  11. [11]

    Asleep at the keyboard? assessing the security of GitHub Copilot’s code contributions

    Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. Asleep at the keyboard? assessing the security of GitHub Copilot’s code contributions. InProceedings of the 2022 IEEE Symposium on Security and Privacy (SP), pages 754–768. IEEE, 2022

    2022

  12. [12]

    Do users write more insecure code with AI assistants? InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS ’23), pages 2785–2799

    Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. Do users write more insecure code with AI assistants? InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS ’23), pages 2785–2799. ACM, 2023

    2023

  13. [13]

    Lost at C: A user study on the security implications of large language model code assistants

    Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt. Lost at C: A user study on the security implications of large language model code assistants. InProceedings of the 32nd USENIX Security Symposium, pages 2205–2222. USENIX Association, 2023

    2023

  14. [14]

    Security weak- nesses of copilot-generated code in GitHub projects: An empirical study.ACM Transactions on Software Engi- neering and Methodology, 2025

    Yujia Fu, Peng Liang, Amjed Tahir, Zengyang Li, Mojtaba Shahin, Jiaxin Yu, and Jinfu Chen. Security weak- nesses of copilot-generated code in GitHub projects: An empirical study.ACM Transactions on Software Engi- neering and Methodology, 2025

    2025

  15. [15]

    Constitutional spec-driven development: Enforcing security by construction in AI-assisted code generation.arXiv preprint arXiv:2602.02584, 2026

    Sudheer Reddy Marri. Constitutional spec-driven development: Enforcing security by construction in AI-assisted code generation.arXiv preprint arXiv:2602.02584, 2026

    arXiv 2026

  16. [16]

    One developer is all you need: A case study of an AI-augmented one-person squad in a brownfield enterprise

    Marcelo Vilas Boas, Gustavo Pinto, Edward Roberto Monteiro, Vinicius Fernandes Carida, and Danilo Ribeiro. One developer is all you need: A case study of an AI-augmented one-person squad in a brownfield enterprise. arXiv preprint arXiv:2605.18461, 2026

    Pith/arXiv arXiv 2026

  17. [17]

    ISO/IEC 25010:2023 — systems and software engineering — systems and software quality require- ments and evaluation (SQuaRE) — product quality model

    ISO/IEC. ISO/IEC 25010:2023 — systems and software engineering — systems and software quality require- ments and evaluation (SQuaRE) — product quality model. International Organization for Standardization, Geneva, 2023

    2023

  18. [18]

    Hevner, Salvatore T

    Alan R. Hevner, Salvatore T. March, Jinsoo Park, and Sudha Ram. Design science in information systems research.MIS Quarterly, 28(1):75–105, 2004

    2004

  19. [19]

    Wieringa.Design Science Methodology for Information Systems and Software Engineering

    Roel J. Wieringa.Design Science Methodology for Information Systems and Software Engineering. Springer, Berlin, Heidelberg, 2014

    2014

  20. [20]

    Large language models for software engineering: A systematic literature review.ACM Transactions on Software Engineering and Methodology, 33(8):220:1–220:79, 2024

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. Large language models for software engineering: A systematic literature review.ACM Transactions on Software Engineering and Methodology, 33(8):220:1–220:79, 2024

    2024

  21. [21]

    Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, and Jie M. Zhang. Large language models for software engineering: Survey and open problems. InProceedings of the IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE), pages 31–53. IEEE, 2023

    2023

  22. [22]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Pith/arXiv arXiv 2021

  23. [23]

    Competition- level code generation with AlphaCode.Science, 378(6624):1092–1097, 2022

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R´emi Leblond, et al. Competition- level code generation with AlphaCode.Science, 378(6624):1092–1097, 2022

    2022

  24. [24]

    PhD thesis, University of California, Berkeley, 2008

    Armando Solar-Lezama.Program Synthesis by Sketching. PhD thesis, University of California, Berkeley, 2008

    2008

  25. [25]

    Program synthesis.Foundations and Trends in Pro- gramming Languages, 4(1–2):1–119, 2017

    Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. Program synthesis.Foundations and Trends in Pro- gramming Languages, 4(1–2):1–119, 2017

    2017

  26. [26]

    Paige, and Alfonso Pierantonio

    Antonio Bucchiarone, Jordi Cabot, Richard F. Paige, and Alfonso Pierantonio. Grand challenges in model-driven engineering: An analysis of the state of the research.Software and Systems Modeling, 19(1):5–13, 2020

    2020

  27. [27]

    design by contract

    Bertrand Meyer. Applying “design by contract”.Computer, 25(10):40–51, 1992

    1992

  28. [28]

    Boehm.Software Engineering Economics

    Barry W. Boehm.Software Engineering Economics. Prentice-Hall, Englewood Cliffs, NJ, 1981

    1981

  29. [29]

    Large language models meet NL2Code: A survey

    Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang, and Jian-Guang Lou. Large language models meet NL2Code: A survey. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023), pages 7443–7464. ACL, 2023

    2023

  30. [30]

    Frederick P. Brooks. No silver bullet: Essence and accidents of software engineering.Computer, 20(4):10–19, 1987. 16 APREPRINT- JULY21, 2026

    1987

  31. [31]

    Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation. InAdvances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023

    2023

  32. [32]

    Guidelines for performing systematic literature reviews in software engineering

    Barbara Kitchenham and Stuart Charters. Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE-2007-01, Keele University and Durham University, 2007

    2007

  33. [33]

    M¨antyl¨a

    Vahid Garousi, Michael Felderer, and Mika V . M¨antyl¨a. Guidelines for including grey literature and conducting multivocal literature reviews in software engineering.Information and Software Technology, 106:101–121, 2019

    2019

  34. [34]

    Human-written vs

    Domenico Cotroneo, Cristina Improta, and Pietro Liguori. Human-written vs. AI-generated code: A large-scale study of defects, vulnerabilities, and complexity.arXiv preprint arXiv:2508.21634, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    An empirical study of automated unit test generation for Python.Empirical Software Engineering, 28(2):36, 2023

    Stephan Lukasczyk, Florian Kroiß, and Gordon Fraser. An empirical study of automated unit test generation for Python.Empirical Software Engineering, 28(2):36, 2023

    2023

  36. [36]

    Lahiri, and Siddhartha Sen

    Caroline Lemieux, Jeevana Priya Inala, Shuvendu K. Lahiri, and Siddhartha Sen. CodaMosa: Escaping cover- age plateaus in test generation with pre-trained large language models. InProceedings of the 45th IEEE/ACM International Conference on Software Engineering (ICSE ’23), pages 919–931. IEEE, 2023

    2023

  37. [37]

    N-version assessment and enhancement of generative AI.IEEE Software, 42(2):76–83, 2025

    Marcus Kessel and Colin Atkinson. N-version assessment and enhancement of generative AI.IEEE Software, 42(2):76–83, 2025. Preprint: arXiv:2409.14071

    Pith/arXiv arXiv 2025

  38. [38]

    Morescient GAI for software engineering.ACM Transactions on Software Engineering and Methodology, 2025

    Marcus Kessel and Colin Atkinson. Morescient GAI for software engineering.ACM Transactions on Software Engineering and Methodology, 2025. DOI: 10.1145/3709354

    doi:10.1145/3709354 2025

  39. [39]

    Big code != big vocabulary: Open-vocabulary models for source code

    Rafael-Michael Karampatsis, Hlib Babii, Romain Robbes, Charles Sutton, and Andrea Janes. Big code != big vocabulary: Open-vocabulary models for source code. InProceedings of the ACM/IEEE 42nd International Conference on Software Engineering (ICSE ’20), pages 1073–1085. ACM, 2020

    2020

  40. [40]

    David L. Parnas. On the criteria to be used in decomposing systems into modules.Communications of the ACM, 15(12):1053–1058, 1972

    1972

  41. [41]

    Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Franc ¸ois Crespo, and Dan Dennison

    D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Franc ¸ois Crespo, and Dan Dennison. Hidden technical debt in machine learning systems. InAdvances in Neural Information Processing Systems 28 (NIPS 2015), pages 2503–2511, 2015

    2015

  42. [42]

    Jane Cleland-Huang, Orlena C. Z. Gotel, Jane Huffman Hayes, Patrick M ¨ader, and Andrea Zisman. Software traceability: Trends and future directions. InFuture of Software Engineering Proceedings (FOSE 2014, ICSE), pages 55–69. ACM, 2014

    2014

  43. [43]

    Requirements engineering challenges in building AI-based com- plex systems

    Hrvoje Belani, Marin Vukovic, and ˇZeljka Car. Requirements engineering challenges in building AI-based com- plex systems. InProceedings of the 27th IEEE International Requirements Engineering Conference Workshops (REW), pages 252–255. IEEE, 2019

    2019

  44. [44]

    Formal verification of smart contracts: Short paper

    Karthikeyan Bhargavan, Antoine Delignat-Lavaud, C´edric Fournet, Anitha Gollamudi, Georges Gonthier, Nadim Kobeissi, Natalia Kulatova, Aseem Rastogi, Thomas Sibut-Pinote, Nikhil Swamy, and Santiago Zanella- B´eguelin. Formal verification of smart contracts: Short paper. InProceedings of the 2016 ACM Workshop on Programming Languages and Analysis for Secur...

    2016