Pith. sign in

REVIEW 2 major objections 3 minor 64 references

ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents

T0 review · 2 major / 3 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Legacy SRS documents can be broken into source-grounded atomic statements, regenerated into typed pre-SRS artifacts that stay traceable to the original requirements, and used to measure how well LLMs preserve requirements information throug

desk verdict A careful, well-scoped empirical study that makes a plausible case for using traceable synthetic pre-SRS artifacts in LLM-based requirements evaluation; the main quantitative claims rest on uncalibrated AlignScore thresholds, but the paper's transparency and human-grounded judge selection keep it honest. read the letter →

arxiv 2607.16564 v1 pith:7AGTG6F2 submitted 2026-07-18 cs.SE cs.AI

Ragib Shahariar Ayon , Rayed Fahmi , Sumon Biswas , Shibbir Ahmed This is my paper
classification cs.SEcs.AI
keywords requirementsengineeringSRSgenerationLLMevaluationatomicdecompositiontraceabilityartifactinformationlosssyntheticbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that ordinary legacy software requirements specifications (SRS) can be turned into a traceable evaluation substrate for LLM-based SRS generation. Instead of judging a generated SRS only by document-level fluency, the proposed pipeline splits each SRS section into atomic claims, routes those atoms to standards-inspired artifact types (use case briefs, quality constraints, context notes), and generates artifacts that remain linked to their source sentences. The study shows this transformation keeps most atoms faithful and artifacts grounded, but downstream reconstruction preserves only part of the original information. A sympathetic reader would care because it offers a way to test whether an LLM actually used the source requirements rather than just producing coherent prose.

What carries the argument

The central object is the atom: a minimal, source-supported proposition extracted from an SRS section. The study uses an atomization step to split sections into such propositions, a semantic-routing step that assigns each atom to an artifact family via plurality voting among LLMs, artifact templates that keep the atom as the source of truth, and a judge-guided refinement loop that enforces faithfulness, support, contextuality, structural alignment, and sparsity. Provenance metadata links each artifact back to its source atom and original SRS location. Two automated metrics carry the evaluation: AlignScore for factual support between generated text and its source, and SBERT cosine similarity

What would settle it

If a human-annotated sample of generated atoms, artifacts, and reconstructed SRS sentences shows that items scored above 0.80 by AlignScore often contain unsupported or distorted claims—or if reconstructing SRSs from the same artifacts with a different compiler model yields SBERT similarities below 0.5 for most atoms—then the central claim about faithfulness and recoverability would be undermined.

Watch

Extended reading notes

Core claim

The central claim is that a three-stage pipeline—decomposing SRS sections into atomic statements, routing those atoms to artifact types through multi-LLM plurality voting, and generating artifacts with constrained prompts plus iterative judge-guided refinement—can create traceable synthetic pre-SRS artifacts from legacy SRS documents. On seven public SRS documents, the resulting atoms show high faithfulness (median alignment scores 0.96–0.99) and low unsupported rates, while generated artifacts remain strongly grounded in their source atoms (alignment scores 0.80–0.94). Downstream SRS reconstruction from those artifacts is only partially successful: semantic similarity scores are moderate, a

Load-bearing premise

The automatic evaluation metrics—AlignScore for factual support and the fixed similarity/threshold values used for coverage and retention—are accurate enough for requirements text that the reported high faithfulness and low unsupported rates reflect genuine semantic support rather than artifact misjudgment.

Editorial extensions

If this is right

  • If the pipeline works as claimed, evaluation of LLM-based SRS generation can move from document-level fluency to claim-level traceability, letting researchers see exactly which source statements support each generated requirement.
  • Public SRS documents can be converted into reproducible benchmarks of synthetic pre-SRS artifacts, reducing the need for manually curated requirements datasets.
  • Stage-wise information-loss analysis can guide future systems to preserve implicit or weakly structured statements rather than discarding them during atomization.
  • The observed gap between high grounding and lower Prometheus pass rates means artifact quality checks must go beyond faithfulness to cover sparsity, completeness, and structural validity.
  • Downstream reconstruction studies should verify factuality across the full chain—source SRS, atoms, artifacts, reconstructed SRS—instead of only checking the final document.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An editorial extension: the same traceability links could enable automatic defect localization in generated SRSs, flagging each generated sentence to the atoms it failed to preserve.
  • The atom-and-artifact schema is not limited to SRSs; the same decomposition-routing-generation pattern could be applied to other documentation types, such as change requests or regulatory texts, where claim-level traceability is valuable.
  • The reliance on embedding-based and alignment-based metrics as proxies for human-judged faithfulness needs calibration against domain-specific requirements language; the paper itself treats these metrics as proxy quality, not complete correctness.
  • A testable extension would be to vary the artifact templates and see whether more permissive templates improve downstream reconstruction without increasing unsupported content, directly probing the faithfulness-completeness tradeoff the paper identifies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper introduces ReqGenX, a controlled three-stage pipeline that converts legacy SRS documents into traceable synthetic pre-SRS artifacts: section-level chunks are decomposed into atomic, source-grounded statements; atoms are semantically routed to IEEE 29148-inspired artifact types via multi-LLM plurality voting with human-in-the-loop review; and typed artifacts are generated under constrained prompts with judge-guided refinement. The evaluation uses seven text-based PURE SRS documents and multiple LLMs (Claude, GPT-4o, GPT-OSS, etc.) to study five research questions: reliability of LLM-based routing and judging, quality of atoms, quality of artifacts, downstream SRS reconstruction, and information loss. The reported results show high atom faithfulness (median AlignScore 0.96–0.99, unsupported rates mostly below 3%), strong artifact grounding (mean AlignScore 0.80–0.94), high internal judge pass rates after refinement, and partial recoverability in a four-dataset reconstruction case study (SBERT means 0.69–0.75). The authors conclude that traceable synthetic pre-SRS artifacts can support fine-grained evaluation of LLM-based SRS generation while exposing tradeoffs among faithfulness, coverage, and artifact completeness.

Significance. If the quantitative claims hold, this is a useful and timely contribution: it provides a reproducible, traceable evaluation substrate for LLM-based SRS generation and a stage-wise characterization of where information is lost. The paper has several genuine strengths. It ships a replication package with processed data, prompts, codebooks, traceability metadata, and evaluation outputs. It calibrates the judge LLM against 291 human-annotated triples (κ=0.84 for atoms, κ=0.97 for artifacts), which is a substantially stronger grounding than is common in LLM-as-a-judge work. The authors are appropriately careful to label the study as a feasibility analysis, not a deployment claim, and they explicitly discuss external validity limitations. The stage-wise analysis of information loss and failure patterns is insightful and goes beyond simple end-task accuracy. However, the central quantitative evidence depends on AlignScore and embedding-similarity thresholds that are not validated for SRS text, which is the main barrier to accepting the feasibility conclusion as stated.

major comments (2)
  1. [§3.2, §4.2–§4.4, Figs. 4/6, Table 5] The central quantitative claims—atoms are faithful (median AlignScore 0.96–0.99, unsupported rates mostly below 3%), artifacts are grounded (AlignScore 0.80–0.94), and reconstruction preserves support (AlignScore medians 0.76–0.84)—all rest on AlignScore with a fixed 0.80 support threshold. No evidence is given that AlignScore is calibrated for SRS language, which is deontic, conditional, and domain-specific, or for the short atom/artifact units used here. A general alignment/NLI model may be systematically lenient about dropped qualifiers or harsh about legitimate compact rephrasing. The threats section (§7) labels AlignScore as approximate, but that does not establish that the threshold separates supported from unsupported content in this domain. Please add a domain-level calibration: human annotation of support for a sample of atoms/artifacts against their sources, an error analysis,
  2. [§3.2, §4.5, Fig. 7] The information-retention and loss percentages (e.g., 'Claude retains 52.7–84.2% end-to-end') rely on a source-coverage threshold of 0.75 cosine similarity over text-embedding-3-large. This threshold is also not validated against human judgments of whether a source unit is genuinely retained after atomization/artifact generation. A faithfully compressed but paraphrased unit may fall below 0.75 and be counted as lost, while a unit with overlapping vocabulary but altered meaning may pass. This directly affects RQ5's conclusion that information loss is stage-specific and reflects evidence granularity rather than generation failure. Please add a sensitivity analysis and a small human check of the covered/lost classifications to demonstrate that the reported tradeoffs are not artifacts of the chosen threshold.
minor comments (3)
  1. [§4.2, Fig. 4] Finding 2 states that 'most unsupported rates remain below 3%,' but QHeadache with Claude (9.2%) and VUB with GPT-4o/GPT-OSS (8.9/8.6%) are notable exceptions. Please name them in the text so readers do not have to extract them from the figure.
  2. [§4.4, Table 5] Table 5 reports only four datasets; the text says this is a limited case study, but the caption should state the inclusion criterion and the distribution of the 473 accepted artifact-backed atoms across datasets.
  3. [§2.3, §4.3] The internal judge pass ratio is close to 1.00 by construction, because the loop iterates until the artifact passes or the refinement budget is exhausted. Please state this explicitly in the metric definition and interpret judge pass ratio as a process check, not an independent quality signal. The Prometheus results already serve this purpose, but the current wording blurs the distinction.

Circularity Check

1 steps flagged · score 2.0 of 10

One minor self-consistency in the judge pass-rate metric, but the central empirical claims remain grounded in external metrics and human-calibrated evaluation.

  1. self definitional [§2.3 Artifact Generation and Review; §4.3 RQ3; Finding 3]
    "Artifacts that meet the acceptance threshold are retained; failed artifacts are revised using judge feedback until they pass or the refinement budget is reached. ... Judge pass ratios are nearly saturated after iterative refinement, remaining close to 1.00 across almost all dataset–model pairs."

    The 'judge pass rate' uses the same rubric judge that defines the acceptance criterion in the refinement loop: artifacts are revised until that judge passes them (or the budget is exhausted). Reporting a near-1.00 pass rate after such revision is therefore partly a consequence of the loop's definition rather than an independent measurement of quality. This is a secondary metric, however: the paper also reports AlignScore grounding and Prometheus pass rates, which are not part of the revision acceptance criterion, and the judge itself was selected against 291 human-annotated triples. Thus the circularity is real but limited to one reported statistic and is not load-bearing for the central feasibility claim.

full rationale

ReqGenX is an empirical characterization, not a derivation, so most of the usual circularity patterns do not apply. The central numbers—median AlignScore for atoms and artifacts, Prometheus quality scores, SBERT reconstruction similarity, and stage-wise coverage—are computed by metrics that are not defined in terms of the generated outputs. The judge LLM is calibrated against 291 reconciled human triples with substantial inter-annotator agreement (κ=0.84 for atoms, κ=0.97 for artifacts), and the AlignScore/SBERT thresholds, while potentially miscalibrated for the SRS domain, are not fitted to the paper's own results; that is a validity threat, not circularity. There is no load-bearing self-citation: citations to PURE, PROMISE, and the authors' own replication package provide data or reproducibility support rather than justifying the main inference. The only genuine by-construction element is the judge pass rate in RQ3: artifacts are revised until the judge accepts them, and then the same judge's acceptance rate is reported as evidence. The paper is honest about this ('satisfy the pipeline's acceptance criteria'), and it triangulates with AlignScore and Prometheus, so this raises the score only slightly. Overall, the study is largely self-contained against external benchmarks, and no core prediction reduces to its inputs by definition.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper does not postulate any new physical or conceptual entities; 'synthetic pre-SRS artifacts' are a methodological artifact type. The central quantitative claims rest mainly on fixed thresholds, the AlignScore model, and the representativeness of the filtered PURE subset.

free parameters (3)
  • AlignScore support threshold = 0.80
    Used in Figure 4 and throughout to compute unsupported rates; selected without reported sensitivity analysis (§3.2).
  • Source-coverage similarity threshold = 0.75
    Overlapping two-sentence units are counted as covered if best-match cosine similarity (text-embedding-3-large) ≥0.75 (§3.2). No sensitivity analysis is reported.
  • Judge pass threshold = ≥4 on a 1–5 scale
    Atoms and artifacts are considered passing if judge or Prometheus scores are ≥4 (§3.2, §4.2). This is a categorical cut chosen by the authors.
assumptions (5)
  • domain assumption AlignScore provides a valid measure of factual support for SRS atom/artifact content
    The grounding analysis treats AlignScore as the ground truth for whether an atom supports a chunk or an artifact supports an atom (§3.2, Figure 4, Figure 6). This is an external model assumption, not independently verified for RE documents.
  • domain assumption The selected seven PURE SRS documents are sufficiently representative text-based legacy SRSs
    Filtering in §3.1 excludes figure-heavy or table-heavy documents; the authors acknowledge this may understate failures for multimodal SRSs. The claim that ReqGenX-derived artifacts support fine-grained evaluation is made on this filtered set.
  • domain assumption Human-reviewed labels (κ=0.84/0.97) are a reliable gold standard
    The judge-selection and routing-agreement analyses in §4.1 rely on two annotators with adjudication; the small annotator pool and potential shared rubric bias are not deeply analyzed.
  • ad hoc to paper LLM judge scores ≥4 indicate usable quality
    The acceptance criteria for atoms and artifacts are defined by the rubric and threshold; this is an operational definition introduced for the study, not a validated standard.
  • domain assumption text-embedding-3-large cosine similarity captures information retention
    Stage-wise retention and source coverage are computed with embedding cosine similarity (§3.2), assuming that semantic similarity corresponds to information preservation in requirements text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents." pith.science (2026). https://pith.science/paper/7AGTG6F2

@misc{pith2026260716564,
  author       = {Pith},
  title        = {Pith review of: ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7AGTG6F2}},
  note         = {Machine review of arXiv:2607.16564}
}
read the original abstract

Background: Evaluating automated Software Requirements Specification (SRS) generation is challenging because few datasets provide fine-grained traceability between source requirements, intermediate elicitation artifacts, and generated specifications. Aims: We aim to study whether legacy SRS documents can be transformed into traceable synthetic pre-SRS artifacts that support fine-grained evaluation of LLM-based SRS generation. Method: We conduct an empirical study using ReqGenX, a controlled pipeline that decomposes SRS sections into source-grounded atomic statements, routes atoms to standards-inspired artifact types through multi-LLM plurality voting, and generates artifacts using constrained prompts with iterative judge-guided refinement. We evaluate ReqGenX on seven PURE SRS documents using grounding, quality, information retention, and downstream reconstruction analyses. Results: ReqGenX produces faithful and usable atoms, with median AlignScore values typically between 0.96 and 0.99 and Prometheus scores ranging from 4.34 to 4.85. Generated artifacts remain strongly grounded in their source atoms, with AlignScore values typically between 0.80--0.94 and judge pass rates near 100%; stricter Prometheus evaluation yields pass rates from 54.8% to 97.1%. In a downstream SRS reconstruction case study, artifact-backed atoms remain recoverable from generated SRSs, with SBERT means between 0.69 and 0.75 and AlignScore medians between 0.76 and 0.84. Conclusions: Traceable synthetic pre-SRS artifacts can support more fine-grained evaluation of LLM-based SRS generation, while exposing tradeoffs among faithfulness, information retention, and artifact completeness.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 2 canonical work pages

  1. [1]

    Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap , year=

    Huang, Kaicheng and Wang, Fanyu and Huang, Yutan and Arora, Chetan , booktitle=. Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap , year=

  2. [2]

    Sentence-bert: Sentence embeddings using siamese bert-networks , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=

    2019

  3. [3]

    Science China Information Sciences , volume=

    A survey on large language models for software engineering , author=. Science China Information Sciences , volume=. 2026 , publisher=

    2026

  4. [4]

    ACM Transactions on Software Engineering and Methodology , volume=

    Large language models for software engineering: A systematic literature review , author=. ACM Transactions on Software Engineering and Methodology , volume=. 2024 , publisher=

    2024

  5. [5]

    Design Requirements Engineering: A Ten-Year Perspective: Design Requirements Workshop, Cleveland, OH, USA, June 3-6, 2007, Revised and Invited Papers , pages=

    Impact of requirements quality on project success or failure , author=. Design Requirements Engineering: A Ten-Year Perspective: Design Requirements Workshop, Cleveland, OH, USA, June 3-6, 2007, Revised and Invited Papers , pages=. 2009 , organization=

    2007

  6. [6]

    Engineering and managing software requirements , pages=

    Quality assurance in requirements engineering , author=. Engineering and managing software requirements , pages=. 2005 , publisher=

    2005

  7. [7]

    IEEE Transactions on Software Engineering , year=

    Can llms generate user stories and assess their quality? , author=. IEEE Transactions on Software Engineering , year=

  8. [8]

    2025 , volume =

    Okamoto, Ryu and Kusumoto, Shinji , booktitle =. 2025 , volume =. doi:10.1109/RE63999.2025.00056 , url =

    arXiv 2025

Show all 64 references
  1. [9]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Conformity in large language models , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  2. [10]

    arXiv preprint arXiv:2203.11171 , year=

    Self-consistency improves chain of thought reasoning in language models , author=. arXiv preprint arXiv:2203.11171 , year=

    arXiv

  3. [11]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    Scalable and domain-general abstractive proposition segmentation , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  4. [12]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Dense x retrieval: What retrieval granularity should we use? , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  5. [13]

    2020 IEEE 28th international requirements engineering conference (RE) , pages=

    Norbert: Transfer learning for requirements classification , author=. 2020 IEEE 28th international requirements engineering conference (RE) , pages=. 2020 , organization=

    2020

  6. [14]

    2023 49th Euromicro conference on software engineering and advanced applications (SEAA) , pages=

    Investigating chatgpt’s potential to assist in requirements elicitation processes , author=. 2023 49th Euromicro conference on software engineering and advanced applications (SEAA) , pages=. 2023 , organization=

    2023

  7. [15]

    Requirements engineering , volume=

    A framework to measure and improve the quality of textual requirements , author=. Requirements engineering , volume=. 2013 , publisher=

    2013

  8. [16]

    2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011) , pages=

    Extracting structured data from natural language documents with island parsing , author=. 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011) , pages=. 2011 , organization=

    2011

  9. [17]

    Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , pages=

    Triad: Automated traceability recovery based on biterm-enhanced deduction of transitive links among artifacts , author=. Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , pages=

  10. [18]

    Journal of Systems and Software , volume=

    The AmbiTRUS framework for identifying potential ambiguity in user stories , author=. Journal of Systems and Software , volume=. 2025 , publisher=

    2025

  11. [19]

    2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) , pages=

    Large language models for software engineering: Survey and open problems , author=. 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) , pages=. 2023 , organization=

    2023

  12. [20]

    2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=

    ReqInOne: A Large Language Model-Based Agent for Software Requirements Specification Generation , author=. 2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=. 2025 , organization=

    2025

  13. [21]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Gaussian transformer: a lightweight approach for natural language inference , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  14. [22]

    2024 IEEE 32nd International Requirements Engineering Conference (RE) , pages=

    Lessons from the use of natural language inference (nli) in requirements engineering tasks , author=. 2024 IEEE 32nd International Requirements Engineering Conference (RE) , pages=. 2024 , organization=

    2024

  15. [23]

    A lign S core: Evaluating Factual Consistency with A Unified Alignment Function

    Zha, Yuheng and Yang, Yichi and Li, Ruichen and Hu, Zhiting. A lign S core: Evaluating Factual Consistency with A Unified Alignment Function. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023. doi:10.18653/v1/...

    2023 doi

  16. [24]

    Advances in neural information processing systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume=

  17. [25]

    2019 IEEE 27th International Requirements Engineering Conference (RE) , pages=

    Requirements classification with interpretable machine learning and dependency parsing , author=. 2019 IEEE 27th International Requirements Engineering Conference (RE) , pages=. 2019 , organization=

    2019

  18. [26]

    Requirements engineering , volume=

    Automated classification of non-functional requirements , author=. Requirements engineering , volume=. 2007 , publisher=

    2007

  19. [27]

    2017 IEEE 25th international requirements engineering conference (RE) , pages=

    Pure: A dataset of public requirements documents , author=. 2017 IEEE 25th international requirements engineering conference (RE) , pages=. 2017 , organization=

    2017

  20. [28]

    Recommended practice for software requirements specifications (ieee) , author=

  21. [29]

    arXiv preprint arXiv:2511.14967 , year=

    MermaidSeqBench: An Evaluation Benchmark for LLM-to-Mermaid Sequence Diagram Generation , author=. arXiv preprint arXiv:2511.14967 , year=

    arXiv

  22. [30]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    AlignScore: Evaluating factual consistency with a unified alignment function , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  23. [31]

    Local prompt optimization , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) , pages=

    2025

  24. [32]

    2024 IEEE 32nd International Requirements Engineering Conference (RE) , pages=

    Using llms in software requirements specifications: An empirical evaluation , author=. 2024 IEEE 32nd International Requirements Engineering Conference (RE) , pages=. 2024 , organization=

    2024

  25. [33]

    2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=

    Exploring the use of llms for requirements specification in an it consulting company , author=. 2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=. 2025 , organization=

    2025

  26. [34]

    2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) , pages=

    Nice: Non-functional requirements identification, classification, and explanation using small language models , author=. 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) , pages=. 2025 , organization=

    2025

  27. [35]

    2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=

    The Good, the Bad, and the Uncanny: Investigating Diversity Aspects of LLM-Generated Personas for Requirements Engineering , author=. 2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=. 2025 , organization=

    2025

  28. [36]

    arXiv preprint arXiv:2506.21138 , year=

    How good are synthetic requirements? Evaluating LLM-Generated datasets for AI4RE , author=. arXiv preprint arXiv:2506.21138 , year=

  29. [37]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Pace: Improving prompt with actor-critic editing for large language model , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  30. [38]

    2025 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) , pages=

    How effective are large language models in generating software specifications? , author=. 2025 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) , pages=. 2025 , organization=

    2025

  31. [39]

    2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=

    LLM-Assisted Requirements Engineering in Agile MDD: Industry Insights and Validation , author=. 2025 IEEE 33rd International Requirements Engineering Conference (RE) , pages=. 2025 , organization=

    2025

  32. [40]

    ACM Computing Surveys (CSUR) , volume=

    Natural language processing for requirements engineering: A systematic mapping study , author=. ACM Computing Surveys (CSUR) , volume=. 2021 , publisher=

    2021

  33. [41]

    (No Title) , year=

    The PROMISE repository of software engineering databases , author=. (No Title) , year=

  34. [42]

    and Horkoff, J

    Li, F.-L. and Horkoff, J. and Mylopoulos, J. and Borgida, A. and Guizzardi, R. and Guizzardi, G. and Liu, L. and Peng, Y. , title =. 2024 , publisher =. doi:10.5281/zenodo.14330485 , url =

    2024 doi

  35. [43]

    2018 , volume=

    , journal=. 2018 , volume=

    2018

  36. [44]

    Humans and Technology, Technical Report , volume=

    Basic use case template , author=. Humans and Technology, Technical Report , volume=

  37. [45]

    Requirements Specification Templates , year=

    Volere , author=. Requirements Specification Templates , year=

  38. [46]

    2009 , publisher=

    A guide to the Business Analysis Body of Knowledge (BABOK Guide) , author=. 2009 , publisher=

    2009

  39. [47]

    2019 , publisher=

    arc42 by Example , author=. 2019 , publisher=

    2019

  40. [48]

    1994 , volume=

    , journal=. 1994 , volume=

    1994

  41. [49]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Prometheus 2: An open source language model specialized in evaluating other language models , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  42. [50]

    arXiv preprint arXiv:2412.05579 , year=

    Llms-as-judges: a comprehensive survey on llm-based evaluation methods , author=. arXiv preprint arXiv:2412.05579 , year=

    arXiv

  43. [51]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    Interpretable preferences via multi-objective reward modeling and mixture-of-experts , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  44. [52]

    ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contests , author=

  45. [53]

    Bridging Natural Language and Formal Specification–Automated Translation of Software Requirements to LTL via Hierarchical Semantics Decomposition Using LLMs , year=

    Ma, Zhi and Wen, Cheng and Su, Zhexin and Liang, Xiao and Tian, Cong and Qin, Shengchao and Yang, Mengfei , booktitle=. Bridging Natural Language and Formal Specification–Automated Translation of Software Requirements to LTL via Hierarchical Semantics Decomposition Using LLMs , year=

  46. [54]

    and Gervasi, V

    Ambriola, V. and Gervasi, V. , title =. Proceedings of the 12th International Conference on Automated Software Engineering (Formerly: KBSE) , pages =. 1997 , isbn =

    1997

  47. [55]

    ACM Trans

    He, Junda and Shi, Jieke and Zhuo, Terry Yue and Treude, Christoph and Sun, Jiamou and Xing, Zhenchang and Du, Xiaoning and Lo, David , title =. ACM Trans. Softw. Eng. Methodol. , month = feb, keywords =. 2026 , publisher =. doi:10.1145/3797276 , abstract =

    2026 doi

  48. [56]

    RE-BERT: automatic extraction of software requirements from app reviews using BERT language model , year =

    de Ara\'. RE-BERT: automatic extraction of software requirements from app reviews using BERT language model , year =. Proceedings of the 36th Annual ACM Symposium on Applied Computing , pages =. doi:10.1145/3412841.3442006 , abstract =

  49. [57]

    ACM Trans

    Hu, Xing and Niu, Feifei and Chen, Junkai and Zhou, Xin and Zhang, Junwei and He, Junda and Xia, Xin and Lo, David , title =. ACM Trans. Softw. Eng. Methodol. , month = dec, keywords =. 2026 , publisher =. doi:10.1145/3786771 , abstract =

    2026 doi

  50. [58]

    Extracting Domain Models from Textual Requirements in the Era of Large Language Models , year =

    Arulmohan, Sathurshan and Meurs, Marie-Jean and Mosser, S\'. Extracting Domain Models from Textual Requirements in the Era of Large Language Models , year =. 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) , pages =...

    2023

  51. [59]

    ACM Trans

    Terragni, Valerio and Vella, Annie and Roop, Partha and Blincoe, Kelly , title =. ACM Trans. Softw. Eng. Methodol. , month = may, articleno =. 2025 , issue_date =. doi:10.1145/3715003 , abstract =

    2025 doi

  52. [60]

    ACM Trans

    He, Junda and Treude, Christoph and Lo, David , title =. ACM Trans. Softw. Eng. Methodol. , month = may, articleno =. 2025 , issue_date =. doi:10.1145/3712003 , abstract =

    2025 doi

  53. [61]

    2025 , issue_date =

    Ferrari, Alessio and Spoletini, Paola , title =. 2025 , issue_date =. doi:10.1016/j.infsof.2025.107697 , journal =

    2025

  54. [62]

    Basak and Dalpiaz, Fabiano and Dell'Anna, Davide and Ferrari, Alessio and Franch, Xavier and Fucci, Davide , title =

    Abualhaija, Sallam and Aydemir, F. Basak and Dalpiaz, Fabiano and Dell'Anna, Davide and Ferrari, Alessio and Franch, Xavier and Fucci, Davide , title =. ACM Trans. Softw. Eng. Methodol. , month = jun, articleno =. 2024 , issue_date =. doi:10.1145/3658669 , abstract =

    2024 doi

  55. [63]

    Wang, Ruiqi and Guo, Jiyu and Gao, Cuiyun and Fan, Guodong and Chong, Chun Yong and Xia, Xin , title =. Proc. ACM Softw. Eng. , month = jun, articleno =. 2025 , issue_date =. doi:10.1145/3728963 , abstract =

    2025 doi

  56. [64]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Quantifying contamination in evaluating code generation capabilities of language models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.