Pith. sign in

REVIEW 4 major objections 6 minor 34 references

The paper claims that converting tabular design documents from CSV into a header-value-distinguishing format (Markdown for natural-language-heavy text, JSON for symbol-heavy text) improves LLM consistency-check recall by 0.43 to 0.63, and t

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Converting tabular design documents into header-aware Markdown or JSON formats lets GPT models catch cross-document inconsistencies with recall up to 0.96 on short documents, but performance collapses beyond 5000 characters.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Worth a look for the format-selection heuristic and the 11-perspective taxonomy, but the headline recall gain is confounded by an extra LLM conversion pass and the evaluation is too thin to support strong claims. the 4 major comments →

arxiv 2509.09975 v1 pith:SQRVRYJN submitted 2025-09-12 cs.SE

Development of Automated Software Design Document Review Methods Using Large Language Models

Takasaburo Fukuda , Takao Nakagawa , Keisuke Miyazaki , Susumu Tokumoto This is my paper
classification cs.SE
keywords software design reviewlarge language modelsconsistency checkCSV to MarkdownCSV to JSONheader-value distinctionautomated document reviewExcel tables
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that general-purpose LLMs can automate at least part of software design document reviews, specifically consistency checks. It organizes design review into 11 perspectives, argues that only lower-level perspectives are feasible with off-the-shelf LLMs, and proposes a preprocessing step: convert spreadsheet-style tables, which dominate Japanese design documents, from CSV into a format that explicitly marks headers versus values. Across GPT-3.5, GPT-4, and GPT-4o, this conversion raises recall of planted inconsistencies by 0.43 to 0.63. The right target format matters: Markdown works better for natural-language-heavy documents and JSON for symbol-heavy ones. The gain holds up to about 5000 characters, with accuracy dropping sharply beyond that.

Core claim

The paper's central discovery is that the reason LLMs fail on Excel-based design documents is not simply token count or prompt wording but the loss of the header/value distinction when tables are flattened to CSV. Converting the table into Markdown or JSON, which marks headers and values explicitly, lets the LLM perform a bidirectional consistency check between two design documents with recall as high as 0.96 for GPT-4 and GPT-4o, up from 0.33 and 0.53 in raw CSV. The paper further shows the conversion format should match the document's character: natural-language-rich documents do better in Markdown, symbol-rich documents do better in JSON, and a part-of-speech-based statistic P_s decides t

What carries the argument

Header-value-distinguishing conversion: before review, the LLM is prompted to convert a CSV-flattened table into Markdown (for natural-language-rich documents) or JSON (for symbol-rich documents), selected by a threshold on the proportion of symbols, characters, and nouns P_s. The conversion is itself an LLM step, drawing on Chain-of-Thought prompting; the resulting markup makes the relationship between column headers and cell values explicit, which is what the consistency check then uses to detect bidirectional mismatches such as a process ID changed from 'execute' to 'execution'.

Load-bearing premise

The load-bearing premise, stated in the paper's motivation and scaling sections, is that the recall gain comes from making header/value relationships explicit rather than from the added LLM conversion step, and that conversion remains faithful up to roughly 5000 characters.

What would settle it

Take the same document pairs and compare (a) raw CSV review, (b) LLM-converted Markdown review, and (c) a control where the LLM is asked to 'convert' to a format that does not distinguish headers from values but has similar token count and processing steps. If (c) matches (b)'s recall, the header-value distinction is not the operative cause. A second check: documents over 5000 characters where the LLM conversion loses no information should still show the recall drop, which would show the length limitation is not merely a conversion artifact.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claim is correct, consistency checks, a frequent review task, can be partially delegated to general-purpose LLMs without fine-tuning, just by changing the input representation.
  • Markdown should be used for natural-language-heavy design documents and JSON for symbol-heavy documents; a simple POS-based rule can pick the format.
  • The recall gain is substantial (0.43-0.63) while precision stays high, so partial automation of design reviews is plausible for short documents.
  • For documents under roughly 5000 characters, GPT-4 and GPT-4o reach recall near 0.9; beyond that, performance collapses, so document length is the main operational constraint.
  • Other Level 1 and Level 2 review perspectives, such as ambiguity or traceability checks, may benefit from the same conversion technique, though the paper only evaluates consistency checks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The conversion step may itself be a form of reasoning: part of the recall gain could come from forcing the LLM to parse the table into a structured representation before answering, not purely from the format. Comparing LLM-performed conversion against a deterministic converter would separate these.
  • The header/value principle likely extends beyond design reviews: any structured spreadsheet input to an LLM, such as test cases, configuration matrices, or requirements traceability tables, should benefit from explicit markup.
  • The P_s heuristic is a rough proxy; a learned classifier or letting the LLM choose the format might close the gap for documents where the POS signal is ambiguous.
  • Because longer documents fail partly due to information loss during conversion, chunking by table regions and converting each chunk independently might extend the 5000-character boundary; the paper suggests RAG as future work, so this is an extension of its direction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes a method for automating consistency-check reviews of software design documents with LLMs. The authors first derive 11 review perspectives and classify them by difficulty and by whether multiple documents must be referenced. For perspectives that general-purpose LLMs can handle, they propose converting tabular Excel/CSV design documents into Markdown (for natural-language-rich documents) or JSON (for symbol-rich documents), using a POS-based measure P_s and threshold θ_s, so that headers and values become distinguishable. They evaluate the method on five pairs of short Japanese design documents from actual business practice, using GPT-3.5, GPT-4, and GPT-4o, and report that recall for injected inconsistencies improves by 0.43–0.63 over unconverted CSV. They also report that format selection matters (Markdown for natural-language-rich, JSON for symbol-rich) and that recall drops sharply for documents above 5000 characters. They conclude that the method is practically applicable for shorter documents.

Significance. If the claims hold, the paper is practically useful: it contributes a taxonomy of review perspectives, a low-cost conversion strategy, and evidence across three GPT models that format affects performance on table-heavy documents. The work addresses an under-explored area (document review rather than code generation) and uses real-world SDEM-based documents. However, the empirical basis is preliminary: the key comparison is confounded by an extra LLM conversion pass, the test set is small and partly selected using the same threshold that the format selector uses, and no uncertainty quantification is provided. The paper's main value is as a proof of concept; the current evidence does not yet establish the causal claim about header-value-distinguishing formats.

major comments (4)
  1. [§IV.A, §V.B–C, Table II] The RQ1 comparison confounds output format with the number of LLM inference passes. In §V.B the proposed pipeline instructs the LLM first to convert the CSV into a header-value-distinguishing format and then to perform the consistency check, whereas the 'Unconverted Method' feeds CSV directly and skips the conversion pass. The 0.43–0.63 recall gain in Table II could therefore be produced by the additional conversion/restatement pass (a form of chain-of-thought) rather than by Markdown's explicit header syntax. A control condition that adds an equally effortful but format-neutral LLM pass—or that uses pre-converted Markdown vs CSV with appended header estimates—is needed to isolate the format effect. Without it, the RQ1 answer overstates the causal role of the conversion format.
  2. [§V.C–E, Tables II–V] The evidence is statistically thin. Each condition uses only five document pairs with two injected defects each; the paper reports aggregate precision/recall but no confidence intervals, per-pair variance, or significance tests, despite stating 'significant differences' in §V.C and 'significantly improves' in the RQ1 answer. At default temperature/top-p, 10 runs per pair should yield run-to-run variance that could be quantified. The strong quantitative claims (e.g., recall 0.87–0.96 for the proposed method) need at least confidence intervals or paired comparisons across the five document pairs.
  3. [§IV.B, §V.D, Eq. (1)] RQ2 is tested on documents that were selected precisely because their P_s values fall on a particular side of θ_s ('specifically selected based on Equation (1)'). Since θ_s was itself fitted on a representative document set with an F1 criterion, the evaluation is circular for validating format selection: it does not test the classifier's ability to choose correctly on unseen documents, and any comparison of Markdown vs JSON is conditional on a threshold that is never reported. An independent held-out set with known labels, a reported θ_s, and classification accuracy is necessary to support the RQ2 conclusion.
  4. [§V.E, Table V] The RQ3 attribution that the >5000-character drop is partly due to 'information lost during the Markdown conversion process' is not tested. No CSV baseline is reported at these lengths, and the LLM performs the conversion, so conversion failures are conflated with format limitations. The paper should report conversion fidelity (e.g., whether the Markdown output preserved all original header/value pairs) and, if possible, compare with longer CSV baselines or chunked controls. This is load-bearing for the scalability conclusion, not just a side observation.
minor comments (6)
  1. [§V.C] Typo: 'presicion' should be 'precision'. Also, table headings in the text read 'EVALUATIONRESULTS' and are missing spaces.
  2. [§V.B] The paper states it uses 'a straightforward prompt that only instructs the LLM to conduct consistency checks,' but the proposed pipeline also instructs the LLM to convert to Markdown/JSON and includes estimated header information. Please clarify the full prompt structure and what exactly differs between the two conditions.
  3. [§IV.B, Eq. (1)] The definitions of 'symbols,' 'characters,' and 'nouns (without dictionary information)' are not operationalized. The morphological analyzer and the value of θ_s are not reported, so the format-selection rule is not reproducible.
  4. [§V.C–E] The paper reports only mean precision/recall over 50 runs, with no standard deviations or confidence intervals. A per-pair breakdown would help the reader assess stability.
  5. [§V.C] The criteria for excluding 'errors generally acceptable by convention' from the precision calculation are subjective. The paper should provide the full annotation guideline or have a second annotator and report inter-rater reliability.
  6. [§II.B] The literature review makes strong negative claims ('only 1.29% ...', 'there are no papers on the application of LLMs in design document reviews'). Citing the systematic review's search scope and limitations would make these claims more precise.

Circularity Check

0 steps flagged

No significant circularity: recall improvements are measured outcomes, not constructed from the fitted threshold; the main concern is an experimental confound rather than a definitional reduction.

full rationale

This paper reports an empirical evaluation, not a formal derivation, so the circularity patterns that require an equation or parameter to be equivalent to its outcome do not apply. The central claim is that converting CSV design-document data to Markdown or JSON improves LLM consistency-check recall. The comparison is between an unconverted CSV baseline and a converted format, with recall and precision measured on introduced defects. No fitted parameter is used to predict defect-detection performance. The θ_s threshold in Equation (1) is fitted for format selection, and the RQ2 documents are selected using that threshold, but the reported recalls are independent measurements on both formats for each document class; the result is not forced by construction. The most serious limitation is experimental: the proposed method adds an LLM conversion pass before review, while the baseline skips that pass, so the recall gain may be partly attributable to the extra processing step rather than to the header/value-distinguishing format. That is a validity threat, not a circularity of the type this pass is designed to flag. There are no load-bearing self-citations, no uniqueness theorems imported from the authors, and no ansatz smuggled in via citation. The review perspectives and consistency-check setup come from expert discussion and real documents, not from the outcome being predicted.

Axiom & Free-Parameter Ledger

1 free parameters · 6 axioms · 0 invented entities

The method depends on a threshold θ_s that is fitted to data and never reported; the causal story relies on an unverified assumption about LLM training data; and the evaluation assumes the hand-injected errors and the manual precision classification are representative. These are the items the reader must accept without independent evidence.

free parameters (1)
  • θ_s (threshold for Markdown vs JSON selection) = not reported
    Equation (1) defines the format decision rule; the threshold is chosen to maximize F1 on representative documents and later set from experience, but its value or fitting data are not disclosed.
axioms (6)
  • domain assumption The CSV data used to train LLMs lacks header-value distinction, causing poor table comprehension.
    Invoked in Section IV.A to justify the need for format conversion; not empirically demonstrated in the paper.
  • domain assumption SDEM design documents and the 11 review perspectives derived from expert consultation are a representative basis for generalizable review.
    Section III.A and Table I rely on Fujitsu's internal SDEM standard and expert opinions, with no external validation.
  • domain assumption General-purpose LLMs lack the domain-specific knowledge needed for review levels 3 and 4, so the study can restrict to levels 1 and 2.
    Section III.A states this without direct experimental evidence.
  • ad hoc to paper The POS-based measure P_s (proportion of symbols, characters, and nouns without dictionary info) captures whether a document is natural-language-rich or symbol-rich.
    Proposed in Section IV.B, Equation (1); no validation that this statistic is predictive beyond the small experiment.
  • domain assumption The manually injected inconsistencies in the test documents are representative of real design-document defects.
    Section V.B describes injecting errors such as changed process IDs and text labels; the representativeness is asserted, not measured.
  • domain assumption The manual classification of detected issues as true or false positives is consistent and unbiased.
    Section V.B defines criteria for counting inconsistencies, but no inter-rater reliability or audit is reported.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Development of Automated Software Design Document Review Methods Using Large Language Models." pith.science (2026). https://pith.science/paper/SQRVRYJN

@misc{pith2026250909975,
  author       = {Pith},
  title        = {Pith review of: Development of Automated Software Design Document Review Methods Using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SQRVRYJN}},
  note         = {Machine review of arXiv:2509.09975}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In this study, we explored an approach to automate the review process of software design documents by using LLM. We first analyzed the review methods of design documents and organized 11 review perspectives. Additionally, we analyzed the issues of utilizing LLMs for these 11 review perspectives and determined which perspectives can be reviewed by current general-purpose LLMs instead of humans. For the reviewable perspectives, we specifically developed new techniques to enable LLMs to comprehend complex design documents that include table data. For evaluation, we conducted experiments using GPT to assess the consistency of design items and descriptions across different design documents in the design process used in actual business operations. Our results confirmed that LLMs can be utilized to identify inconsistencies in software design documents during the review process.

Figures

Figures reproduced from arXiv: 2509.09975 by Keisuke Miyazaki, Susumu Tokumoto, Takao Nakagawa, Takasaburo Fukuda.

Figure 1
Figure 1. Figure 1: Illustration of LLM Utilization in the Review Process. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Example of Usage in Ambiguity Check. out of the 47 standard design documents in the architectural design phase use table formats. There are also design docu￾ments that use images for screen layouts and process flows. However, reading such documents would require additional multimodal support, which is beyond the scope of this study [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example of a Design Document in the Architectural Design Phase of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of the Proposed Method: Converting CSV Input to a Format [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example of Converted Design Document in Markdown Format for [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example of Nested Structure Misalignments and Missed Headers and [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Example of Correct Header and Value Distinction in JSON Format [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Example of Simple Pipe Expression Conversion in Markdown Format [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Demo Screen of Internal Review Tool. suggesting a viable foundation for practical use even at this stage. VII. CONCLUSION In this study, we aimed to reduce the cost and improve the quality of design document reviews by automating the review process using Large Language Models (LLMs). Initially, we categorized review perspectives based on their difficulty level, distinguishing between areas where general-p… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 11 linked inside Pith

  1. [1]

    In NeurIPS (2017)

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin, Attention is all you need. In NeurIPS (2017)

    2017

  2. [2]

    arXiv preprint arXiv:1810.04805 (2018)

    Jacob Devlin, Ming- Wei Chang, Kenton Lee, Kristina Toutanova, Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805 (2018)

    Pith/arXiv arXiv 2018

  3. [3]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, etal. Language models are few-shot learners. In Advances in Neural Information Processing Systems (2020)

    2020

  4. [4]

    Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models

    Priyan Vaithilingam, Tianyi Zhang, Elena L Glassman, Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI Conferenceon Human Factors in Computing Systems Extended Abstracts. ACM, NY NY , USA, pp. 1–7 (2022)

    2022

  5. [5]

    Using LLMs in Software Requirements Specifications: An Empirical Evalua- tion

    Madhava Krishna, Bhagesh Gaur, Arsh Verma, Pankaj Jalote. Using LLMs in Software Requirements Specifications: An Empirical Evalua- tion. arXiv preprint arXiv:2404.17842 (2024)

    Pith/arXiv arXiv 2024

  6. [6]

    A Comparative Analysis of Large Language Models for Code Documentation Generation

    Shubhang Shekhar Dvivedi, Vyshnav Vijay, Sai Leela Rahul Pujari, Shoumik Lodh, and Dhruv Kumar. A Comparative Analysis of Large Language Models for Code Documentation Generation. arXiv preprint arXiv:2312.10349 (2024)

    Pith/arXiv arXiv 2024

  7. [7]

    The 37th Annual Conference of the Japanese Society for Artificial Intelligence (2023)

    Yuta Koreeda, Terufumi Morishita, Osamu Imaichi, Yasuhiro Sogawa, Generating Readme with Heuristics-Augmented Large Language Mod- els. The 37th Annual Conference of the Japanese Society for Artificial Intelligence (2023)

    2023

  8. [8]

    and Wilson, G., eds.)

    Perry, D.: Where do Most Software Flaws Come from?, Making Soft- ware:What Really Works, and Why We Believe It (Oram, A. and Wilson, G., eds.). O’Reilly Media, Chapter25, pp.453-494 (2010)

    2010

  9. [9]

    Journal of Information Processing, vol

    Takasaburo Fukuda, Kentaro Murase, Satoru Nakamura, Kazuo Hiekata, Yoshiaki Oida, Shigeru Matsumoto, Isaku Okada, Development of Impact Analysis System for Specification Change of System with Design Information and Process Dependency. Journal of Information Processing, vol. 59, no. 4, pp. 1240–1249 (2018)

    2018

  10. [10]

    ProMAC 2013 (2013)

    Atsushi Motoyama, Improvement in quality of design specifications by detecting inconsistencies between design items. ProMAC 2013 (2013)

    2013

  11. [11]

    Joint Study

    NTT Com Research and President Inc. Joint Study. ”Survey on Doc- ument Creation by Business People.” NTT Com Research Database. Available at: https://research.nttcoms.com/database/data/001928/ (ac- cessed 2024)

    2024

  12. [12]

    1028–2008 IEEE standard for software reviews and audits

    IEEE. 1028–2008 IEEE standard for software reviews and audits. (2008)

    2008

  13. [13]

    ISO/IEC 20246:2017 Software and systems engineering – Work product reviews (2017)

    2017

  14. [14]

    IEEE Transactions on Software Engineering, 22(12):866-874 (1996)

    Yuri Chernak, A Statistical Approach To The Inspection Checklist Formal Synthesis And Improvement. IEEE Transactions on Software Engineering, 22(12):866-874 (1996)

    1996

  15. [15]

    Basili, Evolving And Packaging Reading Technologies

    Victor R. Basili, Evolving And Packaging Reading Technologies. Journal of Systems and Software38(1), 3-12 (1997)

    1997

  16. [16]

    arXiv preprint arXiv:2308.10620 (2023)

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang, Large language models for software engineering: A systematic literature review. arXiv preprint arXiv:2308.10620 (2023)

    Pith/arXiv arXiv 2023

  17. [17]

    Automated Software Engineering, 30.1: 13, (2023)

    Kristian Kolthoff, Christian Bartelt, and Simone Paolo Ponzetto, Data- driven prototyping via natural-language-based GUI retrieval. Automated Software Engineering, 30.1: 13, (2023)

    2023

  18. [18]

    arXiv preprint arXiv:2304.09181 (2023)

    Shantanu Mandal, Adhrik Chethan, Vahid Janfaza, SM Mahmud, Todd A Anderson, Javier Turek, Jesmin Jahan Tithi, and Abdullah Muzahid, Large Language Models Based Automatic Synthesis of Software Spec- ifications. arXiv preprint arXiv:2304.09181 (2023)

    Pith/arXiv arXiv 2023

  19. [19]

    arXiv preprint arXiv:2303.07839 (2023)

    Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith, and Dou- glas C Schmidt, Chatgpt prompt patterns for improving code quality, refactoring, requirements elicitation, and software design. arXiv preprint arXiv:2303.07839 (2023)

    Pith/arXiv arXiv 2023

  20. [20]

    Zhang, Large Language Models for Software Engineering: Survey and Open Problems

    Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, Jie M. Zhang, Large Language Models for Software Engineering: Survey and Open Problems. arXiv preprint arXiv:2310.03533 (2023)

    Pith/arXiv arXiv 2023

  21. [21]

    On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML

    Javier C ´amara, Javier Troya, Lola Burgue˜no, and Antonio Vallecillo. On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML. Softw. Syst. Model. 22, 3, 781–793, (2023)

    2023

  22. [22]

    Software Engineering Symposium 2023 (SES2023), 238-239, (2023)

    Hideyuki Kanuka, Genta Koreki, Ryo Soga, Kazu Nishikawa, An experiment for applying ChatGPT to bidirectional traceability problem between design model and code. Software Engineering Symposium 2023 (SES2023), 238-239, (2023)

    2023

  23. [23]

    arXiv preprint arXiv:2308.00352, (2023)

    Sirui Hong and Xiawu Zheng and Jonathan Chen and Yuheng Cheng and Jinlin Wang and Ceyao Zhang and Zili Wang and Steven Ka Shing Yau and Zijuan Lin and Liyang Zhou and Chenyu Ran and Lingfeng Xiao and Chenglin Wu, Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, (2023)

    Pith/arXiv arXiv 2023

  24. [24]

    arXiv preprint arXiv:2404.17739 (2024)

    Beian Wang, Chong Wang, Peng Liang, Bing Li, Cheng Zeng, How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts. arXiv preprint arXiv:2404.17739 (2024)

    Pith/arXiv arXiv 2024

  25. [25]

    arXiv preprint arXiv:2404.14370 (2024)

    G ´abor Antal, Rich ´ard V oz ´ar, Rudolf Ferenc, Assessing GPT-4- Vision’s Capabilities in UML-Based Code Generation. arXiv preprint arXiv:2404.14370 (2024)

    Pith/arXiv arXiv 2024

  26. [26]

    Information-technology Promotion Agency, Japan (IPA), Software De- velopment Analysis Data Collection 2022 (Financial and Insurance Industries Edition) (2022)

    2022

  27. [27]

    Liang, Chenyang Yang, Brad A

    Jenny T. Liang, Chenyang Yang, Brad A. Myers, A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Chal- lenges, arXiv preprint arXiv:2308.17125, (2023)

    arXiv 2023

  28. [28]

    Proceedings of the 2002 International Symposium on Empirical Soft- ware Engineering, IEEE Computer Society, (2002)

    Sabaliauskaite Giedre, Fumikazu, Matsukawa, Shinji, Kusumoto, Kat- suro, Inoue, An Experimental Comparison of Checklist-Based Reading and Perspective-Based Reading for UML Design Document Inspection. Proceedings of the 2002 International Symposium on Empirical Soft- ware Engineering, IEEE Computer Society, (2002)

    2002

  29. [29]

    Fujitsu Journal, 63.2, 193-199 (2012)

    Kenji Muronaka, Naoaki Hara, Standard Processes for System Planning, Development, and Operation and Maintenance: SDEM. Fujitsu Journal, 63.2, 193-199 (2012)

    2012

  30. [30]

    The Proceedings of Design and Systems Conference (2014)

    Kazuo Hiekata, Hiroyuki Yamato, Naoto Fukuda, Satoru Nakamura, Isaac Okada, Minoru Saito, Yoshiaki Oida, Ikuo Watanabe, Shigeru Matsumoto, Development of Impact Analysis System for Specification Change of System. The Proceedings of Design and Systems Conference (2014)

    2014

  31. [31]

    arXiv preprint arXiv:2210.06710 (2022)

    Wenhu Chen, Large Language Models are few(1)-shot Table Reasoners. arXiv preprint arXiv:2210.06710 (2022)

    Pith/arXiv arXiv 2022

  32. [32]

    Grappa: Grammar-augmented pre-training for table semantic parsing

    Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir R Radev, Richard Socher, Caiming Xiong. Grappa: Grammar-augmented pre-training for table semantic parsing. InICLR (2021)

    2021

  33. [33]

    In Advances in Neural Information Processing Systems (2022)

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoc V Le, Denny Zhou, Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (2022)

    2022

  34. [34]

    Atsushi Motoyama, Takako Nakatani, A metamodel for design review derived from design specification templates. JCKBSE 2012, 108-117 (2012.) APPENDIX TABLE A.1 REVIEWPERSPECTIVES ANDTHEIRDESCRIPTION Review Perspective Description Sufficiency Check Whether it follows the design standards set by the project Standard/Regulation Check Whether it follows the dev...

    2012

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.