Pith. sign in

REVIEW 3 major objections 6 minor 40 references

An Empirical Study on the Capability of LLMs in Decomposing Bug Reports

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that large language models can decompose bug reports only when given well-chosen examples: adding three demonstrations raised correct decompositions from 10 to 24 (ChatGPT) and 11 to 29 (DeepSeek) out of 127 reports.

desk verdict First useful benchmark for LLM bug decomposition, undermined by subjective labels and likely few-shot contamination. read the letter →

arxiv 2504.20911 v1 pith:QDQRPI5D submitted 2025-04-29 cs.SE

Zhiyuan Chen , Vanessa Nava-Camal , Ahmad Suleiman , Yiming Tang , Daqing Hou , Weiyi Shang This is my paper · ORCID
classification cs.SE
keywords bugreportdecompositionlargelanguagemodelsfew-shotpromptingzero-shotpromptengineeringempiricalstudyprivacy-relatedreports
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to test whether large language models can take a single complex bug report—one that often mixes several distinct issues, code, logs, and suggested fixes—and split it into smaller, self-contained bugs that could be assigned separately. Using 127 resolved privacy-related bug reports from a public open-source bug tracker, the authors find that both ChatGPT and DeepSeek do this poorly with a plain zero-shot prompt: only 10 and 11 reports are decomposed correctly. Adding three hand-picked worked examples to the prompt lifts the correct counts to 24 and 29, increases of 140% and 163.64%. The conclusion is that LLMs are usable for bug decomposition only with careful prompt design and human verification, and that over-decomposition is the most common failure mode.

What carries the argument

The load-bearing mechanism is the few-shot prompt: a small set of three hand-crafted decomposition demonstrations inserted into the prompt, one for a straightforward report, one for a report containing code or logs, and one for a report with a suggested solution. These demonstrations are intended to teach the model the output format and, more importantly, to steer it around the failure modes found in the zero-shot round—especially the instruction not to analyze code or to treat solutions as separate bugs. The argument also rests on a manual annotation pipeline in which two reviewers independently judged each decomposition by whether the number of units was correct and whether each unit's description was accurate, with a third reviewer mediating disagreements.

What would settle it

Run the identical zero-shot and few-shot prompts on a fresh, independent sample of several hundred bug reports from other domains, with gold decompositions produced by independent annotators and inter-rater agreement reported. If correct-decomposition rates do not rise from about 8-9% to about 19-23%, or if the labels themselves show near-chance agreement, the claim that few-shot prompting improves LLM bug decomposition would be unsupported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that prompt quality, not model capability alone, determines whether an LLM can decompose bug reports. With zero-shot instructions, ChatGPT correctly decomposed 10 of 127 reports and DeepSeek 11; after a few-shot prompt containing three curated examples, ChatGPT reached 24 and DeepSeek 29. The authors attribute the failures to four recurring behaviors: over-decomposition (splitting one bug into several versions of the same issue), over-analysis (turning logs and code into separate sub-bugs), incorrect interpretation of solutions (treating a suggested fix as a bug), and lacking key information (omitting links, code, or logs). The paper's central claim is that LLMs show genuine potential for decomposition, but are not yet reliable enough to automate it, and that their output must be checked by a human.

Load-bearing premise

The results stand on the assumption that the two reviewers' judgment of the correct number and wording of decomposed bugs is an objective ground truth, yet the paper reports no inter-rater agreement metric and the few-shot examples come from the same corpus and were manually enhanced by the authors.

Editorial extensions

If this is right

  • Tool builders cannot rely on zero-shot LLM calls for bug decomposition; a few demonstrations are a cheap but large accuracy lever, roughly tripling correct outputs in this dataset.
  • Over-decomposition is the dominant error, so practical systems should add a verification step that asks the model whether two units describe the same underlying issue before accepting a split.
  • LLM decomposition is safe to use only as a triage aid with human review, not as a fully automated pipeline.
  • Reporters can improve automated analysis by filing atomic, well-structured, concise reports, as the paper's discussion recommends.
  • Because correctly decomposed reports had shorter resolution times, decomposition success may serve as a rough indicator of bug complexity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The few-shot demonstrations were selected from the same 127-report corpus and manually enhanced by the authors, so the measured 140% and 163.64% gains may overstate what a user would see with demonstrations drawn from a separate source; a held-out demonstration set would give a cleaner estimate.
  • A natural next experiment is to add a second-pass consistency check in which the model must map each decomposed unit back to a distinct sentence or paragraph of the original report; this could directly attack the dominant over-decomposition failure.
  • The finding that easier-to-resolve bugs are more likely to be decomposed correctly suggests LLM decomposition quality could be used as a proxy for bug complexity in triage, though the paper does not test this use.
  • Adding an inter-rater agreement measure in future work would make the ground-truth labels more transparent, since the current evaluation relies on two reviewers' judgment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper reports an empirical study of 127 privacy-related bug reports from Apache Jira, evaluating ChatGPT (GPT-4o) and DeepSeek (DeepSeek-V1) on the task of decomposing bug reports into smaller, self-contained units. The authors use zero-shot prompting for RQ1 and few-shot prompting with three hand-crafted demonstrations for RQ2, and manually label each model output as a correct or incorrect decomposition. They report that the true decomposition rate rose from 10 to 24 for ChatGPT and from 11 to 29 for DeepSeek after few-shot prompting, and they identify four failure modes: over-decomposition, over-analysis, incorrect interpretation of solutions, and lacking key information. The paper also includes a preliminary question on resolution time and priority levels, and a discussion section offering recommendations for bug reporters.

Significance. If the claims are supported, the study would be a useful first step in understanding LLM-based bug report decomposition, an under-explored task with practical relevance for issue triage and resolution. The error taxonomy (over-decomposition, over-analysis, incorrect solution interpretation, missing key information) is a plausible and potentially reusable contribution, and the paper provides a replication package. The improvement from few-shot prompting is dramatic and, if valid, would support the general conclusion that prompt quality matters for this task. However, the central quantitative claims currently rest on a manual labeling process with no reported inter-rater reliability, and on few-shot demonstrations that may not be held out from the evaluation set. Both issues must be resolved before the findings can be accepted as evidence.

major comments (3)
  1. [§IV RQ2 (demonstration selection)] The three few-shot demonstrations in Tables XI–XIII are described as based on 'prior LLM results and our manual enhancements,' but the paper never states that these bug reports were excluded from the 127-report evaluation set. The examples shown (Kafka website, generated stub, CouchDB) appear consistent with the privacy-related Apache Jira corpus. If any demonstration report is also in the evaluation set, that test instance is given a near-gold input-output pair in the prompt, making its correct decomposition trivial. More broadly, demonstrations drawn from the same corpus allow the few-shot condition to encode the authors' preferred output format on the evaluation distribution itself, inflating the reported gains (140% for ChatGPT and 163.64% for DeepSeek) and undermining the conclusion that performance 'strongly depends on the quality of the prompts.' The paper must either explicitly confirm that the demonstrations were excluded from the 127 reports, or re-run RQ2 with held-out demonstrations.
  2. [§IV RQ1/RQ2 and §VI (manual analysis)] The correctness labels are produced by the authors' manual review with no inter-rater reliability metric such as Cohen's kappa, despite the statement that two reviewers independently analyzed the results. Because the entire quantitative comparison (10 vs. 24 and 11 vs. 29) is based on these binary subjective judgments about the 'correct' number of decomposed bugs and the accuracy of each description, the central numbers are not verifiable without an agreement measure. Additionally, in Section D3 the paper claims the difference in resolution times between correctly and incorrectly decomposed reports is 'statistically significant,' but no statistical test, effect size, or p-value is reported anywhere in the manuscript. Either add the missing analysis or remove the significance claim.
  3. [Table XIV] Table XIV contains internal arithmetic inconsistencies that affect the reported results. For ChatGPT, the false-case counts sum to 103 (56 over-decomposition + 28 over-analysis + 18 incorrect solution interpretation + 1 lacking key information), but the table reports an FP total of 101. For DeepSeek, the counts sum to 93 (52 + 25 + 12 + 4), but the table reports 98. Since the 'promotion' percentages in Table XVII and the error-distribution discussion in Section IV rely on these totals, they must be recomputed and corrected.
minor comments (6)
  1. [§III-B] The dataset collection description says the study 'focused only on resolved bug reports,' but then adds reports 'not explicitly labeled as resolved' whose status is 'fixed.' The paper should clarify whether 'fixed' is a subset of 'resolved' and how this affects the resolution-time analysis in PQ1.
  2. [Table XVII] The column header 'Promotion (%)' is ambiguous: for the False rows, the values 88.03% and 84.48% represent the remaining fraction of false cases, not a promotion. A clearer label such as 'Percentage of RQ1 count' is needed.
  3. [Abstract and §IV] The abstract says 'ChatGPT' while Section IV RQ1 states the model is GPT-4o; the paper should consistently identify the exact model versions and access dates for both ChatGPT and DeepSeek, since model updates are a known source of variability.
  4. [§IV RQ1] The summary paragraph before the RQ2 approach is labeled 'Summary of RQ2' but reports RQ1 results; this appears to be a typographical error and should be fixed.
  5. [Replication package] The replication package link is a bare GitHub URL with no version or archival DOI; providing a versioned or archived snapshot would improve reproducibility and long-term accessibility.
  6. [Tables VI–X] Several illustrative tables have minor formatting issues (e.g., missing spaces in code snippets, inconsistent punctuation in decomposition outputs), and the paper should state whether these outputs are verbatim model transcripts or lightly edited for space.

Circularity Check

1 steps flagged · score 4.0 of 10

Few-shot demonstrations are built from the same corpus and manually enhanced with gold decompositions, partially conditioning the RQ2 improvement.

  1. fitted input called prediction [Section IV, RQ2 Approach (demonstration selection)]
    "Eventually, we selected three bug decomposition examples as demonstrations. The decomposed bug results are based on prior LLM results and our manual enhancements."

    The few-shot demonstrations are not independent of the evaluation. 'Prior LLM results' are the RQ1 outputs produced on the same 127-report corpus, and the authors 'manually enhance' them to their gold-standard decompositions. These gold-conditioned examples are then placed inside the few-shot prompt whose outputs are scored against the same gold standard. The RQ2 true-decomposition increase (ChatGPT 10 to 24; DeepSeek 11 to 29) is therefore partly a measure of how well the model copies the authors' preferred output format from in-context gold examples, not a clean prediction on unseen reports. If any demonstration report is itself among the 127, that test instance receives a complete gold input-output pair in the prompt.

full rationale

This paper has no mathematical derivation chain; it is an empirical evaluation, so most of the traditional circularity patterns (self-definitional equations, imported uniqueness theorems, ansatz-smuggling citations) do not apply. The zero-shot RQ1 results (10/127 and 11/127) and the qualitative error taxonomy are self-contained, externally observable, and do not reduce to their inputs. There is no load-bearing self-citation: the reference list contains no prior work by the same authors, and no uniqueness or existence proof is invoked. The main concern is localized to RQ2: the few-shot demonstrations are selected from prior RQ1 outputs on the same 127-report corpus and manually enhanced to match the authors' gold-standard decompositions. The reported few-shot improvement is therefore partially forced by construction: the prompt itself encodes the target labeling style, and if any demonstration corresponds to a scored report, that instance is trivially correct. The paper does not disclose whether the three examples were held out, and its Threats to Validity section discusses manual subjectivity and model variability but not this contamination. The central claim retains independent content from RQ1 and the error analysis, so the score is moderate rather than maximal.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The study introduces no free parameters and no invented entities. Its claims rest on the validity of manual labels, the representativeness of the bug corpus, and the selection of few-shot demonstrations, all of which are subjective or domain-specific assumptions.

assumptions (3)
  • domain assumption Manual two-reviewer labels constitute a valid ground truth for correct bug decomposition.
    The entire performance metric depends on the authors' subjective interpretation of what counts as a correct decomposition, and no inter-rater agreement statistic is reported.
  • domain assumption The 127 privacy-related Apache Jira bug reports are representative enough to support general claims about LLM bug decomposition.
    The authors acknowledge in Section VI that the restricted domain limits generalizability, yet they draw general conclusions about LLM capabilities.
  • ad hoc to paper The three few-shot demonstrations are correct and representative examples of successful decomposition.
    The demonstrations were selected based on prior LLM results and manually enhanced by the authors (Section IV RQ2), so they encode the authors' preferred output format rather than an externally validated standard.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Empirical Study on the Capability of LLMs in Decomposing Bug Reports." pith.science (2026). https://pith.science/paper/QDQRPI5D

@misc{pith2026250420911,
  author       = {Pith},
  title        = {Pith review of: An Empirical Study on the Capability of LLMs in Decomposing Bug Reports},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QDQRPI5D}},
  note         = {Machine review of arXiv:2504.20911}
}
read the original abstract

Background: Bug reports are essential to the software development life cycle. They help developers track and resolve issues, but are often difficult to process due to their complexity, which can delay resolution and affect software quality. Aims: This study investigates whether large language models (LLMs) can assist developers in automatically decomposing complex bug reports into smaller, self-contained units, making them easier to understand and address. Method: We conducted an empirical study on 127 resolved privacy-related bug reports collected from Apache Jira. We evaluated ChatGPT and DeepSeek using different prompting strategies. We first tested both LLMs with zero-shot prompts, then applied improved prompts with demonstrations (using few-shot prompting) to measure their abilities in bug decomposition. Results: Our findings show that LLMs are capable of decomposing bug reports, but their overall performance still requires further improvement and strongly depends on the quality of the prompts. With zero-shot prompts, both studied LLMs (ChatGPT and DeepSeek) performed poorly. After prompt tuning, ChatGPT's true decomposition rate increased by 140\% and DeepSeek's by 163.64\%. Conclusions: LLMs show potential in helping developers analyze and decompose complex bug reports, but they still need improvement in terms of accuracy and bug understanding.

Figures

Figures reproduced from arXiv: 2504.20911 by the authors.

Figure 1
Figure 1. Workflow of studying decomposition capability of LLMs. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 20 canonical work pages

  1. [1]

    Make your llm fully utilize the context

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. Make your llm fully utilize the context. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems , volume 37, pages 62160– 62188. Curran Associates, Inc., 2024. URL https: //proceedings...

    work page 2024

  2. [2]

    From text to source: Results in detecting large language model- generated content, 2024

    Wissam Antoun, Beno ˆıt Sagot, and Djam´e Seddah. From text to source: Results in detecting large language model- generated content, 2024. URL https://arxiv.org/abs/2309. 13322

    work page 2024

  3. [3]

    John Anvik, Lyndon Hiew, and Gail C. Murphy. Who should fix this bug? In Proceedings of the 28th Interna- tional Conference on Software Engineering , ICSE ’06, page 361–370, New York, NY , USA, 2006. Association for Computing Machinery. ISBN 1595933751. doi: 10.1145/1134285.1134336. URL https://doi.org/10.1145/ 1134285.1134336

    arXiv 2006

  4. [4]

    Quality of bug reports in eclipse

    Nicolas Bettenburg, Sascha Just, Adrian Schr ¨oter, Cathrin Weiß, Rahul Premraj, and Thomas Zimmermann. Quality of bug reports in eclipse. In Proceedings of the 2007 OOPSLA Workshop on Eclipse Technology EX- change, eclipse ’07, page 21–25, New York, NY , USA,

    work page 2007

  5. [5]

    Nicolas Bettenburg, Sascha Just, Adrian Schr ¨oter, Cathrin Weiss, Rahul Premraj, and Thomas Zimmer- mann. What makes a good bug report? In Proceed- ings of the 16th ACM SIGSOFT International Sym- posium on Foundations of Software Engineering , SIG- SOFT ’08/FSE-16, page 308–318, New York, NY , USA,

    work page

  6. [6]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christo- pher Hesse, Mark Chen, Eric Sigler, Mateusz Lit...

    arXiv 2020

  7. [7]

    Not all bugs are the same: Under- standing, characterizing, and classifying bug types

    Gemma Catolino, Fabio Palomba, Andy Zaidman, and Filomena Ferrucci. Not all bugs are the same: Under- standing, characterizing, and classifying bug types. J. Syst. Softw. , 152:165–181, 2019. doi: 10.1016/J.JSS. 2019.03.002. URL https://doi.org/10.1016/j.jss.2019.03. 002

    doi:10.1016/j.jss 2019

  8. [9]

    What’s in a bug report? In Proceedings of the 8th ACM/IEEE Interna- tional Symposium on Empirical Software Engineering and Measurement , ESEM ’14, New York, NY , USA,

    Steven Davies and Marc Roper. What’s in a bug report? In Proceedings of the 8th ACM/IEEE Interna- tional Symposium on Empirical Software Engineering and Measurement , ESEM ’14, New York, NY , USA,

    work page

Show all 40 references
  1. [10]

    Carlos D. A. de Almeida, Diego N. Feij ´o, and Lin- coln S. Rocha. Studying the impact of continuous delivery adoption on bug-fixing time in apache’s open- source projects. In Proceedings of the 19th International Conference on Mining Software Repositories , MSR ’22, page 132–...

    2022

  2. [11]

    Clement, Guillermo Serrato, and Neel Sundaresan

    Dawn Drain, Colin B. Clement, Guillermo Serrato, and Neel Sundaresan. Deepdebug: Fixing python bugs using stack traces, backtranslation, and code skeletons, 2021. URL https://arxiv.org/abs/2105.09352

    2021 arXiv

  3. [12]

    Guo, Thomas Zimmermann, Nachiappan Na- gappan, and Brendan Murphy

    Philip J. Guo, Thomas Zimmermann, Nachiappan Na- gappan, and Brendan Murphy. ”not my bug!” and other reasons for software bug report reassignments. In Proceedings of the ACM 2011 Conference on Com- puter Supported Cooperative Work , CSCW ’11, page 395–404, New York, NY , USA, ...

    2011

  4. [13]

    German, Jesus M

    Israel Herraiz, Daniel M. German, Jesus M. Gonzalez- Barahona, and Gregorio Robles. Towards a simplification of the bug report form in eclipse. In Proceedings of the 2008 International Working Conference on Mining Soft- ware Repositories, MSR ’08, page 145–148, New York, NY , ...

    2008

  5. [14]

    Large language models for software engineering: A systematic literature review

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. Large language models for software engineering: A systematic literature review. ACM Trans. Softw. Eng. Methodol. , 33(8), December 2024. ISSN 1049-331X. doi: 10.1...

    2024 doi

  6. [15]

    Software maintenance implications on cost and schedule

    Bob Hunt, Bryn Turner, and Karen McRitchie. Software maintenance implications on cost and schedule. In 2008 IEEE Aerospace Conference , pages 1–6, 2008. doi: 10. 1109/AERO.2008.4526688

    2008

  7. [16]

    Uncovering the causes of emotions in soft- ware developer communication using zero-shot llms

    Mia Mohammad Imran, Preetha Chatterjee, and Kostadin Damevski. Uncovering the causes of emotions in soft- ware developer communication using zero-shot llms. In Proceedings of the IEEE/ACM 46th International Con- ference on Software Engineering , ICSE ’24, New York, NY , USA, 2...

    2024

  8. [17]

    Jira software catagories

    Apache Jira. Jira software catagories. URL https://issues.apache.org/jira/secure/BrowseProjects. jspa?selectedCategory=all&selectedProjectType= software&sortColumn=name&sortOrder=ascending&s= view projects&page=1

  9. [18]

    A survey of gpt-3 family large language models including chatgpt and gpt- 4, 2023

    Katikapalli Subramanyam Kalyan. A survey of gpt-3 family large language models including chatgpt and gpt- 4, 2023. URL https://arxiv.org/abs/2310.12321

    2023 arXiv

  10. [19]

    Automatic unsupervised bug report categorization

    Nachai Limsettho, Hideaki Hata, Akito Monden, and Kenichi Matsumoto. Automatic unsupervised bug report categorization. In 2014 6th International Workshop on Empirical Software Engineering in Practice, pages 7–12,

    2014

  11. [20]

    Summary of chatgpt- related research and perspective towards the future of large language models

    Yiheng Liu, Tianle Han, Siyuan Ma, Jiayue Zhang, Yuanyuan Yang, Jiaming Tian, Hao He, Antong Li, Mengshen He, Zhengliang Liu, Zihao Wu, Lin Zhao, Dajiang Zhu, Xiang Li, Ning Qiang, Dingang Shen, Tianming Liu, and Bao Ge. Summary of chatgpt- related research and perspective tow...

    2023

  12. [21]

    Ausum: approach for unsupervised bug report summarization

    Senthil Mani, Rose Catherine, Vibha Singhal Sinha, and Avinava Dubey. Ausum: approach for unsupervised bug report summarization. In Proceedings of the ACM SIG- SOFT 20th International Symposium on the Foundations of Software Engineering, FSE ’12, New York, NY , USA,

  13. [22]

    Murphy, and Gabriel Murray

    Sarah Rastkar, Gail C. Murphy, and Gabriel Murray. Automatic summarization of bug reports. IEEE Trans- actions on Software Engineering , 40(4):366–380, 2014. doi: 10.1109/TSE.2013.2297712

    2014

  14. [23]

    doi: 10.1109/IWESEP.2014.8

    2014 doi

  15. [24]

    Correlation coefficients: appropriate use and interpreta- tion

    Patrick Schober, Christa Boer, and Lothar A Schwarte. Correlation coefficients: appropriate use and interpreta- tion. Anesthesia & analgesia , 126(5):1763–1768, 2018

    2018

  16. [25]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,

  17. [26]

    The significance of bug report elements

    Mozhan Soltani, Felienne Hermans, and Thomas B ¨ack. The significance of bug report elements. Empir. Softw. Eng. , 25(6):5255–5294, 2020. doi: 10.1007/ S10664-020-09882-Z. URL https://doi.org/10.1007/ s10664-020-09882-z

    2020

  18. [27]

    Improved du- plicate bug report identification

    Yuan Tian, Chengnian Sun, and David Lo. Improved du- plicate bug report identification. In 2012 16th European Conference on Software Maintenance and Reengineer- ing, pages 385–390, 2012. doi: 10.1109/CSMR.2012.48

    2012 doi

  19. [28]

    Control-flow based anomaly detection in the bug-fixing process of open-source projects

    Veena Saini, Paramvir Singh, and Ashish Sureka. Control-flow based anomaly detection in the bug-fixing process of open-source projects. In Proceedings of the 13th Innovations in Software Engineering Conference (Formerly Known as India Software Engineering Confer- ence), ISEC ’...

    2020

  20. [29]

    Chain-of-thought prompting elicits rea- soning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits rea- soning in large language models, 2023. URL https: //arxiv.org/abs/2201.11903

    2023 arXiv

  21. [30]

    Chatgpt prompt patterns for improving code quality, refactoring, requirements elicitation, and software design

    Jules White, Sam Hays, Quchen Fu, Jesse Spencer- Smith, and Douglas C Schmidt. Chatgpt prompt patterns for improving code quality, refactoring, requirements elicitation, and software design. In Generative ai for effective software development , pages 71–108. Springer, 2024

    2024

  22. [31]

    Using spearman’s correlation coef- ficients for exploratory data analysis on big dataset

    Chengwei Xiao, Jiaqi Ye, Rui M ´aximo Esteves, and Chunming Rong. Using spearman’s correlation coef- ficients for exploratory data analysis on big dataset. Concurrency and Computation: Practice and Experience, 28(14):3866–3878, 2016

    2016

  23. [32]

    Deepseek-prover: Advancing theorem proving in llms through large-scale synthetic data, 2024

    Huajian Xin, Daya Guo, Zhihong Shao, Zhizhou Ren, Qihao Zhu, Bo Liu, Chong Ruan, Wenda Li, and Xiaodan Liang. Deepseek-prover: Advancing theorem proving in llms through large-scale synthetic data, 2024. URL https: //arxiv.org/abs/2405.14333

    2024 arXiv

  24. [33]

    A survey on bug-report analysis

    Jie Zhang, Xiaoyin Wang, Dan Hao, Bing Xie, Lu Zhang, and Hong Mei. A survey on bug-report analysis. Sci. China Inf. Sci. , 58(2):1–24, 2015

    2015

  25. [34]

    Hybrid llm/rule-based approaches to business insights generation from structured data, 2024

    Aliaksei Vertsel and Mikhail Rumiantsau. Hybrid llm/rule-based approaches to business insights generation from structured data, 2024. URL https://arxiv.org/abs/ 2404.15604

    2024 arXiv

  26. [35]

    Combining text mining and data mining for bug report classification

    Yu Zhou, Yanxiang Tong, Ruihang Gu, and Harald Gall. Combining text mining and data mining for bug report classification. Journal of Software: Evolution and Process, 28(3):150–176, 2016. 13

    2016

  27. [40]

    Tao Zhang, He Jiang, Xiapu Luo, and Alvin T. S. Chan. A literature review of research in bug resolution: Tasks, challenges and future directions. Comput. J., 59(5):741– 773, 2016. doi: 10.1093/COMJNL/BXV114. URL https: 12 //doi.org/10.1093/comjnl/bxv114

    2016 doi

  28. [2007]

    ISBN 9781605580159

    Association for Computing Machinery. ISBN 9781605580159. doi: 10.1145/1328279.1328284. URL https://doi.org/10.1145/1328279.1328284

  29. [2008]

    ISBN 9781595939951

    Association for Computing Machinery. ISBN 9781595939951. doi: 10.1145/1453101.1453146. URL https://doi.org/10.1145/1453101.1453146

  30. [2012]

    ISBN 9781450316149

    Association for Computing Machinery. ISBN 9781450316149. doi: 10.1145/2393596.2393607. URL https://doi.org/10.1145/2393596.2393607

  31. [2014]

    ISBN 9781450327749

    Association for Computing Machinery. ISBN 9781450327749. doi: 10.1145/2652524.2652541. URL https://doi.org/10.1145/2652524.2652541

  32. [2023]

    URL https://arxiv.org/abs/2303.17580

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.