Pith. sign in

REVIEW 3 major objections 3 minor 24 references

Leveraging Resolved Incident History for LLM-Assisted Software Bug Diagnosis

T0 review · 3 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM bug diagnosis fails when it retrieves design documentation rather than structured records of past failures; OM-RAG stores resolved incidents as symptom–root cause–resolution triples and reaches 0.931 diagnosis accuracy on 1,172 real…

desk verdict The retrieval-target idea is genuinely new and the benchmark is useful, but the ground-truth extraction loop and missing artifacts keep me from trusting the headline numbers. read the letter →

arxiv 2607.21911 v1 pith:JBF5DZMP submitted 2026-07-24 cs.SE cs.DL

Boyuan Guan , Hailu Xu , Jamie Rogers This is my paper · ORCID
classification cs.SEcs.DL
keywords retrieval-augmentedgenerationoperationalmemorybugdiagnosisrootcauseanalysisLLMagentsepisodiccase-basedreasoningself-adaptivesystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to establish that LLM-based software bug diagnosis fails for two independent reasons, both rooted in what the retrieval system brings back. The first is a source mismatch: typical RAG systems retrieve design documentation, which describes how software is supposed to work, rather than records of how it has actually failed. The second is a structure mismatch: even when resolved incidents are retrieved, text chunking breaks the causal chain connecting symptom, root cause, and resolution. The authors propose Operational Memory RAG (OM-RAG), which stores each resolved incident as a structured triple and retrieves the most similar precedent with a single embedding lookup. On 1,172 real issues from the issue tracker of a large open-source research-data platform, OM-RAG achieves 0.931 diagnosis accuracy and 0.809 fix correctness, outperforming chunk-based retrieval by 186% and documentation-anchored graph retrieval by 77%; the paper presents this as evidence that operational memory, not more documentation, is the missing ingredient for diagnostic RAG.

What carries the argument

Operational Memory RAG (OM-RAG) is a four-phase pipeline whose load-bearing object is the structured diagnostic triple: symptom, root cause, and resolution, extracted from each closed bug report and kept in a flat embedding index. Phase 1 harvests closed bug reports and their comment threads from the system's issue tracker; Phase 2 prompts an LLM to distill each issue into a triple plus a self-assigned quality score, keeping only triples scored at least 3; Phase 3 concatenates symptom and root cause into one string, embeds it with a dense encoder, and stores the records in a flat database; Phase 4 embeds the new issue, retrieves the top-3 most similar triples by cosine similarity, and prompts the LLM to reason explicitly from those precedents. The index deliberately has no graph layer: the paper argues that bug diagnosis is a single-hop lookup problem, so cross-incident graph structure adds path-traversal noise without improving precision, and the ablation's documentation-graph configuration supports that claim by underperforming the flat incident index.

What would settle it

A direct test is to rebuild the knowledge base with triples written by human maintainers from the same 1,172 resolved issues and rerun the four-configuration ablation: if OM-RAG's diagnosis accuracy falls toward the chunk baseline when the index and ground truth no longer share the same LLM extraction, the 0.931 advantage is largely extraction self-consistency. A cheaper model-free check is to take a random sample of retrieved top-1 triples and have a human maintainer confirm that the root cause matches the actual resolution discussion, independent of any LLM judge.

Watch

Extended reading notes

Core claim

The paper's central claim is that the right retrieval target for bug diagnosis is a structured operational case, not a documentation fragment, and that shifting this target alone produces large, consistent, and measurement-stable gains. The demonstration is a four-configuration ablation on 1,172 real bug reports: no retrieval, chunk-based retrieval from the same resolved issues, a documentation-anchored concept graph, and OM-RAG's structured incident triples. OM-RAG dominates the other configurations on every evaluation dimension—diagnosis accuracy, fix correctness, groundedness, and specificity—and within every domain category, with non-overlapping bootstrap confidence intervals; the observed ordering C-zero < C-chunk < C-doc-kg < OM-RAG is the principal result. The two baselines each display one of the two diagnosed failure modes: chunk retrieval is highly grounded but cannot reconstruct the causal chain, and documentation-graph retrieval routes through conceptual detours that dilute operational specificity. The authors conclude that only structured operational triples address both mismatches simultaneously, and they corroborate the LLM-judged scores with cross-judge checks, positional-bias tests, manual review of a 123-issue subset, and model-free retrieval similarity of 0.880.

Load-bearing premise

The load-bearing premise is that the ground-truth triples used for scoring and the knowledge-base triples used for retrieval are both generated by the same LLM extraction pipeline, with the quality filter assigned by the extracting model itself, so the reported accuracy could partly reflect how well retrieval matches the extractor's own representation of root causes rather than how well it matches independently verifiable reality.

Editorial extensions

If this is right

  • If the paper is right, engineering effort for diagnostic agents should go into structuring and indexing resolved incident history, not into building larger document corpora or richer knowledge graphs.
  • A flat single-hop index is sufficient for bug diagnosis on this task; graph-based retrieval adds cost and conceptual indirection without a precision gain.
  • Feeding resolved incidents back into the index turns the Knowledge component of a self-adaptation loop into an accumulating episodic memory, so a deployed agent improves as it operates.
  • The decomposed four-dimension scoring rubric provides a reusable template for evaluating diagnostic RAG systems beyond this benchmark.
  • Operational-memory RAG is economically viable at production scale: the full 1,172-issue evaluation cost about $65, or $0.056 per issue, including inference and judging.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the result generalizes, the most valuable data for coding and operations assistants is the trail of past failures that organizations already keep but rarely structure; the paper's single-system design leaves that generality unproven.
  • A natural testable extension is to apply the same triple-extraction and flat-index recipe to cloud incident postmortems or issue trackers of other open-source systems and check whether the ordering C-zero < C-chunk < C-doc-kg < OM-RAG reproduces.
  • Because extraction and evaluation share one LLM, a stronger design would decouple them—human-authored root causes for a held-out subset and a different model family for index construction—to isolate the retrieval-structure contribution from extraction bias.
  • The 'retrieved but wrong' framing suggests a broader design principle for RAG: knowledge stores for diagnostic tasks should be episodic and case-shaped rather than semantic and rule-shaped, which may transfer to technical-support and clinical RAG where past cases are authoritative.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. This paper introduces OM-RAG, a retrieval-augmented generation system that indexes resolved GitHub issues as structured symptom–root-cause–resolution triples and retrieves top-k precedents via single-hop embedding to help an LLM diagnose software bugs. The authors evaluate OM-RAG against three baselines (no retrieval, chunk-based retrieval from the same issues, and a documentation-anchored concept graph) on 1,172 issues from the IQSS/dataverse repository. Using claude-sonnet-4-6 as an LLM judge, they report that OM-RAG achieves Diagnosis Accuracy 0.931 and Fix Correctness 0.809, with the ordering C-zero < C-chunk < C-doc-kg < OM-RAG across all four rubric dimensions and all domain categories. Additional analyses include bootstrap CIs, a 23-issue cross-judge/positional-bias study, a 123-issue manual review, and model-free retrieval metrics.

Significance. If the reported effect is real, the paper makes a useful contribution: it demonstrates a concrete instantiation of the 'operational memory' idea, with a production deployment and a public benchmark. The design of the ablation (C-chunk vs OM-RAG isolating structure, C-doc-kg vs OM-RAG isolating source) is clean, and the bootstrap CIs and cross-judge agreement lend quantitative support. However, the central claim is weakened by the fact that the ground-truth triples and the OM-RAG knowledge base are generated by the same LLM extraction pipeline, a concern that the manual verification does not address. The model-free metrics, as reported, do not corroborate the ranking because they are only computed for OM-RAG.

major comments (3)
  1. [Sections III-B and IV-B] The ground-truth triples used to compute Diagnosis Accuracy and Fix Correctness and the OM-RAG knowledge base are both produced by the same gpt-4o-mini extraction step, with the quality_score >= 3 filter assigned by the extracting model itself. This links the scoring target to the retrieved evidence in vocabulary, schema, and systematic extraction bias. Because the judge (claude-sonnet-4-6) is asked to compare the generated root cause against the extracted ground-truth root_cause, OM-RAG can be rewarded for reproducing the extractor's canonical phrasing of a root cause, while C-chunk and C-doc-kg are disadvantaged by presenting information in a less normalized form. The manual verification in Section IV-F checks judge decisions against the extracted root cause, not the extraction against the original issue threads, so it does not break this loop. To support the central ranking claim, the authors should either (a) have the ground-truth triples validated by human annotation on a sample of issues before scoring, or (b) construct an independent ground truth from developer commits/PRs that are referenced in the resolution discussions, and show that the OM-RAG advantage persists.
  2. [Section IV-G and Abstract] The abstract and Section IV-G state that model-free retrieval metrics (mean similarity 0.880) 'independently corroborate the ranking.' However, flat_rc_score and gt_keyword_hit_rate are reported only for OM-RAG; there is no model-free comparison for C-chunk or C-doc-kg. Thus these metrics cannot corroborate a ranking across configurations. Additionally, flat_rc_score measures embedding similarity between the query and the retrieved record, which is expected to be high for a KB built from the same extraction pipeline, and does not measure diagnostic correctness. The claim of independent corroboration is therefore overstated and should be removed or supported with analogous metrics for all configurations.
  3. [Section IV-B] The paper states that ground truth triples 'represent actual human-verified resolutions.' This is not supported by the described methodology: the 1,172 triples are LLM extractions, and the only manual step is the 123-issue verification of judge decisions against the extracted root causes. The sentence should be qualified or replaced by a description of the actual validation performed, and the Limitations section should acknowledge that the evaluation target itself is model-generated, not just the judge.
minor comments (3)
  1. [Section IV-C] The construction of C-chunk chunks (chunk size, overlap) and C-doc-kg (node types, edges, traversal) is not described in enough detail to permit replication; please add these details.
  2. [Section IV-B] The relationship between the 1,457 triples from ~1,396 issues and the 1,172-issue benchmark is unclear for issues that yield multiple triples; specify how ground truth is assigned and how the inference-time exclusion handles such issues.
  3. [Section IV-F] The paper does not report a quantitative judge-error rate from the 123-issue subset; 'all decisions were confirmed as reasonable' is not the same as a measured accuracy, and the effect-size robustness argument would be strengthened by reporting the number of judge errors out of 123.

Circularity Check

3 steps flagged · score 6.0 of 10

Ground truth and OM-RAG index both come from the same gpt-4o-mini extraction, so the central DA/FC ablation partly measures matching the extractor's own canonical root-cause phrasing rather than externally verified diagnosis.

  1. self definitional [Section III-B (Phase 2), Section IV-B (Benchmark Construction), Section IV-D (DA definition)]
    "For each enriched issue, we prompt an LLM (gpt-4o-mini) to extract a structured triple ... Issues with quality_score<3 are excluded ... 1,172 issues with complete, unambiguous ground truth form the evaluation benchmark. ... Ground truth triples are derived from developer-authored resolution discussions ... DA(Diagnosis Accuracy): Does the response correctly identify the root cause stated in the ground truth?"

    The DA target is the root_cause field of a gpt-4o-mini triple, and the OM-RAG index is built from the same extractor's triples: Phase 3 concatenates symptom and root_cause into the retrieved symptom-RC string. Thus OM-RAG presents the judge with the extractor's canonical root-cause phrasing as a precedent, and a response that echoes it is scored DA=1. C-chunk and C-doc-kg do not receive this pre-canonicalized target phrasing, so the reported +186% and +77% margins are inflated by format self-consistency. Excluding the issue's own record prevents exact leakage but not cross-issue canonicalization. The 123-issue manual check validates judge decisions against the extracted root cause, not the extraction against the original threads, so it cannot break the loop.

  2. other [Section III-B (quality_score definition) and Section IV-B (benchmark construction)]
    "quality_score∈[1,5]: How clearly the issue defines all three fields ... Issues with quality_score<3 are excluded. ... The benchmark applies the Phase 1–2 pipeline of Section III (2,283 bug-labeled issues→ 1,642 comment-enriched→ 1,641 extracted→ 1,172 retained at quality_score≥3)."

    The benchmark inclusion criterion is the extracting model's own confidence that it can form a clean triple. This selects issues whose threads the extractor can canonicalize, which is exactly the condition under which OM-RAG's retrieved triples align with the scored target. The paper labels the result 'complete, unambiguous ground truth', but the completeness judgment is made by the same gpt-4o-mini that produces both the KB records and the ground-truth fields, so the test set is partly optimized for the OM-RAG representation.

1 more flagged steps
  1. other [Section IV-D (Model-free metrics) and Section IV-G (Retrieval Quality at Scale)]
    "Two corroborating metrics require no LLM judge: flat_rc_score: cosine similarity between the query embedding and the retrieved RC record embedding ... and gt_keyword_hit_rate: fraction of ground truth keywords appearing in the generated response, a model-free proxy for answer correctness, not subject to judge bias."

    These metrics are free of the claude judge, but they are not free of the extraction loop. The 'RC record embedding' is computed from the extracted root_cause, and the 'ground truth keywords' come from the extracted ground-truth triple, so both metrics operate entirely within the same gpt-4o-mini canonical representation that OM-RAG already exposes. They can corroborate retrieval quality inside that representation, but they cannot independently validate the extracted ground truth against the original developer threads, which is the step that would break the circularity.

full rationale

The central ablation is real and carefully controlled in several respects: the four configurations share the same inference LLM, the same issue set, the same judge, the same own-issue exclusion, and the reported bootstrap intervals are non-overlapping. The manual 123-issue subset and cross-judge agreement show that claude-sonnet-4-6 applies the rubric consistently. However, the evaluation target and the OM-RAG knowledge base are produced by the same gpt-4o-mini extraction step: the ground-truth root_cause and the retrieved symptom-RC records are both fields of that extraction, and the quality_score>=3 benchmark filter is self-assigned by that extractor. Consequently, OM-RAG's retrieval structure directly exposes the canonical form in which the judge expects the answer, while the chunk and documentation baselines must reconstruct that form from raw or concept-level text. This makes a substantial part of the DA/FC advantage a self-consistency effect between target and evidence, not an externally verified diagnostic gain. The Limitations section concedes that primary scoring relies on an LLM judge rather than blind expert annotation and that the domain labels come from the same extractor, but it does not concede the more damaging point that the scored target itself is model-generated. No load-bearing self-citation chain or imported uniqueness theorem is present; the self-citations [7,8] are contextual. Because the central comparative claim is partly forced by the shared extraction pipeline, but the benchmark is public, the comparisons are statistically meaningful, and some judge-side controls exist, the appropriate score is 6 (partial circularity), not 8 or 10.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the correctness of LLM-extracted triples used both as the knowledge base and as ground truth, and on the representativeness of a single system's issue history. No free parameters are fitted to the test set in a predictive sense, but the hand-chosen thresholds and k directly shape the benchmark and the reported effect sizes.

free parameters (3)
  • quality_score threshold = 3
    Issues with LLM-assigned quality_score < 3 are excluded from the benchmark (Section III-B). The threshold is chosen by hand and is assigned by the same model that performs extraction, with no external validation of the filtering.
  • top-k retrieval count = 3
    Inference retrieves the top 3 most similar records (Section III-C). The choice of k is not ablated, and the central comparison does not test sensitivity to this parameter.
  • minimum substantive comments = 2
    Issues are filtered to those with at least two substantive comments before extraction (Section III-A). This threshold is hand-chosen and not varied.
assumptions (4)
  • domain assumption LLM-extracted triples with quality_score >= 3 constitute correct ground truth
    Section III-B: quality filtering is performed by the same gpt-4o-mini model that extracts the triples. No full human audit of the 1,172 benchmark is reported.
  • domain assumption claude-sonnet-4-6 LLM judge is an approximate oracle for DA, FC, G, S
    Section IV-D/F: judge reliability is checked on a 23-issue subset and manual verification covers 123 of 1,172 issues. The full benchmark scoring still depends on this assumption.
  • domain assumption Dataverse issue tracker is representative for generalizing source and structure effects
    Section IV-I: single production system limitation is acknowledged. The magnitude of the reported improvements may vary in systems with different issue-tracking practices.
  • domain assumption Single-hop retrieval suffices for bug diagnosis; added graph structure adds noise
    Section III-C: this is asserted and empirically supported by the C-doc-kg comparison, but is not proven across systems or incident types.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Resolved Incident History for LLM-Assisted Software Bug Diagnosis." pith.science (2026). https://pith.science/paper/JBF5DZMP

@misc{pith2026260721911,
  author       = {Pith},
  title        = {Pith review of: Leveraging Resolved Incident History for LLM-Assisted Software Bug Diagnosis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JBF5DZMP}},
  note         = {Machine review of arXiv:2607.21911}
}
read the original abstract

Effective software bug diagnosis requires two ingredients: the right knowledge source (operational failure history, not just system documentation) and the right retrieval structure (structured records, not unstructured chunks). Current retrieval-augmented generation (RAG) approaches fall short on one or both dimensions. We propose Operational Memory RAG (OM-RAG), which indexes resolved issues as structured symptom-root cause-resolution triples and retrieves the most similar historical precedent via single-hop embedding. OM-RAG powers a purpose-built large language model (LLM) administrator that has operated a production Dataverse instance for over six months, operationalizing the Knowledge component of the Monitor-Analyze-Plan-Execute-Knowledge (MAPE-K) feedback loop with the system's own resolved failure history. In a controlled four-configuration ablation, LLM-based judging covers all 1,172 issues and manual verification a 123-issue subset. OM-RAG achieves Diagnosis Accuracy of 0.931 and Fix Correctness of 0.809, outperforming chunk-based retrieval (+186%) and a documentation-anchored concept graph (+77%). Model-free retrieval metrics (mean similarity 0.880) independently corroborate the ranking.

Figures

Figures reproduced from arXiv: 2607.21911 by the authors.

Figure 1
Figure 1. The four-phase OM-RAG pipeline. Phase 1 collects GitHub issues with comment threads. Phase 2 extracts structured triples via LLM and quality-filters to records. Phase 3 embeds symptoms into a flat dense index. Phase 4 retrieves the top-k most similar triples for a new symptom and prompts the LLM to generate a grounded diagnosis. alongside the full triple and issue metadata. At inference, the top-k (k = 3) most simil… view at source ↗
Figure 2
Figure 2. Full benchmark retrieval quality (n=1,172 issues). Left: flat_rc_score distribution (cosine similarity to retrieved RC triple); 55% of issues score ≥0.90. Center: GT keyword hit rate distribution (mean = 0.722). Right: mean flat_rc_score by domain; 13 primary categories shown plus other (n=11). (DA, FC, G, S) independently rather than overall response quality. Effect-size robustness. The three bias mitigation studie… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 13 canonical work pages

  1. [1]

    Seven failure points when engineering a retrieval augmented generation system,

    S. Barnett, S. Kurniawan, S. Thudumu, Z. Brannelly, and M. Abdelrazek, “Seven failure points when engineering a retrieval augmented generation system,” inProceedings of the IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN), 2024, pp. 194–199

    work page 2024

  2. [2]

    Episodic and semantic memory,

    E. Tulving, “Episodic and semantic memory,” inOrganization of Memory, E. Tulving and W. Donaldson, Eds. New York: Academic Press, 1972, pp. 381–403

    work page 1972

  3. [3]

    On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices,

    H. P. A. Boshuizen and H. G. Schmidt, “On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices,” Cognitive Science, vol. 16, no. 2, pp. 153–184, 1992

    work page 1992

  4. [4]

    The vision of autonomic computing,

    J. O. Kephart and D. M. Chess, “The vision of autonomic computing,” Computer, vol. 36, no. 1, pp. 41–50, 2003

    2003

  5. [5]

    Generative AI for self-adaptive systems: State of the art and research roadmap,

    J. Li, M. Zhang, N. Li, D. Weyns, Z. Jin, and K. Tei, “Generative AI for self-adaptive systems: State of the art and research roadmap,”arXiv preprint arXiv:2512.04680, 2025

    work page arXiv 2025

  6. [6]

    Case-based reasoning: Foundational issues, methodological variations, and system approaches,

    A. Aamodt and E. Plaza, “Case-based reasoning: Foundational issues, methodological variations, and system approaches,”AI Communications, vol. 7, no. 1, pp. 39–59, 1994

    work page 1994

  7. [7]

    A dual-helix governance approach towards reliable agentic AI for WebGIS development,

    B. Guan, W. Cui, and L. Juhasz, “A dual-helix governance approach towards reliable agentic AI for WebGIS development,”arXiv preprint arXiv:2603.04390, 2026

    arXiv 2026

  8. [8]

    Exploring robust multi- agent workflows for environmental data management,

    B. Guan, J. Liu, Y . Wu, and K. Bahreini, “Exploring robust multi- agent workflows for environmental data management,” inPractice and Experience in Advanced Research Computing (PEARC), 2026

    work page 2026

Show all 24 references
  1. [9]

    RAGAS: Automated evaluation of retrieval augmented generation,

    S. Es, J. James, L. Espinosa-Anke, and S. Schockaert, “RAGAS: Automated evaluation of retrieval augmented generation,”arXiv preprint arXiv:2309.15217, 2023

    2023 arXiv

  2. [10]

    Large language models are not fair evaluators,

    P. Wang, L. Li, L. Chen, D. Zhu, B. Lin, Y . Cao, Q. Liu, T. Liu, and Z. Sui, “Large language models are not fair evaluators,”arXiv preprint arXiv:2305.17926, 2023

    2023 arXiv

  3. [11]

    Judging the judges: Evaluating alignment and vulnerabilities in LLMs-as-Judges,

    A. S. Thakuret al., “Judging the judges: Evaluating alignment and vulnerabilities in LLMs-as-Judges,”arXiv preprint arXiv:2406.12624, 2024

    2024 arXiv

  4. [12]

    From local to global: A graph RAG approach to query-focused summarization,

    D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson, “From local to global: A graph RAG approach to query-focused summarization,”arXiv preprint arXiv:2404.16130, 2024

    2024 arXiv

  5. [13]

    HippoRAG: Neurobiologically inspired long-term memory for large language models,

    B. J. Gutierrez, Y . Zeng, Q. Han, Y . Zhang, Y . Yang, and J. Chen, “HippoRAG: Neurobiologically inspired long-term memory for large language models,” inAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  6. [14]

    LightRAG: Simple and fast retrieval-augmented genera- tion,

    Z. Guoet al., “LightRAG: Simple and fast retrieval-augmented genera- tion,”arXiv preprint arXiv:2410.05779, 2024

    2024 arXiv

  7. [15]

    RAPTOR: Recursive abstractive processing for tree-organized retrieval,

    P. Sarthi, S. Abdullah, A. Tuli, S. Khanna, A. Goldie, and C. D. Manning, “RAPTOR: Recursive abstractive processing for tree-organized retrieval,” inInternational Conference on Learning Representations (ICLR), 2024

    2024

  8. [16]

    When to use graphs in RAG: A comprehensive analysis for graph retrieval-augmented generation,

    Z. Xianget al., “When to use graphs in RAG: A comprehensive analysis for graph retrieval-augmented generation,”arXiv preprint arXiv:2506.05690, 2025, iCLR 2026 poster

    2025

  9. [17]

    eARCO: Efficient automated root cause analysis with prompt optimization,

    S. Goel, M. Magazine, S. Ghosh, A. Nambi, A. Deshpande, X. Zhang, C. Bansal, and S. Rajmohan, “eARCO: Efficient automated root cause analysis with prompt optimization,” inProceedings of the AAAI Conference on Artificial Intelligence, 2025, arXiv:2504.11505

    2025 arXiv

  10. [18]

    Automated root causing of cloud incidents using in- context learning with GPT-4,

    X. Zhang, S. Ghosh, C. Bansal, R. Wang, M. Ma, Y . Kang, and S. Rajmohan, “Automated root causing of cloud incidents using in- context learning with GPT-4,”arXiv preprint arXiv:2401.13810, 2024

    2024 arXiv

  11. [19]

    Automatic root cause analysis via large language models for cloud incidents,

    Y . Chen, H. Xie, M. Ma, Y . Kang, X. Gao, L. Shi, Y . Cao, X. Gao, H. Fan, M. Wen, J. Zeng, S. Ghosh, X. Zhang, C. Zhang, Q. Lin, S. Rajmohan, D. Zhang, and T. Xu, “Automatic root cause analysis via large language models for cloud incidents,” inProceedings of the Nineteenth E...

    2024

  12. [20]

    Diagnosing and resolving cloud platform instability with multi-modal RAG LLMs,

    T. Wanget al., “Diagnosing and resolving cloud platform instability with multi-modal RAG LLMs,”Proceedings of EuroMLSys, 2025

    2025

  13. [21]

    SWE-bench: Can language models resolve real-world GitHub issues?

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “SWE-bench: Can language models resolve real-world GitHub issues?” inInternational Conference on Learning Representations (ICLR), 2024

    2024

  14. [22]

    Prometheus: Towards long-horizon codebase navigation for repository-level problem solving,

    Y . Panet al., “Prometheus: Towards long-horizon codebase navigation for repository-level problem solving,”arXiv preprint arXiv:2507.19942, 2025

    2025

  15. [23]

    Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” inAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  16. [24]

    MAPER: Extending MAPE-K with LLM-based reasoning to manage unanticipated situations in self-adaptive systems,

    P. Maia, L. Vieira, G. L. B. De Oliveira, M. Chagas, A. Bandeira, and C. Rocha, “MAPER: Extending MAPE-K with LLM-based reasoning to manage unanticipated situations in self-adaptive systems,” inProceedings of the International Symposium on Software Engineering for Adaptive and...

    2026

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.