Pith. sign in

REVIEW 3 major objections 5 minor 42 references

Refining bug reports from unrepaired agent trajectories raises automated repair success by large margins.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 16:04 UTC pith:SSSBUYA5

load-bearing objection Solid empirical APR paper: refining reports from unverified trajectories lifts Pass@1 a lot, with a real but not fatal confound about exploration budget vs. true specification. the 3 major comments →

arxiv 2607.07882 v1 pith:SSSBUYA5 submitted 2026-07-08 cs.SE

Bug Report Specification Refinement with Trajectory Guidance for Automated Program Repair

S M Farah Al Fahim , Md Nakhla Rafi , Md Ahasanuzzaman , Zeyang Ma , Dong Jae Kim , Shaowei Wang , Tse-Hsun (Peter) Chen This is my paper
classification cs.SE
keywords automated program repairbug report enhancementspecification refinementtrajectory-guided agentsrepository-level repairlarge language modelsSWE-Bench
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Bug reports that tell a repair agent only what failed often leave out how the failure arises, what behavior should hold, and which code is in scope. The paper argues that an agent run that explores the pre-fix repository without trusting any patch it produces still leaves a useful trail of searches, inspections, and intermediate diagnoses. TrajSpec turns that trail into a three-level evidence structure (high-level interpretation, supporting diagnostic findings, and concrete code observations), drafts a fuller report, and reviews every claim against the repository so unsupported material is cut or hedged. On a standard 300-issue suite the refined reports raise first-try repair rates from 41% to about 60% with one model and from about 55% to 64% with another; the same reports also lift two other repair agents by similar amounts. Ablations show that both the hierarchical evidence and the repository review are necessary for most of the gain. The practical claim is that better task specifications, not only stronger repair agents, are a high-leverage way to improve repository-level automated repair.

Core claim

TrajSpec shows that an unverified exploration trajectory, when abstracted into hierarchical evidence for failure mechanism, behavioral requirement, and implementation scope and then reviewed against the pre-fix repository, yields refined bug reports that substantially raise Pass@1 for multiple downstream repair agents, without using developer patches, post-fix code, or test outcomes during refinement.

What carries the argument

TrajSpec: a pipeline that collects an ordered thought-action-observation trajectory, organizes extracted evidence into three branches (mechanism, requirement, scope) each at three levels (high / mid / low), drafts a fixed-schema report, and subjects every added claim to repository-based review before handing the report to a repair agent.

Load-bearing premise

That an unrepaired agent’s exploration of the original report and pre-fix code systematically surfaces the right repair-relevant facts, and that repository review can filter the noise without access to the true fix or tests.

What would settle it

Re-run the same 300-issue suite with trajectories from a deliberately weak explorer or with review disabled permanently; if Pass@1 gains disappear or fall to the raw-trajectory baseline, the claim that hierarchical abstraction plus repository review recovers usable specifications fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Repair systems can treat report refinement as a reusable preprocessing step rather than only improving localization or patch generation.
  • The same refined report can be shared across different agent architectures and still improve first-try success.
  • Hierarchy plus review are both load-bearing: dropping either erases roughly a fifth of the Pass@1 gain.
  • End-to-end monetary cost rises modestly, while downstream repair token use falls once the refined report exists.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If trajectories already encode useful scope and requirement evidence, similar refinement could help other underspecified software tasks such as feature implementation or regression diagnosis.
  • The large gains from report quality alone imply that many current agent failures are specification failures rather than pure search or generation failures.
  • A natural next test is whether refined reports remain helpful when the trajectory-collection model is weaker or cheaper than the final repair model.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that repository-level APR agents are limited by underspecified bug reports that omit failure mechanism, behavioral requirement, and implementation scope, and proposes TrajSpec to refine reports before repair. TrajSpec runs an unverified trajectory-collection agent on the original report and pre-fix repository, extracts trajectory-derived evidence into a three-branch, three-level hierarchical representation M, drafts a fixed-schema refined report, and applies repository-based review to remove unsupported claims and add repository-supported detail. On all 300 SWE-Bench Lite instances with Mini-SWE-Agent V2, refined reports raise Pass@1 from 41.00% to 59.67% (GPT-5-mini) and 54.67% to 64.33% (MiniMax M2.5); on a stratified 100-instance sample they also improve Agentless (41%→71%) and AutoCodeRover (47%→72%). Ablations attribute large drops to removing review or hierarchy, and a cost study reports modest end-to-end overhead with reduced downstream repair tokens.

Significance. If the interpretation holds, the work shifts repository-level APR from strengthening agents alone toward improving the task specification they receive—an important and under-addressed lever. Strengths include a full 300-instance evaluation, two LLM backbones, three downstream agents (two on a stratified sample), an Agentic-Base trajectory baseline, ablations of hierarchy and review (Table IV), repaired-instance overlap analysis (Figure 3), and transparent cost accounting (Table V). The design deliberately discards candidate patches and avoids developer patches, post-fix code, and oracle outcomes during refinement, which is the right isolation for a specification-refinement claim. The motivating example (astropy-14365) concretely illustrates incomplete symptom-only reports. These empirical and design choices make the paper a solid contribution to APR and bug-report enhancement if the causal story is tightened.

major comments (3)
  1. [§IV-A, RQ1 baselines, Tables II & IV] The central framing is that gains come from repository-supported specification refinement (failure mechanism, behavioral requirement, implementation scope), not merely from amortized exploration. Trajectory collection (§IV-A) is itself a full agent run on b and Rc; both Agentic-Base and TrajSpec therefore inject exploration the original-report baseline never receives. Agentic-Base (raw τ → report, no hierarchy/review) underperforms TrajSpec in Table II, and Table IV shows hierarchy and review matter, but neither control tests whether a second independent exploration budget, or a localization-only report that pastes low-level code locations/files from τ into the original report without rewriting RootCause/ExpectedBehavior, would recover most of the Pass@1 lift. Without such a control (or a clear limitation reframing TrajSpec as amortized localization/context packing), the load-bearing cla
  2. [§I contributions; §IV-B–C; §V RQ1–RQ3; §VI Construct Validity] Evaluation measures only end-to-end Pass@1. The paper’s three claimed specification dimensions (failure mechanism, behavioral requirement, implementation scope) and the hi/mid/lo structure of M are never scored independently—e.g., whether refined reports correctly expand implementation scope beyond the reported symptom (as in the astropy-14365 motivating example), or whether ExpectedBehavior claims survive review as supported vs. over-inferred. §VI Construct Validity acknowledges that Pass@1 is not a direct report-quality metric, but this gap is load-bearing for the contribution statement in the abstract and §I: without even a small manual or automatic audit of refined-report content against developer patches or held-out tests, it is hard to distinguish better specifications from better localization hints packaged as prose.
  3. [§V RQ2, Table III] RQ2’s cross-agent claim (Agentless 41%→71%, AutoCodeRover 47%→72%) rests on a 100-instance stratified sample rather than the full 300 used for Mini-SWE-Agent V2. While cost is a fair reason, the absolute resolved counts and repository-level stability of the large relative gains are not reported at the same granularity as Table II. Given that generalization beyond the primary agent is a listed contribution, either expand the sample or report confidence intervals / per-repository breakdowns so the cross-agent result can be weighed against the full-benchmark primary result.
minor comments (5)
  1. [Figure 3] Figure 3’s Venn-style overlap is informative but hard to parse from the text alone (e.g., the meaning of the four numeric regions per backbone). A short legend or explicit set-size labels would help.
  2. [§II] Related work positions TrajSpec against CodeScout, SGAgent, SHERLOC, and historical-trajectory reuse (ExpeRepair, ConRAD). A one-sentence table or bullet contrasting evidence source (static pre-exploration vs. current-issue unverified trajectory vs. historical verified patches) and output artifact (refined report vs. in-agent guidance) would sharpen novelty.
  3. [§IV-C Draft Specification Generation] The fixed report schema (Title, Description, RootCause, StepsToReproduce, ExpectedBehavior, ObservedBehavior) is justified by prior bug-report studies, but it is unclear whether RootCause is allowed to prescribe edits or only describe mechanism/scope; a short policy sentence would reduce ambiguity for reimplementation.
  4. [§V RQ4, Table V] Table V reports average tokens/cost but not variance or correlation between report-generation cost and instance difficulty. A brief note on outliers would strengthen the “modest overhead” claim.
  5. [Throughout] Minor polish: consistent hyphenation of “pre-fix” / “repository-level”; ensure all arXiv-dated 2025–2026 references remain stable at camera-ready.

Circularity Check

0 steps flagged

No significant circularity: TrajSpec is an empirical pipeline whose refined reports are evaluated by independent Pass@1 oracles, with no fitted-as-prediction or self-definitional reduction.

full rationale

The paper proposes a concrete engineering pipeline (unverified trajectory collection on original report b and pre-fix Rc, hierarchical evidence M, draft generation, repository-based review) and measures success solely by whether a separate downstream repair agent’s first patch passes the SWE-Bench Lite fail-to-pass/pass-to-pass oracle. Refinement is explicitly isolated from ground truth: candidate patches from the trajectory run are discarded, and developer patches, post-fix code, and benchmark outcomes are never used (§IV, §VI). The Agentic-Base baseline and ablations (Table IV) further separate raw trajectory injection from the hierarchical+review steps. There is no equation, uniqueness theorem, or parameter fit that is later re-labeled a “prediction”; self-citations (e.g., Fahim et al. [25]) appear only in related-work positioning and are not load-bearing for the Pass@1 claims. Mild contamination or same-model-family risks are external validity concerns, not circular reductions of the claimed derivation. The result is therefore self-contained against the external benchmark.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 3 invented entities

Empirical SE paper: load-bearing content is a pipeline design plus Pass@1 measurements, not free-parameter physics. The claim rests on domain assumptions that incomplete reports hurt APR, that trajectories encode recoverable specification evidence, and that Pass@1 under SWE-Bench oracles measures specification utility. Invented entities are the TrajSpec artifacts (hierarchical M, review protocol). No numerical constants are fitted to produce the central percentages; model choice and sampling are experimental factors.

free parameters (2)
  • LLM backbone and decoding settings for trajectory collection, abstraction, draft, and review = GPT-5-mini / MiniMax M2.5 as reported; decoding hyperparameters not fully specified
    Results depend on which models and (unstated) temperatures/sampling settings are used for each stage; not fitted to a closed-form target but chosen experimentally and affect Pass@1.
  • Fixed refined-report schema fields (Title, Description, RootCause, StepsToReproduce, ExpectedBehavior, ObservedBehavior) = Six named fields as in §IV-C
    Schema is a design choice that shapes what the draft and review can express; different schemas could change downstream agent behavior.
axioms (5)
  • domain assumption Bug reports often omit failure mechanism, behavioral requirement, and implementation scope needed for repository-level APR.
    Stated in abstract and §I; motivates the whole pipeline; supported by citations but not re-proven here.
  • domain assumption An ordered thought–action–observation trajectory from an unverified agent run is a usable (if noisy) source of specification evidence for the current issue.
    Core premise of §IV-A; if false, hierarchical extraction cannot help.
  • domain assumption Pass@1 under SWE-Bench Lite fail-to-pass and pass-to-pass tests is a valid measure of refined-report utility for APR.
    Construct validity discussion in §VI; does not measure human report quality.
  • ad hoc to paper Repository-based LLM review with read-only tools can remove unsupported claims without ground-truth patches.
    Design claim of §IV-C; ablation shows review matters, but correctness of individual claims is not independently labeled.
  • standard math Standard LLM agent and APR evaluation practices (SWE-Bench Lite, named agents) are appropriate experimental instruments.
    Community benchmark and agents treated as given measurement tools rather than derived results.
invented entities (3)
  • Hierarchical evidence representation M = (M_mech, M_req, M_scope) each with hi/mid/lo levels no independent evidence
    purpose: Structure trajectory-derived findings so draft generation preserves both diagnosis and concrete code observations.
    Introduced in §IV-B; ablation shows removing it drops Pass@1 to 47.67%. No independent existence outside this pipeline.
  • TrajSpec end-to-end refinement pipeline (trajectory collection → extraction → hierarchy → draft → repository review) no independent evidence
    purpose: Produce repository-supported refined bug reports for any downstream repair agent.
    Central invented method of the paper; evaluated only via downstream Pass@1.
  • Agentic-Base baseline (raw trajectory → refined report without hierarchy or review) no independent evidence
    purpose: Isolate contribution of hierarchy and repository review versus naive trajectory-conditioned rewriting.
    Constructed comparison method in RQ1; not claimed as a prior published system.

pith-pipeline@v1.1.0-grok45 · 22846 in / 3721 out tokens · 42574 ms · 2026-07-10T16:04:28.394340+00:00 · methodology

0 comments
read the original abstract

Bug reports serve as task specifications for repository-level automated program repair (APR) agents, but they often describe only the observed failure and omit repair-relevant information such as the failure-inducing behavior, behavioral requirement, and implementation scope. As a result, a repair agent may inspect irrelevant code, infer an incorrect requirement, or generate a patch that addresses the reported symptom without restoring the intended repository behavior. We present TrajSpec, a trajectory-guided approach for repository-supported bug report specification refinement. Given an original report and a pre-fix repository, TrajSpec runs a trajectory-collection agent and uses the resulting unverified trajectory as a source of trajectory-derived specification evidence. It organizes this evidence into a three-level representation consisting of a high-level interpretation of the issue, diagnostic findings supporting that interpretation, and concrete repository observations. TrajSpec then generates a draft refined report and applies repository-based review to remove unsupported claims, revise uncertain claims, and add repository-supported details. We evaluate TrajSpec on all 300 SWE-Bench Lite instances using Mini-SWE-Agent V2. TrajSpec's refined reports improve Pass@1 from 41.00% to 59.67% with GPT-5-mini and from 54.67% to 64.33% with MiniMax M2.5. On a stratified sample of 100 instances, TrajSpec's refined reports also improve Pass@1 from 41.00% to 71.00% with Agentless and from 47.00% to 72.00% with AutoCodeRover. Ablation results show that removing repository-based review or the hierarchical evidence representation reduces Pass@1 from 59.67% to 48.00% and 47.67%, respectively. Overall, TrajSpec provides actionable repository-supported context that consistently improves repair performance.

Figures

Figures reproduced from arXiv: 2607.07882 by Dong Jae Kim, Md Ahasanuzzaman, Md Nakhla Rafi, Shaowei Wang, S M Farah Al Fahim, Tse-Hsun (Peter) Chen, Zeyang Ma.

Figure 1
Figure 1. Figure 1: Simplified before-and-after report excerpts for [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of TrajSpec. Given an original bug report b and the pre-fix repository snapshot Rc, TrajSpec produces a final refined report ˆbf that serves as a repository-supported specification for downstream repair. TrajSpec first runs a trajectory￾collection agent using only b and Rc, discards any candidate patch produced during this run, and retains the execution trajectory τ . It then uses b and τ to recov… view at source ↗
Figure 3
Figure 3. Figure 3: Overlap of repaired SWE-Bench Lite instances across [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages · 13 internal anchors

  1. [1]

    Swe-agent: Agent- computer interfaces enable automated software engi- neering,

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “Swe-agent: Agent- computer interfaces enable automated software engi- neering,”Advances in Neural Information Processing Systems, vol. 37, pp. 50 528–50 652, 2024

    work page 2024

  2. [2]

    Autocoderover: Autonomous program improvement,

    Y . Zhang, H. Ruan, Z. Fan, and A. Roychoudhury, “Autocoderover: Autonomous program improvement,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. Associ- ation for Computing Machinery, 2024, p. 1592–1604

    work page 2024

  3. [3]

    Repairagent: An autonomous, llm-based agent for program repair,

    I. Bouzenia, P. Devanbu, and M. Pradel, “Repairagent: An autonomous, llm-based agent for program repair,” in2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 2025, pp. 2188– 2200

    work page 2025

  4. [4]

    Sgagent: Suggestion-guided llm-based multi-agent framework for repository-level software re- pair,

    Q. Zhang, C. Gao, Y . Han, Y . Shang, C. Fang, Z. Chen, and L. Xiao, “Sgagent: Suggestion-guided llm-based multi-agent framework for repository-level software re- pair,”ACM Transactions on Software Engineering and Methodology, 2026

    work page 2026

  5. [5]

    SHERLOC: Structured Diagnostic Localization for Code Repair Agents

    H. Tamoyan, S. Narenthiran, E. Arakelyan, M. Mezini, and B. Ginsburg, “Sherloc: Structured diagnostic lo- calization for code repair agents,”arXiv preprint arXiv:2606.24820, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  6. [6]

    Swe-bench: Can lan- guage models resolve real-world github issues?

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “Swe-bench: Can lan- guage models resolve real-world github issues?” inIn- ternational Conference on Learning Representations, vol. 2024, 2024, pp. 54 107–54 157

    work page 2024

  7. [7]

    What makes a good bug report?

    N. Bettenburg, S. Just, A. Schr ¨oter, C. Weiss, R. Premraj, and T. Zimmermann, “What makes a good bug report?” inProceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, 2008, pp. 308–318

    work page 2008

  8. [8]

    The significance of bug report elements,

    M. Soltani, F. Hermans, and T. B ¨ack, “The significance of bug report elements,”Empirical Software Engineering, vol. 25, no. 6, pp. 5255–5294, 2020

    work page 2020

  9. [9]

    Demystifying the challenges and benefits of analyzing user-reported logs in bug reports,

    A. R. Chen, T.-H. P. Chen, and S. Wang, “Demystifying the challenges and benefits of analyzing user-reported logs in bug reports,”Empirical Software Engineering, vol. 26, no. 1, pp. 1–30, 2021

    work page 2021

  10. [10]

    Pathidea: Improving information retrieval-based bug localization by re-constructing execution paths using logs,

    A. R. Chen, T.-H. Chen, and S. Wang, “Pathidea: Improving information retrieval-based bug localization by re-constructing execution paths using logs,”IEEE Transactions on Software Engineering, vol. 48, no. 8, pp. 2905–2919, 2021

    work page 2021

  11. [11]

    Root cause prediction based on bug reports,

    T. Hirsch and B. Hofer, “Root cause prediction based on bug reports,” in2020 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 2020, pp. 171–176

    work page 2020

  12. [12]

    The impact of bug localization based on crash report mining: A developers’ perspective,

    M. Medeiros, U. Kulesza, R. Coelho, R. Bonif ´acio, C. Treude, and E. A. Barbosa, “The impact of bug localization based on crash report mining: A developers’ perspective,” inProceedings of the 46th International Conference on Software Engineering: Software Engineer- ing in Practice, 2024, pp. 13–24

    work page 2024

  13. [13]

    Codescout: Contextual problem statement enhancement for software agents,

    M. Suri, X. Li, M. Shojaie, S. Han, C.-C. Hsu, S. Garg, A. A. Deshmukh, and V . Kumar, “Codescout: Contextual problem statement enhancement for software agents,” in Findings of the Association for Computational Linguis- tics: ACL 2026, 2026, pp. 40 902–40 931

    work page 2026

  14. [14]

    REAgent: Requirement-Driven LLM Agents for Software Issue Resolution

    S. Kuang, Z. Tian, K. Lin, C. Tao, S. Wang, H. Bai, L. Shang, and J. Chen, “Reagent: Requirement-driven llm agents for software issue resolution,”arXiv preprint arXiv:2604.06861, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  15. [15]

    Demysti- fying llm-based software engineering agents,

    C. S. Xia, Y . Deng, S. Dunn, and L. Zhang, “Demysti- fying llm-based software engineering agents,” vol. 2, no. FSE, 2025, pp. 801–824

    work page 2025

  16. [16]

    mini-swe-agent v2: The minimal ai software engineering agent,

    P. N. Group and Contributors, “mini-swe-agent v2: The minimal ai software engineering agent,” https://github. com/SWE-agent/mini-swe-agent, 2025, accessed: 2026- 06-26

    work page 2025

  17. [17]

    Orcaloca: An llm agent frame- work for software issue localization,

    Z. Yu, H. Zhang, Y . Zhao, H. Huang, M. Yao, K. Ding, and J. Zhao, “Orcaloca: An llm agent frame- work for software issue localization,”arXiv preprint arXiv:2502.00350, 2025

    work page arXiv 2025

  18. [18]

    Live-swe-agent: Can software engineering agents self- evolve on the fly?

    C. S. Xia, Z. Wang, Y . Yang, Y . Wei, and L. Zhang, “Live-swe-agent: Can software engineering agents self- evolve on the fly?”arXiv preprint arXiv:2511.13646, 2025

    work page arXiv 2025

  19. [19]

    Openhands: An open platform for ai software developers as generalist agents,

    X. Wang, B. Li, Y . Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y . Song, B. Li, J. Singhet al., “Openhands: An open platform for ai software developers as generalist agents,” inInternational Conference on Learning Repre- sentations, vol. 2025, 2025, pp. 65 882–65 919

    work page 2025

  20. [20]

    Summa- rizing software artifacts: a case study of bug reports,

    S. Rastkar, G. C. Murphy, and G. Murray, “Summa- rizing software artifacts: a case study of bug reports,” inProceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 1, 2010, pp. 505–514

    work page 2010

  21. [21]

    Jcharming: A bug reproduction approach using crash traces and directed model checking,

    M. Nayrolles, A. Hamou-Lhadj, S. Tahar, and A. Lars- son, “Jcharming: A bug reproduction approach using crash traces and directed model checking,” in2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 2015, pp. 101–110

    work page 2015

  22. [22]

    Re- bucket: A method for clustering duplicate crash reports based on call stack similarity,

    Y . Dang, R. Wu, H. Zhang, D. Zhang, and P. Nobel, “Re- bucket: A method for clustering duplicate crash reports based on call stack similarity,” in2012 34th International Conference on Software Engineering (ICSE). IEEE, 2012, pp. 1084–1093

    work page 2012

  23. [23]

    Can We Enhance Bug Report Quality Using LLMs?: An Empirical Study of LLM-Based Bug Report Generation

    J. Acharya and G. Ginde, “Can we enhance bug report quality using llms?: An empirical study of llm-based bug report generation,”arXiv preprint arXiv:2504.18804, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    Llput: Investigating large language models for bug report-based input generation,

    A. Al Hasan, S. Saha, M. M. Imran, and T. S. Zaman, “Llput: Investigating large language models for bug report-based input generation,” inProceedings of the 33rd ACM International Conference on the Foundations of Software Engineering, 2025, pp. 1652–1659

    work page 2025

  25. [25]

    Crash report enhancement with large language models: An empirical study,

    S. Fahim, M. N. Rafi, Z. Ma, D. J. Kimet al., “Crash report enhancement with large language models: An empirical study,”arXiv preprint arXiv:2509.13535, 2025

    work page arXiv 2025

  26. [26]

    Enhancing repository-level software repair via repository-aware knowledge graphs,

    B. Yang, J. Ren, S. Jin, Y . Liu, F. Liu, B. Le, and H. Tian, “Enhancing repository-level software repair via repository-aware knowledge graphs,”arXiv preprint arXiv:2503.21710, 2025

    work page arXiv 2025

  27. [27]

    Reporepair: Leveraging code documentation for repository-level automated program repair,

    Z. Pan, C. Li, W. Zhong, Y . Feng, B. Luo, and V . Ng, “Reporepair: Leveraging code documentation for repository-level automated program repair,”arXiv preprint arXiv:2603.01048, 2026

    work page arXiv 2026

  28. [28]

    EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair

    F. Mu, J. Wang, L. Shi, S. Wang, S. Li, and Q. Wang, “Experepair: Dual-memory enhanced llm- based repository-level program repair,”arXiv preprint arXiv:2506.10484, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  29. [29]

    From Historical Patches to Repair Plans: Outcome-Conditioned Reasoning for Repository-Level Program Repair

    C. Li, Y . Xu, Z. Wang, S. H. Tanet al., “Outcome- conditioned reasoning distillation for resolving software issues,”arXiv preprint arXiv:2601.23257, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  30. [30]

    SWE Context Bench: A Benchmark for Context Learning in Coding

    J. Zhu, J. Wu, M. Hu, S. Zhu, J. Pan, W. Shen, Y . Yang, F. Liu, J. Hao, Y . Jinet al., “Swe context bench: A benchmark for context learning in coding,”arXiv preprint arXiv:2602.08316, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  31. [31]

    Understanding software engineering agents: A study of thought-action-result tra- jectories,

    I. Bouzenia and M. Pradel, “Understanding software engineering agents: A study of thought-action-result tra- jectories,” in2025 40th IEEE/ACM International Confer- ence on Automated Software Engineering (ASE), 2025, p. 2846–2857

    work page 2025

  32. [32]

    Specification vibing for automated program repair,

    T. Zhu, L. C. Cordeiro, M. A. Mustafa, and Y . Sun, “Specification vibing for automated program repair,” arXiv preprint arXiv:2602.08263, 2026

    work page arXiv 2026

  33. [33]

    DynaFix: Iterative Automated Program Repair Driven by Execution-Level Dynamic Information

    Z. Huang, L. Xu, C. Liu, W. Sun, X. Zhang, Y . Lei, M. Yan, and H. Zhang, “Dynafix: Iterative automated program repair driven by execution-level dynamic infor- mation,”arXiv preprint arXiv:2512.24635, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  34. [34]

    ImproBR: Bug Report Improver Using LLMs

    E. F. Akyol, M. Dedeler, and E. T ¨uz¨un, “Improbr: Bug report improver using llms,”arXiv preprint arXiv:2604.26142, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  35. [35]

    SCOPE: Prompt Evolution for Enhancing Agent Effectiveness

    Z. Pei, H.-L. Zhen, S. Kai, S. J. Pan, Y . Wang, M. Yuan, and B. Yu, “Scope: Prompt evolution for enhancing agent effectiveness,”arXiv preprint arXiv:2512.15374, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  36. [36]

    AgentSZZ: Teaching the LLM Agent to Play Detective with Bug-Inducing Commits

    Y . Lyu, J. Shi, H. J. Kang, R. Widyasari, J. He, Y . Niu, C. Yang, J. Chen, Z. Yang, J. Lawallet al., “Agentszz: Teaching the llm agent to play detective with bug- inducing commits,”arXiv preprint arXiv:2604.02665, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  37. [37]

    TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing

    P. Yang, W. Chen, T. Yang, P. Feng, J. Xing, W. Guo, Y . Yao, Y . Han, H. Li, X. Wanget al., “Twinrouterbench: Fast static and live dynamic evaluation for realistic agentic llm routing,”arXiv preprint arXiv:2605.18859, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  38. [38]

    DualEval: Joint Model-Item Calibration for Unified LLM Evaluation

    A. J. Li, H. Huang, Y . Park, Y . Ma, W.-L. Chiang, L. Chen, C.-J. Hsieh, B. Yu, and I. Stoica, “Dualeval: Joint model-item calibration for unified llm evaluation,” arXiv preprint arXiv:2606.26429, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  39. [39]

    Inference Time Context Sparsity: Illusion or Opportunity?

    S. Joshi, P. Dixit, A. Chowdhury, A. Shrivastava, J. E. Gonzalez, I. Stoica, K. K. Agrawal, and A. Desai, “In- ference time context sparsity: Illusion or opportunity?” arXiv preprint arXiv:2605.24168, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  40. [40]

    Coda- bench: Can code agents handle data-intensive tasks?

    Y . Zhang, J. Fan, M. Fan, S. Zhang, and X. Du, “Coda- bench: Can code agents handle data-intensive tasks?” arXiv preprint arXiv:2606.15300, 2026

    work page arXiv 2026

  41. [41]

    Gpt-5 mini model,

    OpenAI, “Gpt-5 mini model,” https://developers.openai. com/api/docs/models/gpt-5-mini, 2025, accessed: 2026- 06-26

    work page 2025

  42. [42]

    Minimax m2.5: Built for real-world pro- ductivity,

    MiniMax, “Minimax m2.5: Built for real-world pro- ductivity,” https://www.minimax.io/news/minimax-m25, 2026, accessed: 2026-06-26

    work page 2026