Pith. sign in

REVIEW 2 major objections 4 minor 40 references

By abstracting past repair trajectories into a multi-level hierarchy and adapting them into issue-specific plans, STAIR lifts coding-agent Pass@1 to 81.2% on a public 500-issue benchmark and transfers to a different agent.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 02:24 UTC pith:2QSZ2JUO

load-bearing objection Solid, novel framework with a credible cross-agent transfer result, but the core ablation conflates abstraction with adaptation and the paper lacks significance testing. the 2 major comments →

arxiv 2607.29658 v1 pith:2QSZ2JUO submitted 2026-07-31 cs.SE

Reusing Past Repairs Through Hierarchical Trajectory Abstraction for Coding Agents

Yisen Xu , Jiayuan Zhou , Ruiqi Pan , Tse-Hsun Chen This is my paper
classification cs.SE
keywords automated program repairLLM agentstrajectory abstractionexperience reusehierarchical abstractionplan adaptationprocedural knowledgePass@1
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that LLM-based repair agents discard the procedural knowledge accumulated in past repair runs, and that this loss is avoidable. STAIR converts successful repair trajectories into a multi-level abstraction tree — from concrete diagnostic steps to general repair strategies — and, for each new issue, retrieves relevant nodes and adapts them into stage-specific plans for localization, planning, and verification. On a public benchmark of 500 real-world issues, STAIR reaches 81.2% Pass@1 with one open-weight LLM and 79.2% with a commercial LLM. The same plans, injected into a structurally different repair agent with no code changes, lift its Pass@1 from 75.8% to 81.0% while slightly reducing token usage. The paper concludes that the structure given to past experience — multi-level abstraction plus issue-specific adaptation — matters more than the agent or model that supplies it.

Core claim

Central claim: past repairs become reusable when organized as a multi-level abstraction tree and adapted per issue. STAIR segments each trajectory by repair stage, then groups consecutive steps and abstracts each group into parent nodes, from low-level sub-goals to high-level principles. For a new issue it retrieves relevant nodes per stage, orders them, and adapts them into an executable, stage-specific plan. On a 500-issue benchmark this yields 81.2% and 79.2% Pass@1 with two LLMs, and transfers to another agent scaffold (75.8% to 81.0%). Single-level abstractions lose 16–22 points; raw trajectories lose 22.4 points.

What carries the argument

The central object is the hierarchical trajectory abstraction tree. Leaf nodes are individual agent steps (thought, action, observation); a grouping operator partitions them by shared procedural intent, and an abstraction operator summarizes each group into a structured parent node with fields for intent, key actions, applicable conditions, and pitfalls. This is applied iteratively until the top two root nodes are reached, producing low-, medium-, and high-level abstractions. Retrieval then scores candidate nodes by textual similarity and an LLM verifier filters them; a plan-generation LLM adapts the ordered guidance into stage-specific executable plans. The hierarchy is what allows the fram

Load-bearing premise

The claim that the multi-level hierarchy itself drives the gains depends on an ablation whose 'raw trajectories' baseline is not described as including the plan-adaptation step, so the 22.4-point gap may reflect missing adaptation rather than missing abstraction.

What would settle it

Run the same 125-instance ablation with four conditions — raw trajectories with and without plan adaptation, and hierarchical abstraction with and without it. If raw-plus-adaptation approaches the full system's 80% resolution, the hierarchy is not the main driver; if the gap persists, abstraction itself is load-bearing.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Repair agents can improve by reusing their own past trajectories as abstracted plans, rather than treating each issue as independent.
  • The plans are agent-agnostic: a different scaffold, with no code changes, gains 5.2 Pass@1 points and slightly reduces token usage.
  • Multi-level abstraction is necessary: any single level, or raw trajectories, drops performance by 16 to 22.4 points on the ablation subset.
  • The framework should extend to any setting with executable validation tests, so the knowledge-reuse structure generalizes beyond one benchmark.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper never states whether the raw-trajectory ablation also ran the plan-adaptation step; if it did not, part of the measured 22.4-point gap could be due to missing adaptation rather than missing abstraction, so the hierarchy's unique contribution needs a two-by-two ablation to be isolated.
  • If the cross-agent transfer holds beyond this benchmark, a shared 'repair memory' distilled from one lineage of agents could be reused to boost many scaffolds, decoupling accumulated experience from the agent that earned it.
  • The manual gain categories — wrong fault location, partial fix, over-modification — are plan-level properties, which suggests a cheaper evaluation protocol: score plans directly on these properties instead of running the full agent.
  • Because raw trajectories transfer so much worse than any abstracted level, a non-hierarchical denoising step might capture part of the benefit; testing that would separate noise filtering from multi-granularity strategy retention.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. STAIR is an offline knowledge-reuse framework for LLM-based program repair. It takes successful historical repair trajectories, splits them into localization, planning, and execution-and-verification stages, and iteratively abstracts each stage into a tree of low-, medium-, and high-level procedural nodes using GPT-5 for grouping and abstraction. For a new issue, STAIR retrieves relevant nodes per stage, adapts them into issue-specific stage plans, and injects those plans into the repair agent's prompt. On SWE-bench Verified, STAIR with the Lingxi scaffold reaches 81.2% Pass@1 with MiniMax M2.5 and 79.2% with GPT-5. The plans also lift mini-SWE-agent v2 from 75.8% to 81.0% without agent-code changes. A stratified 125-instance ablation reports that multi-level abstraction outperforms all single-level variants and that raw trajectories lose 22.4 points relative to the full system.

Significance. If the reported effects are real, the paper makes a strong empirical contribution to automated program repair and LLM-agent experience reuse: it evaluates on the full 500-instance SWE-bench Verified, includes a controlled same-scaffold comparison against Lingxi on the same backbones, demonstrates cross-agent transfer with reduced token usage, and supports the transfer analysis with a manual gain/regression study at Cohen's kappa = 0.85. The headline 81.2% Pass@1 is a notable result. The paper's central mechanistic claim—that the multi-level hierarchy, and separately plan adaptation, cause the gains—is not yet established because the ablation design does not isolate these components, and the absence of statistical inference weakens several comparative claims. These gaps are addressable.

major comments (2)
  1. [§IV, RQ4; Table IV; §III-C] The 'No Abstraction (Raw Trajectories)' variant is defined only as 'directly uses unprocessed trajectory logs without hierarchical abstraction.' The paper never states whether these logs pass through the LLM-based plan-adaptation step described in §III-C. If they do not, Raw Trajectories differs from the Full System in two ways: no abstraction and no adaptation, so the −22.4% drop in Table IV is a joint effect rather than a measure of the hierarchy. The single-level variants appear to retain adaptation, so they do not control for it either. Consequently, the §I contribution bullet claiming that 'both the multi-level hierarchy and plan adaptation contribute to the observed gains' is not established by any reported variant. Please add a raw+adaptation condition and an abstraction-without-adaptation condition, or explicitly state that the raw variant already includes adaptation and provide
  2. [§IV, Tables I–IV] All Pass@1 results come from a single run per configuration, with no confidence intervals, bootstrap estimates, or paired significance tests. For example, Table III reports 33 gained vs. 7 regressed for the mini-SWE-agent transfer (a net of 26 instances on 500), and 43 vs. 10 for STAIR vs. Lingxi; both are immediately testable with McNemar's exact test. The same applies to the RQ4 125-instance ablation. Without such evidence, comparative statements such as 'consistently outperforms' and the conclusion's causal attribution are not statistically supported. This is a fixable reporting gap, but it is load-bearing for the paper's comparative claims.
minor comments (4)
  1. [Table I] Row labels 'STAIR GP T5' and 'STAIR M iniM ax' contain odd spacing; clearly a typesetting issue.
  2. [§IV, RQ1] The statement that MiniMax and GPT-5 are 'comparatively weaker backbones' than Claude 4.5 Opus is asserted without a capability benchmark or same-backbone runs of the top baselines. The same-backbone Lingxi comparisons are the rigorous evidence and should be the primary support for this claim.
  3. [§V-A] Threats to Validity refers to a 'plan compatibility filter' that is not introduced in §III-C; the methodology describes an LLM-based relevance verifier. Please align the terminology.
  4. [§IV-C / reproducibility] No code/artifact link is provided, and implementation details omit retrieval parameters (number of candidates, similarity thresholds, verifier prompt details). Given the complexity of the pipeline, an artifact appendix would materially aid reproducibility.

Circularity Check

0 steps flagged

No circular derivation: STAIR's Pass@1 numbers are measured on SWE-bench Verified, not implied by their inputs. Only a minor non-load-bearing self-citation to Lingxi is present.

full rationale

STAIR is an empirical system paper, not a derivation chain. The headline results (81.2% Pass@1 with MiniMax M2.5, 79.2% with GPT-5, and the 75.8% to 81.0% transfer to mini-SWE-agent v2) are direct measurements on SWE-bench Verified, an external human-curated benchmark with developer-written tests, so they are externally falsifiable rather than forced by construction. There is no fitted parameter later renamed a prediction, no uniqueness theorem imported from prior work, and no definitional identity between an input and an output. The paper does cite Lingxi [11], which overlaps in authorship, and uses Lingxi as the underlying scaffold and as a same-backbone baseline; however, the Lingxi numbers in Table I are produced by running Lingxi in the present evaluation, not taken on faith from the citation, so the self-citation is not load-bearing. The main validity concern in RQ4 is that the 'No Abstraction (Raw Trajectories)' variant is described only as 'directly uses unprocessed trajectory logs without hierarchical abstraction'; the paper does not state whether this variant also performs the LLM plan-adaptation step of Section III-C. If it does not, the -22.4% drop is a joint effect of removing abstraction and adaptation, undermining the contribution claim that 'both the multi-level hierarchy and plan adaptation contribute to the observed gains.' That is a missing-control/threat-to-internal-validity issue, not circularity: the measured outputs are not definitionally equal to the inputs. Similarly, using GPT-5 both to construct abstraction nodes and as the repair agent is a possible same-model confound, but not a self-reference that makes the result true by construction. Hence no circular step meets the evidence bar; the score of 2 reflects only the minor, non-load-bearing self-citation to Lingxi.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The framework rests on a sequence of domain assumptions about stage decomposability and LLM node faithfulness rather than on free parameters in a derivational sense. The main hand-chosen knobs are the hierarchy depth K, the three-level taxonomy, retrieval counts, and the ablation subset design.

free parameters (4)
  • K (root-node stop count) = 2
    Chosen by hand as the stopping criterion for hierarchical abstraction (Section III.B: 'until |L_{ℓ+1}| ≤ K, where K is the number of root nodes (set to 2 in our experiments)'). Not fit, but a design knob that controls tree depth.
  • Number and definition of abstraction levels = 3 (low, medium, high)
    The abstraction levels are fixed to three granularities (Section III.B); single-level ablations use only one of them. The choice and the definitions are hand-specified, and the performance depends on this taxonomy.
  • Ablation subset allocation = 92 medium/hard, 33 easy (125 total)
    The ablation is run on a difficulty-stratified subset rather than the full 500 instances; the 3:1 ratio of non-trivial to trivial is a hand-chosen sampling strategy (Section IV RQ4).
  • Retrieval candidate counts
    The number of candidate nodes retrieved/verified per stage is not specified precisely; it is an implicit tuning choice in the retrieval pipeline (Section III.C), making exact replication difficult.
axioms (5)
  • domain assumption Repair processes decompose into three stages: localization, planning, execution-and-verification
    Section III.A partitions trajectories into these stages citing [27], [28]; if real repairs do not follow this stage structure, the entire abstraction and retrieval design is misaligned.
  • domain assumption LLM-based grouping (G_ℓ) and abstraction (A_ℓ) operators produce faithful, stable, and transferable nodes
    Equations (1)-(2) rely on GPT-5 to group and summarize steps; the paper itself concedes in Section V.A that node quality may vary with trajectory length and complexity. There is no automated fidelity check.
  • domain assumption Pass@1 on SWE-bench Verified measures repair correctness
    Acknowledged in Section V.C: passing the provided tests does not guarantee semantic correctness; this limits the construct validity of the headline metric.
  • domain assumption Cross-paper leaderboard numbers are comparable to the authors' runs
    Table I mixes numbers from other papers (OpenHands, TRAE, Sonar, etc.) with the authors' own runs; Section IV.B claims same settings across 'models and agent scaffolds' but external baselines may use different harnesses, prompts, and infrastructure.
  • domain assumption Historical issues from the same repository created before the target issue provide non-leaked, transferable procedural knowledge
    Section III.A restricts the knowledge pool by repo and timestamp; this assumes temporal ordering is sufficient to avoid leakage and that earlier issues are procedurally relevant to later ones.

pith-pipeline@v1.3.0-daily-deepseek · 16046 in / 15712 out tokens · 144438 ms · 2026-08-03T02:24:49.976273+00:00 · methodology

0 comments
read the original abstract

Although LLM-driven repair agents can tackle complex, repository-level issues, they treat every issue independently and discard the procedural knowledge accumulated from previous repairs. We introduce STAIR, a framework that converts historical repair trajectories into hierarchical, reusable plans that can be adapted to steer future repairs. Each past trajectory is transformed into a multi-level tree that ranges from fine-grained diagnostic actions to high-level repair strategies, encoding experience at several granularities. When a new issue arrives, STAIR selects relevant plan nodes from multiple abstraction levels, tailors them into executable, issue-specific plans, and supplies them to the agent through its prompt. On SWE-bench Verified, STAIR integrated with Lingxi reaches 81.2% Pass@1 using MiniMax M2.5 and 79.2% using GPT-5. The generated plans also generalize across agents: without any code change, they lift the Pass@1 of a structurally different agent, mini-SWE-agent v2, from 75.8% to 81.0%. Ablation experiments further show that mixing multiple abstraction levels surpasses any single level and that raw, unabstracted trajectories transfer substantially worse.

Figures

Figures reproduced from arXiv: 2607.29658 by Jiayuan Zhou, Ruiqi Pan, Tse-Hsun Chen, Yisen Xu.

Figure 1
Figure 1. Figure 1: An overview of STAIR. An LLM-based abstraction operator Aℓ(·) then abstracts each group into a parent node, producing the next level: Lℓ+1 = {Aℓ(Cℓ,j ) | 1 ≤ j ≤ mℓ}. (2) For instance, as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Simplified prompt template for the grouping operator [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Simplified prompt template for the abstraction operator [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of gain categories for STAIRMiniMax and mini-SWE￾agent v2 + Plans. Without STAIR’s plan, the agent cannot recover from this early mistake and all subsequent steps are wasted. The plan guides the agent to the correct location from the start (e.g., fixing the upstream serializer rather than the downstream loader). • Original Fix Covers Only Part of the Affected Code (Partial Fix): the baseline i… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 5 canonical work pages

  1. [1]

    Characteristics of applications software maintenance,

    B. P. Lientz, E. B. Swanson, and G. E. Tompkins, “Characteristics of applications software maintenance,”Commun. ACM, vol. 21, no. 6, pp. 466–471, 1978. [Online]. Available: https://doi.org/10.1145/359511. 359522

    work page doi:10.1145/359511 1978

  2. [3]

    An empirical study on real bug fixes,

    H. Zhong and Z. Su, “An empirical study on real bug fixes,” in 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015, Florence, Italy, May 16-24, 2015, Volume 1, A. Bertolino, G. Canfora, and S. G. Elbaum, Eds. IEEE Computer Society, 2015, pp. 913–923. [Online]. Available: https://doi.org/10.1109/ICSE.2015.101

    work page doi:10.1109/icse.2015.101 2015

  3. [4]

    Stratified case-based reasoning in non-refinable abstraction hierarchies,

    K. Branting, “Stratified case-based reasoning in non-refinable abstraction hierarchies,” inCase-Based Reasoning Research and Development, Second International Conference, ICCBR-97, Providence, Rhode Island, USA, July 25-27, 1997, Proceedings, ser. Lecture Notes in Computer Science, D. B. Leake and E. Plaza, Eds., vol

    1997

  4. [5]

    Case-based reasoning: Foundational issues, methodological variations, and system approaches,

    A. Aamodt and E. Plaza, “Case-based reasoning: Foundational issues, methodological variations, and system approaches,”AI Commun., vol. 7, no. 1, pp. 39–59, 1994. [Online]. Available: https://doi.org/10. 3233/AIC-1994-7104

    1994

  5. [6]

    Swe-agent: Agent-computer interfaces enable automated software engineering,

    J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “Swe-agent: Agent-computer interfaces enable automated software engineering,” inAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, A. Globersons,...

    2024

  6. [7]

    Autocoderover: Autonomous program improvement,

    Y . Zhang, H. Ruan, Z. Fan, and A. Roychoudhury, “Autocoderover: Autonomous program improvement,” inProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, Vienna, Austria, September 16-20, 2024, M. Christakis and M. Pradel, Eds. ACM, 2024, pp. 1592–1604. [Online]. Available: https://doi.org/10.1145/365021...

    arXiv 2024

  7. [8]

    Live-swe-agent: Can software engineering agents self-evolve on the fly?

    C. S. Xia, Z. Wang, Y . Yang, Y . Wei, and L. Zhang, “Live-swe-agent: Can software engineering agents self-evolve on the fly?”arXiv preprint, 2025

    2025

  8. [9]

    Introducing sonar foundation agent,

    H. Ruan, “Introducing sonar foundation agent,” 2025, accessed: Nov. 14, 2025. [Online]. Available: https://www.sonarsource.com/blog/ introducing-sonar-foundation-agent/

    2025

  9. [10]

    Swe-exp: Experience-driven software issue resolution,

    S. Chen, S. Lin, X. Gu, Y . Shi, H. Lian, L. Yun, D. Chen, W. Sun, L. Cao, and Q. Wang, “Swe-exp: Experience-driven software issue resolution,”CoRR, vol. abs/2507.23361, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2507.23361

    doi:10.48550/arxiv.2507.23361 2025

  10. [11]

    Lingxi: Repository-level issue resolution framework enhanced by procedural knowledge guided scaling,

    X. Yang, J. Zhou, M. Pacheco, W. Zhu, P. He, S. Wang, K. Liu, and R. Pan, “Lingxi: Repository-level issue resolution framework enhanced by procedural knowledge guided scaling,”CoRR, vol. abs/2510.11838,

    arXiv

  11. [12]

    EXPEREPAIR: dual-memory enhanced llm-based repository-level program repair,

    F. Mu, J. Wang, L. Shi, S. Wang, S. Li, and Q. Wang, “EXPEREPAIR: dual-memory enhanced llm-based repository-level program repair,”CoRR, vol. abs/2506.10484, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2506.10484

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.10484 2025

  12. [13]

    Planning in a hierarchy of abstraction spaces,

    E. D. Sacerdoti, “Planning in a hierarchy of abstraction spaces,” Artif. Intell., vol. 5, no. 2, pp. 115–135, 1974. [Online]. Available: https://doi.org/10.1016/0004-3702(74)90026-5

    work page doi:10.1016/0004-3702(74)90026-5 1974

  13. [14]

    SWE-bench: Can language models resolve real- world github issues?

    C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan, “SWE-bench: Can language models resolve real- world github issues?” inThe Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https: //openreview.net/forum?id=VTF8yNQM66

    2024

  14. [15]

    Demystifying llm-based software engineering agents,

    C. S. Xia, Y . Deng, S. Dunn, and L. Zhang, “Demystifying llm-based software engineering agents,”Proc. ACM Softw. Eng., vol. 2, no. FSE, pp. 801–824, 2025. [Online]. Available: https://doi.org/10.1145/3715754

    doi:10.1145/3715754 2025

  15. [16]

    Openhands: An open platform for AI software developers as generalist agents,

    X. Wang, B. Li, Y . Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y . Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y . Shao, N. Muennighoff, Y . Zhang, B. Hui, J. Lin, and et al., “Openhands: An open platform for AI software developers as generalist agents,” in The Thirteenth International Conference on Learning Representations, ICLR 2...

    2025

  16. [17]

    Prometheus: Towards long-horizon codebase navigation for repository-level problem solving,

    Y . Pan, Z. Chen, S. Lu, Z. Chu, X. Li, H. Li, Y . Feng, C. L. Goues, F. Sarro, M. Monperrus, and H. Ye, “Prometheus: Towards long-horizon codebase navigation for repository-level problem solving,”

  17. [18]

    Trae agent: An llm-based agent for software engineering with test-time scaling,

    T. R. Team, P. Gao, Z. Tian, X. Meng, X. Wang, R. Hu, Y . Xiao, Y . Liu, Z. Zhang, J. Chen, C. Gao, Y . Lin, Y . Xiong, C. Peng, and X. Liu, “Trae agent: An llm-based agent for software engineering with test-time scaling,” 2025. [Online]. Available: https://arxiv.org/abs/2507.23370

    Pith/arXiv arXiv 2025

  18. [19]

    Expel: LLM agents are experiential learners,

    A. Zhao, D. Huang, Q. Xu, M. Lin, Y . Liu, and G. Huang, “Expel: LLM agents are experiential learners,” inThirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February...

    2024

  19. [20]

    Available: https://arxiv.org/abs/2507.19942

    [Online]. Available: https://arxiv.org/abs/2507.19942

    arXiv

  20. [21]

    Autoguide: Automated generation and selection of context-aware guidelines for large language model agents,

    Y . Fu, D. Kim, J. Kim, S. Sohn, L. Logeswaran, K. Bae, and H. Lee, “Autoguide: Automated generation and selection of context-aware guidelines for large language model agents,” inAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024...

    2024

  21. [22]

    Agent workflow memory,

    Z. Z. Wang, J. Mao, D. Fried, and G. Neubig, “Agent workflow memory,” inForty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, ser. Proceedings of Machine Learning Research, A. Singh, M. Fazel, D. Hsu, S. Lacoste-Julien, F. Berkenkamp, T. Maharaj, K. Wagstaff, and J. Zhu, Eds. PMLR / OpenReview.net, ...

    2025

  22. [23]

    Reflexion: language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: language agents with verbal reinforcement learning,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. H...

    2023

  23. [24]

    Se-agent: Self-evolution trajectory optimization in multi-step reasoning with llm- based agents,

    J. Lin, Y . Guo, Y . Han, S. Hu, Z. Ni, L. Wang, M. Chen, H. Liu, R. Chen, Y . He, D. Jiang, B. Jiao, C. Hu, and H. Wang, “Se-agent: Self-evolution trajectory optimization in multi-step reasoning with llm- based agents,”CoRR, vol. abs/2508.02085, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2508.02085

    doi:10.48550/arxiv.2508.02085 2025

  24. [25]

    When agents go astray: Course-correcting SWE agents with prms,

    S. Gandhi, J. Tsay, J. Ganhotra, K. Kate, and Y . Rizk, “When agents go astray: Course-correcting SWE agents with prms,”CoRR, vol. abs/2509.02360, 2025. [Online]. Available: https://doi.org/10.48550/ arXiv.2509.02360

    doi:10.48550/arxiv.2509.02360 2025

  25. [26]

    On the role of abstraction in case-based reasoning,

    R. Bergmann and W. Wilke, “On the role of abstraction in case-based reasoning,” inAdvances in Case-Based Reasoning, Third European Workshop, EWCBR-96, Lausanne, Switzerland, November 14-16, 1996, Proceedings, ser. Lecture Notes in Computer Science, I. F. C. Smith and B. Faltings, Eds., vol. 1168. Springer, 1996, pp. 28–43. [Online]. Available: https://doi...

    work page doi:10.1007/bfb0020600 1996

  26. [27]

    A critical review on the evaluation of automated program repair systems,

    K. Liu, L. Li, A. Koyuncu, D. Kim, Z. Liu, J. Klein, and T. F. Bissyand´e, “A critical review on the evaluation of automated program repair systems,”J. Syst. Softw., vol. 171, p. 110817, 2021. [Online]. Available: https://doi.org/10.1016/j.jss.2020.110817

    arXiv 2021

  27. [28]

    Self-abstraction from grounded experience for plan-guided policy refinement,

    H. Hayashi, B. Pang, W. Zhao, Y . Liu, A. Gokul, S. Bansal, C. Xiong, S. Yavuz, and Y . Zhou, “Self-abstraction from grounded experience for plan-guided policy refinement,”CoRR, vol. abs/2511.05931, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2511.05931

    doi:10.48550/arxiv.2511.05931 2025

  28. [29]

    Openai GPT-5 system card,

    OpenAI, “Openai GPT-5 system card,”CoRR, vol. abs/2601.03267,

    Pith/arXiv arXiv

  29. [30]

    Minimax m2.5,

    Minimax, “Minimax m2.5,” 2026, accessed: Feb. 12, 2026. [Online]. Available: https://www.minimax.io/news/minimax-m25/

    2026

  30. [31]

    Langgraph,

    L. Inc., “Langgraph,” 2024, accessed: 2024-12-02. [Online]. Available: https://langchain-ai.github.io/langgraph/

    2024

  31. [32]

    A coefficient of agreement for nominal scales,

    J. Cohen, “A coefficient of agreement for nominal scales,”Educational and psychological measurement, vol. 20, no. 1, pp. 37–46, 1960

    1960

  32. [33]

    Automated program repair,

    C. Le Goues, M. Pradel, and A. Roychoudhury, “Automated program repair,”Commun. ACM, vol. 62, no. 12, pp. 56–65, 2019. [Online]. Available: https://doi.org/10.1145/3318162

    doi:10.1145/3318162 2019

  33. [34]

    Is the cure worse than the disease? overfitting in automated program repair,

    E. K. Smith, E. T. Barr, C. Le Goues, and Y . Brun, “Is the cure worse than the disease? overfitting in automated program repair,” in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, Bergamo, Italy, August 30 - September 4, 2015, E. D. Nitto, M. Harman, and P. Heymans, Eds. ACM, 2015, pp. 532–543. [Online]....

    arXiv 2015

  34. [35]

    Overfitting in semantics-based automated program repair,

    X. D. Le, F. Thung, D. Lo, and C. Le Goues, “Overfitting in semantics-based automated program repair,” inProceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018, M. Chaudron, I. Crnkovic, M. Chechik, and M. Harman, Eds. ACM, 2018, p. 163. [Online]. Available: https://doi.org/10.1145/3...

    arXiv 2018

  35. [39]

    The strength of random search on automated program repair,

    Y . Qi, X. Mao, Y . Lei, Z. Dai, and C. Wang, “The strength of random search on automated program repair,” in36th International Conference on Software Engineering, ICSE ’14, Hyderabad, India - May 31 - June 07, 2014, P. Jalote, L. C. Briand, and A. van der Hoek, Eds. ACM, 2014, pp. 254–265. [Online]. Available: https://doi.org/10.1145/2568225.2568254

    arXiv 2014

  36. [1266]

    Springer, 1997, pp. 519–530. [Online]. Available: https: //doi.org/10.1007/3-540-63233-6 521

    work page doi:10.1007/3-540-63233-6 1997

  37. [2023]

    Available: http://papers.nips.cc/paper files/paper/2023/ hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html

    [Online]. Available: http://papers.nips.cc/paper files/paper/2023/ hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html

    2023

  38. [2024]

    Available: http://papers.nips.cc/paper files/paper/2024/ hash/d8efbb5dd415974eb095c3f06bff1f48-Abstract-Conference.html

    [Online]. Available: http://papers.nips.cc/paper files/paper/2024/ hash/d8efbb5dd415974eb095c3f06bff1f48-Abstract-Conference.html

    2024

  39. [2025]

    Available: https://doi.org/10.48550/arXiv.2510.11838

    [Online]. Available: https://doi.org/10.48550/arXiv.2510.11838

    doi:10.48550/arxiv.2510.11838

  40. [2026]

    Available: https://doi.org/10.48550/arXiv.2601.03267

    [Online]. Available: https://doi.org/10.48550/arXiv.2601.03267

    Pith/arXiv arXiv doi:10.48550/arxiv.2601.03267