Pith. sign in

REVIEW 4 major objections 3 minor 50 references

An 8B model trained to compare candidate fix proposals and synthesize a consolidated 'golden proposal' outperforms a frontier closed model at proposal selection on a manager benchmark, and lifts end-to-end patch success by 7.1% in a three-r

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:17 UTC pith:BEHOOH6O

load-bearing objection Solid Manager benchmark result; the P2A end-to-end claim is confounded and needs controls or toning down. the 4 major comments →

arxiv 2601.22956 v2 pith:BEHOOH6O submitted 2026-01-30 cs.SE

SWE-Manager: Selecting and Synthesizing Golden Proposals Before Coding

Boyin Tan , Haoning Deng , Junyuan Zhang , Junjielong Xu , Pinjia He , Youcheng Sun This is my paper
classification cs.SE
keywords proposal selectiongolden proposalreinforcement learningreasoning modelsoftware engineeringissue resolutionpatch generationdecision making
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Before writing a patch, human maintainers often choose among several proposed fixes, weighing risk, depth, and fit. The paper treats this selection as a learnable reasoning task and trains an 8B model to pick the best candidate and also to synthesize a 'golden proposal' that merges the candidates' strengths. On the Manager benchmark, the trained model selects the maintainer-matched proposal more often and earns more of the available reward than a frontier closed model. In an end-to-end workflow with separate proposal, review, and implementation stages, the golden proposal yields a 55.6% patch pass rate, matching the frontier model and beating the strongest single-agent baseline by 7.1%. The paper's central point is that proposal selection and consolidation, not just code generation, are learnable and consequential.

Core claim

The paper's central claim is that proposal selection is a decision problem that can be taught to a small model through outcome-aligned reinforcement learning, and that requiring the model to also write a consolidated golden proposal improves selection. Given an issue and a set of candidate proposals, SWE-Manager-8B emits a reasoning trace, a selected proposal ID, a justification, and a golden proposal. It reports 53.21% match rate and 57.75% earned rate on the Manager benchmark, surpassing a frontier closed model by 9.06% in match and 20.00% in earned rate. In the proposed three-role workflow, the synthesized golden proposal achieves a 55.6% pass rate on the IC benchmark, tying the frontier

What carries the argument

The load-bearing mechanism is the golden-proposal objective: the model must consolidate complementary strengths from competing candidates into a single actionable plan, which forces explicit cross-proposal comparison instead of shallow index prediction. The training reward combines a verifiable selection signal with softer similarity-based rewards for reasoning, justification, and golden-proposal quality, optimized with a reinforcement-learning algorithm using decoupled clipping and dynamic sampling after supervised fine-tuning. An ablation shows that a variant trained only to output the selected ID underperforms, supporting the claim that synthesis drives selection quality.

Load-bearing premise

The end-to-end gain is credited to the synthesized golden proposal, but the experiments never compare against giving the implementer the raw selected proposal or a generic structured plan, so a simpler 'any plan helps' effect could explain the improvement.

What would settle it

Run the three-role workflow again while handing the implementation agent (a) the selected raw proposal instead of the golden proposal, and (b) a generic structured plan not derived from the candidates; if either condition keeps the pass rate near 55.6%, golden-proposal synthesis is not the driver.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • An 8B parameter model can outperform much larger closed models at proposal selection, so scale is not the only route to good managerial judgment.
  • Requiring golden-proposal synthesis during training improves selection accuracy over index-only training, making the consolidation objective a reusable design choice.
  • The three-role workflow solves issues that none of the candidate-proposal sources solves alone, indicating that merging imperfect plans can substitute for a single perfect plan.
  • Selection accuracy and robustness drop when the candidate pool exceeds three proposals, because training data is concentrated on smaller pools; this identifies a concrete scaling limit.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • (editorial inference) The golden-proposal step may be acting as implicit ensembling: its value could come from averaging multiple imperfect plans rather than from selection per se; a direct test would compare against an averaged or generic plan.
  • (editorial inference) The P2A improvement could be driven by the extra planning stage itself rather than by comparison-based synthesis; a control that hands the implementer the selected raw proposal would settle this.
  • (editorial inference) The same select-and-consolidate objective could transfer to other design decisions, such as architecture selection or dependency choice, though the paper only tests fix proposals.
  • (editorial inference) Because the manager never reads code, a tool-augmented variant that checks repository facts before deciding might recover the high-value cases where the agent baseline wins.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper studies proposal selection before code changes. It first presents a manual study of 339 GitHub issue discussions, coding maintainers' rationales into four themes (risk/safety, fix depth, maintainability, socio-technical factors). It then introduces SWE-Manager-8B, an 8B model trained from Qwen3-8B via SFT followed by DAPO reinforcement learning. The model takes an issue and several candidate proposals as input and outputs a reasoning trace, a selected proposal ID, a justification, and a synthesized 'golden proposal.' On the SWE-Lancer Manager benchmark, it reports 53.21% match rate and 57.75% earned rate ($152,750), surpassing GPT-5-based selectors. To test downstream value, the paper proposes P2A, a three-role framework (Proposal Agent, Technical Manager, Implementation Agent); with SWE-Manager-8B as the Technical Manager it reports a 55.6% pass rate on SWE-Lancer IC (110/198 issues), matching P2A with GPT-5 and improving over the strongest mini-SWE-agent baseline by 7.1%.

Significance. The Manager benchmark result is externally grounded and, if correct, is a meaningful demonstration that a small open-weight model can beat a frontier closed model on a real-world selection task with verifiable labels. The authors also release code and data, and the manual-study taxonomy of selection rationales is a useful contribution in itself. However, the end-to-end P2A contribution is currently under-supported: the comparison changes multiple system components at once and lacks the controls needed to attribute the 7.1% improvement to proposal selection and golden-proposal synthesis. The Manager result is less affected by this confound, so the overall work is a candidate for major revision rather than rejection.

major comments (4)
  1. [§4.5, Table 4, Figure 5] The central end-to-end claim—that golden-proposal synthesis drives the 7.1% pass-rate improvement (96 to 110 issues) over mini-SWE-agent—is confounded. Relative to the baseline, P2A changes two things at once: a Technical Manager produces an intermediate plan (the selected proposal plus a synthesized golden proposal), and the Implementation Agent is switched from autonomous exploration to plan-following behavior. No control feeds the selected candidate proposal directly to the Implementation Agent, and no control uses a generic/non-comparative structured plan. If any structured plan—or simply selecting one of the three candidate proposals—produces roughly 110 solves, the paper's attribution to SWE-Manager's RL-trained synthesis is unsupported. Additionally, Table 4 shows that P2A with SWE-Manager-8B earns $69,875 versus $70,875 for the strongest mini-SWE-agent baseline, so in reward term
  2. [§4.5, Figure 5(c)] The union analysis in Figure 5(c) weakens rather than supports the consolidation narrative. The figure panel is labeled 'P2A (GPT-5) vs Union,' yet the surrounding text uses its 16 P2A-only successes to argue that 'consolidating their proposals into a golden proposal enables successful downstream implementation' for the P2A framework generally. Moreover, the depicted relation shows 38 Union-only instances that P2A misses while adding only 16 P2A-only successes; the union of the three proposal-source models solves 132 issues, whereas P2A solves 110. The paper needs to report the SWE-Manager-specific union comparison and explain why the proposed consolidation mechanism is beneficial when it is 22 solved instances worse than simply taking any of the three proposal sources' outputs.
  3. [§3.1.4, Eq. (1)] The RL reward is not fully specified. The three soft terms r_think, r_justi, and r_gold are described only as 'normalized similarity-based scores,' with no definition of the similarity metric, the reference representation, or the normalization procedure. Since these three terms constitute 60% of the reward (weights 0.2 each), the training objective cannot be reproduced from the paper. Furthermore, the supervision targets (reasoning traces, justifications, and golden proposals) are generated by GPT-5, which is also used as the strongest zero-shot baseline in Table 3. The selection label is externally anchored, so the headline Manager result is not circular, but the claim that gains come from 'decision-aligned rewards' is hard to separate from a reward that partly encourages similarity to GPT-5 outputs. Please specify the similarity metric, release the reward computation code, and ideally
  4. [§5.1] The internal-validity argument for proposal guidance is numerically inconsistent. The text states that '31.82% of P2A's successful cases fall outside the baseline,' but Figure 5(b) shows 31 P2A-only successes out of 110, which is 28.18%. The 14.55% union-outsider figure (16/110) is consistent. This discrepancy should be corrected. More importantly, overlap analysis alone does not establish attribution; it must be paired with the control conditions described above.
minor comments (3)
  1. [§4.5, Figure 5] The figure caption and the main text use inconsistent panel labels. Panel (c) is drawn as 'P2A (GPT-5) vs Union,' but the text in §4.5 refers to 'P2A' without specifying which selector; clarify whether the union analysis is for GPT-5, SWE-Manager-8B, or both, and adjust the narrative accordingly.
  2. [§3.2] The sentence 'This role-based decomposition allows us to isolate and rigorously assess the contribution of proposal selection' overstates what the experiments show. Given the confounds in the P2A comparison, the decomposition does not currently isolate the selection contribution.
  3. [§2.2, Table 1] The manual study is described as a sample of 339 issues from a population of 2,852, but the relationship between this study and the training set used in §3.1.2 is not fully clear. Explain whether the same 2,852-issue curation is used for SFT/RL data and for the manual taxonomy, and whether benchmark instances were removed from the training set as stated in §5.2.1.

Circularity Check

0 steps flagged

No significant circularity; central Manager result is anchored to external ground-truth labels, and the P2A confound is a validity threat, not a definitional reduction.

full rationale

The paper's headline Manager-benchmark result is not circular: selection accuracy is measured against external ground-truth labels (maintainer-merged proposals), and the training signal (SFT targets and RL rewards) is generated from those labels plus GPT-5 rationales conditioned on the labels. The evaluation is independent of the training signal, so the 53.21% match rate is not equivalent to an input by construction. The RL reward in Eq. 1 gives 60% weight to similarity to GPT-5-generated reasoning, justification, and golden-proposal targets, but these similarity scores are training objectives, not evaluation metrics; the downstream P2A pass rate is test-based and external, so the teacher-student relationship with GPT-5 does not reduce the claim to its inputs. The P2A comparison does suffer from a real confound: relative to the mini-SWE-agent baseline, P2A adds both a planning/review stage and switches the implementation agent to plan-following behavior, so the 7.1% improvement cannot be unambiguously attributed to the golden proposal. The paper itself acknowledges this coupling in Section 5.1 ('it couples the model's proposal-synthesis quality with the implementation agent's ability to implement the plan, which can confound attribution'), but this is a causal-identification problem, not a circularity. Self-citations in the paper (e.g., refs [9], [38]/[39], [40]) appear only in methodology and related-work contexts and are not load-bearing; there is no imported uniqueness theorem, no ansatz smuggled in via self-citation, and no fitted parameter renamed as a prediction. Overall, the central derivation chain is self-contained against external benchmarks, so the circularity score is low.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The central claim relies on the external validity of the SWE-Lancer Manager labels, on the quality of GPT-5-generated training targets, on the undefined soft reward metrics, and on the representative nature of a 339-issue manual coding effort. The P2A claim further rests on the assumption that the golden proposal, not the extra planning stage, causes the observed pass-rate improvement. No new physical or mathematical entities are introduced; the 'golden proposal' is an output artifact, not an invented entity.

free parameters (3)
  • Reward weights in Eq. (1) = 0.4 / 0.2 / 0.2 / 0.2
    Chosen by hand with no sensitivity analysis; they control the balance between the verifiable selection reward and the three soft similarity rewards, and directly shape the trained policy.
  • Similarity scoring for soft rewards = Unspecified
    r_think, r_justi, and r_gold are called normalized similarity-based scores, but the exact metric (cosine, BERTScore, LLM-as-judge, etc.) is never defined, making the training objective unverifiable and unreproducible.
  • P2A proposal-pool size = 3
    The end-to-end evaluation fixes the candidate pool at three proposals. The authors' own robustness analysis shows SWE-Manager degrades for larger pools, so this design choice may favor the method relative to settings with more candidates.
axioms (6)
  • domain assumption SWE-Lancer Manager ground-truth proposal IDs correctly reflect the maintainer's pre-implementation choice.
    Used as the external label for training and evaluation (Section 4.1). If the ground truth is in fact a retrospective match to the merged code, the selection accuracy measures alignment with the final diff rather than with a manager's deliberation.
  • ad hoc to paper GPT-5-generated reasoning traces, justifications, and golden proposals are valid supervision targets.
    Section 3.1.2 uses GPT-5 to create all structured training targets. The quality of these targets is taken for granted, and the same model is later used as a zero-shot baseline, creating an asymmetry.
  • ad hoc to paper The 'normalized similarity-based' soft rewards are valid proxies for reasoning, justification, and golden-proposal quality.
    Eq. (1) gives these terms 60% of the total reward, but the scoring functions are not specified, so the optimization target is not fully defined.
  • domain assumption A good proposal can be identified from issue text and proposal text alone, without executing code or running tests.
    The whole Manager formulation excludes code execution (Section 1, Figure 1). Real maintainers sometimes run experiments, so a text-only selection policy may miss empirical evidence that would change the choice.
  • standard math The manually sampled 339 issues are representative of the 2,852-issue population for coding selection rationales.
    The Krejcie-Morgan sample size formula with ±5% margin and 95% confidence yields 339 (Section 2.1). This assumes simple random sampling and honest, consistent annotation by the first two authors.
  • domain assumption There is no train/benchmark leakage because candidate proposals contain little concrete code.
    Section 5.2.1 argues that structured natural-language proposals are unlikely to encode implementation details from benchmark issues, but this is asserted rather than measured.

pith-pipeline@v1.3.0-alltime-deepseek · 20758 in / 14624 out tokens · 149003 ms · 2026-08-03T06:17:29.789470+00:00 · methodology

0 comments
read the original abstract

Large language model (LLM) research in software engineering has largely focused on tasks such as code generation and bug repair. In practice, teams often draft multiple candidate proposals for fixing an issue and then deliberate on one golden proposal for implementation. This selection requires not only assessing the issue's scope, impact, and urgency, but also a clear understanding of each proposal's strengths and weaknesses. A good selection could make issue resolution more reliable while reducing regression and operational risk, whereas a poor choice can increase risk and even cause unpredictable failures. We first conduct a manual study of real-world issues to characterize the rationales maintainers use when selecting among competing proposals. Motivated by these findings, we introduce SWE-Manager, a joint selection and synthesis approach that selects the best proposal and synthesizes a golden proposal. SWE-Manager is an 8B model trained via reinforcement learning (RL) to compare proposals, justify its choice, and synthesize a golden proposal for implementation. We view proposal selection as a reasoning task, mirroring how technical managers review competing proposals by weighing issue context and each proposal's solution without executing code or running tests. On the SWE-Lancer Manager benchmark, SWE-Manager achieves 53.21 selection accuracy and 57.75 earn rate, earning 152,750 dollars and outperforming strong baselines including GPT-5. To further evaluate the effectiveness of SWE-Manager in real-world issue resolution, we design the P2A framework, which simulates a real-world workflow where multiple proposals are drafted, reviewed, and a golden proposal is selected for implementation ...

Figures

Figures reproduced from arXiv: 2601.22956 by Boyin Tan, Haoning Deng, Junjielong Xu, Junyuan Zhang, Pinjia He, Youcheng Sun.

Figure 1
Figure 1. Figure 1: Manager task overview. A real GitHub issue is paired with multiple independently written candidate [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SWE-Manager methodology overview. We formulate proposal selection as taking an issue and its candidate proposals as input, and producing a reasoning trace, a selected proposal ID, a selection justification, and a synthesized golden proposal as output. Then, we curate GitHub issues paired with multiple candidate proposals and a maintainer-selected best-proposal ID. Conditioned on the issue, proposals, and t… view at source ↗
Figure 3
Figure 3. Figure 3: P2A comprises three roles: Proposal Agent, Technical Manager, and Implementation Agent. The Proposal Agent samples multiple candidate proposals; each proposal includes an issue restatement, a root-cause analysis, and a proposed solution. The Technical Manager compares proposals and decides on the final proposal for implementation. SWE-Manager plays a Technical Manager role in P2A. Finally, the Implementati… view at source ↗
Figure 4
Figure 4. Figure 4: Left: counts of correct selections and errors grouped by the number of candidate proposals per issue. Right: counts of correct selections and errors grouped by the task reward money. The figure highlights how performance varies with candidate-set size and reward level. As the number of candidate proposals grows, the performance of each method drops. SWE-Manager achieves the strongest performance across mos… view at source ↗
Figure 5
Figure 5. Figure 5: Overlap of solved SWE-Lancer IC instances. Panels (a) and (b) show that P2A and mini-SWE-agent [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of proposal-pool size on model robustness. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation study of SWE-Manager-8B. We compare three training variants: [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 2 canonical work pages

  1. [1]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, John Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Chris Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, E Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, J Landau, Kamal Ndousse, Kamil˙e Lukoši¯ut˙e...

  2. [2]

    Christoph Becker, Dawn Walker, and Curtis McCord. 2017. Intertemporal Choice: Decision Making and Time in Software Engineering. In2017 IEEE/ACM 10th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE). 23–29. doi:10.1109/CHASE.2017.6

    work page doi:10.1109/chase.2017.6 2017

  3. [3]

    Clements

    Paul C. Clements. 2007. An Economic Model for Software Architecture Decisions. In2007 First International Workshop on the Economics of Software and Computation. 1–1. doi:10.1109/ESC.2007.2

    work page doi:10.1109/esc.2007.2 2007

  4. [4]

    José Adson O. G. da Cunha, Fabio Queda Bueno da Silva, Hermano Perrelli de Moura, and Francisco J. S. Vasconcellos

  5. [5]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiaoling Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bing-Li Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Dama...

    Pith/arXiv arXiv 2025

  6. [6]

    Zheng Gao, Christian Bird, and Earl T. Barr. 2017. To Type or Not to Type: Quantifying Detectable Bugs in JavaScript.2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE)(2017), 758–769. https: //api.semanticscholar.org/CorpusID:11255188

    2017

  7. [7]

    Ali Ghanbari and Andrian Marcus. 2022. Patch correctness assessment in automated program repair based on the impact of patches on production and test code. InProceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis(Virtual, South Korea)(ISSTA 2022). Association for Computing Machinery, New York, NY, USA, 654–665. doi:10...

    arXiv 2022

  8. [8]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence.ArXivabs/2401.14196 (2024). https://api.semanticscholar.org/CorpusID:267211867

    Pith/arXiv arXiv 2024

  9. [9]

    Pinjia He, Zhuangbin Chen, Shilin He, and Michael R. Lyu. 2018. Characterizing the Natural Language Descriptions in Software Logging Statements.2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE) (2018), 178–189. https://api.semanticscholar.org/CorpusID:52068379

    2018

  10. [10]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2025. A Survey on Large Language Models for Code Generation.ACM Trans. Softw. Eng. Methodol.(July 2025). doi:10.1145/3747588 Just Accepted

    doi:10.1145/3747588 2025

  11. [11]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?ArXivabs/2310.06770 (2023). https://api. semanticscholar.org/CorpusID:263829697

    Pith/arXiv arXiv 2023

  12. [12]

    René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: a database of existing faults to enable controlled testing studies for Java programs. InInternational Symposium on Software Testing and Analysis. https://api.semanticscholar. org/CorpusID:12796895

    2014

  13. [13]

    Kazman, M

    R. Kazman, M. Klein, M. Barbacci, T. Longstaff, H. Lipson, and J. Carriere. 1998. The architecture tradeoff analysis method. InProceedings. Fourth IEEE International Conference on Engineering of Complex Computer Systems (Cat. No.98EX193). 68–78. doi:10.1109/ICECCS.1998.706657 20 Trovato et al

    arXiv 1998

  14. [14]

    Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. 2022. The Stack: 3 TB of permissively licensed source code.Preprint(2022)

    2022

  15. [15]

    Krejcie and Daryle W

    Robert V. Krejcie and Daryle W. Morgan. 1970. Determining Sample Size for Research Activities.Educational and Psychological Measurement30 (1970), 607 – 610. https://api.semanticscholar.org/CorpusID:143169474

    1970

  16. [16]

    Ajay Mahaputra Kumar and Kamaldeep Kaur. 2022. MCDM- Based Framework to Solve Decision Making Problems in Software Engineering.2022 3rd International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT)(2022), 1–5. https://api.semanticscholar.org/CorpusID:257658432

    2022

  17. [17]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

  18. [18]

    Martini, and Francesca Arcelli Fontana

    Valentina Lenarduzzi, Terese Besker, Davide Taibi, A. Martini, and Francesca Arcelli Fontana. 2021. A systematic literature review on Technical Debt prioritization: Strategies, processes, factors, and tools.J. Syst. Softw.171 (2021), 110827. https://api.semanticscholar.org/CorpusID:225128141

    2021

  19. [19]

    Jiaye Lin, Yifu Guo, Yuzhen Han, Sen Hu, Ziyi Ni, Licheng Wang, Mingguang Chen, Hongzhang Liu, Ronghao Chen, Yangfan He, Daxin Jiang, Binxing Jiao, Chen Hu, and Huacan Wang. 2025. SE-Agent: Self-Evolution Tra- jectory Optimization in Multi-Step Reasoning with LLM-Based Agents.ArXivabs/2508.02085 (2025). https: //api.semanticscholar.org/CorpusID:280422229

    arXiv 2025

  20. [20]

    Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Jian Zhu, Terry Yue Zhuo, Evgenii Z...

    Pith/arXiv arXiv 2024

  21. [21]

    Magabaleh, Lana L

    Aws A. Magabaleh, Lana L. Ghraibeh, Afnan Y. Audeh, Ahmed Shihab Albahri, Muhammet Deveci, and Jurgita Antuchevičien˙e. 2024. Systematic review of software engineering uses of multi-criteria decision-making methods: Trends, bibliographic analysis, challenges, recommendations, and future directions.Appl. Soft Comput.163 (2024), 111859. https://api.semantic...

    2024

  22. [22]

    Samuel Miserendino, Michele Wang, Tejal Patwardhan, and Johannes Heidecke. 2025. SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?ArXivabs/2502.12115 (2025). https: //api.semanticscholar.org/CorpusID:276421817

    Pith/arXiv arXiv 2025

  23. [23]

    OpenAI. [n. d.].GPT-5 is here. https://openai.com/gpt-5/

  24. [24]

    Albert Örwall. 2025. Moatless Tools. https://github.com/aorwall/moatless-tools. GitHub repository. Version 0.0.2 (released 2025-06-07). Accessed 2026-01-14

    2025

  25. [25]

    Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. 2022. Training language models to follow instructions ...

    Pith/arXiv arXiv 2022

  26. [26]

    Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. 2025. Training Software Engineering Agents and Verifiers with SWE-Gym. InForty-second International Conference on Machine Learning. https://openreview.net/forum?id=Cq1BNvHx74

    2025

  27. [27]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2019. ZeRO: Memory optimizations Toward Training Trillion Parameter Models.SC20: International Conference for High Performance Computing, Networking, Storage and Analysis(2019), 1–16. https://api.semanticscholar.org/CorpusID:269617042

    2019

  28. [28]

    Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, I. Evtimov, Joanna Bitton, Manish P Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre D’efossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Sci...

    Pith/arXiv arXiv 2023

  29. [29]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Jun-Mei Song, Mingchuan Zhang, Y. K. Li, Yu Wu, and Daya Guo

  30. [30]

    Smrithi Rekha V and Henry Muccini. 2018. Group decision-making in software architecture: A study on industrial practices.Inf. Softw. Technol.101 (2018), 51–63. https://api.semanticscholar.org/CorpusID:49384683

    2018

  31. [31]

    Hans van Vliet and Antony Tang. 2016. Decision making in software architecture.J. Syst. Softw.117 (2016), 638–644. https://api.semanticscholar.org/CorpusID:4092090

    2016

  32. [32]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. 2025. OpenHands: An Open Platform for A...

    2025

  33. [33]

    Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriele Synnaeve, Rishabh Singh, and Sida Wang. 2025. SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution.ArXivabs/2502.18449 (2025). https://api.semanticscholar.org/CorpusID:276580226

    Pith/arXiv arXiv 2025

  34. [34]

    Chun Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. Agentless: Demystifying LLM-based Software Engineering Agents.ArXivabs/2407.01489 (2024). https://api.semanticscholar.org/CorpusID:270870279

    Pith/arXiv arXiv 2024

  35. [35]

    Chun Xia and Lingming Zhang. 2023. Automated Program Repair via Conversation: Fixing 162 out of 337 Bugs for $0.42 Each using ChatGPT.Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis(2023). https://api.semanticscholar.org/CorpusID:257913714

    2023

  36. [36]

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2025. Demystifying LLM-Based Software Engineering Agents.Proc. ACM Softw. Eng.2, FSE, Article FSE037 (June 2025), 24 pages. doi:10.1145/3715754

    doi:10.1145/3715754 2025

  37. [37]

    Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, and Kai Chen. 2025. SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution.ArXivabs/2501.05040 (2025). https://api. semanticscholar.org/CorpusID:275405540

    Pith/arXiv arXiv 2025

  38. [38]

    Junjielong Xu, Ying Fu, Shin Hwei Tan, and Pinjia He. 2024. Aligning the Objective of LLM-Based Program Repair.2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE)(2024), 2548–2560. https://api.semanticscholar. org/CorpusID:269148552

    2024

  39. [40]

    Junjielong Xu, Boyin Tan, Xiaoyuan Liu, Chao Peng, Pengfei Gao, and Pinjia He. 2025. Scalable Supervising Software Agents with Patch Reasoner. arXiv:2510.22775 [cs.CL] https://arxiv.org/abs/2510.22775

    arXiv 2025

  40. [41]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxin Yang, Jingren Zhou, Jingren Zhou, Junyan Lin, Kai Dang, Keqin Bao, Ke-Pei Ya...

    Pith/arXiv arXiv 2025

  41. [42]

    Yang, Sophia Kolak, Vincent Hellendoorn, Ruben Martins, and Claire Le Goues

    Aidan Z.H. Yang, Sophia Kolak, Vincent Hellendoorn, Ruben Martins, and Claire Le Goues. 2025. Revisiting Unnatu- ralness for Automated Program Repair in the Era of Large Language Models. In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). 2561–2573. doi:10.1109/ICSE55347.2025.00089

    arXiv 2025

  42. [43]

    John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R Narasimhan, and Ofir Press

  43. [44]

    John Yang, Kilian Lieret, Carlos E Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. 2025. SWE-smith: Scaling Data for Software Engineering Agents. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. https: //openreview.net/forum?id=63iVrXc8cC

    2025

  44. [45]

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Honglin Yu, Weinan Dai, Yuxuan Song, Xiang Wei, Haodong Zhou, Jingjing Liu, ...

    Pith/arXiv arXiv 2025

  45. [46]

    Linghao Zhang, Shilin He, Chaoyun Zhang, Yu Kang, Bowen Li, Chengxing Xie, Junhao Wang, Maoquan Wang, Yufan Huang, Shengyu Fu, Elsie Nallipogu, Qingwei Lin, Yingnong Dang, Saravan Rajmohan, and Dongmei Zhang. 2025. SWE-bench Goes Live!. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and 22 Trovato et al. Benchmarks ...

    2025

  46. [47]

    InThe Thirty-eighth Annual Conference on Neural Information Processing Systems

    SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems. https://arxiv.org/abs/2405.15793

    Pith/arXiv arXiv

  47. [51]

    Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, Wenmeng Zhou, and Yingda Chen. 2024. SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning. arXiv:2408.05517 [cs.CL] https://arxiv.org/abs/2408.05517

    Pith/arXiv arXiv 2024

  48. [2016]

    https://api.semanticscholar

    Decision-Making in Software Project Management: A Qualitative Case Study of a Private Organization.2016 IEEE/ACM Cooperative and Human Aspects of Software Engineering (CHASE)(2016), 26–32. https://api.semanticscholar. org/CorpusID:16671994

    2016

  49. [2022]

    https://api.semanticscholar

    Constitutional AI: Harmlessness from AI Feedback.ArXivabs/2212.08073 (2022). https://api.semanticscholar. org/CorpusID:254823489 SWE-Manager : Selecting and Synthesizing Golden Proposals Before Coding 19

    Pith/arXiv arXiv 2022

  50. [2024]

    https://api.semanticscholar.org/CorpusID:267412607 SWE-Manager : Selecting and Synthesizing Golden Proposals Before Coding 21

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.ArXivabs/2402.03300 (2024). https://api.semanticscholar.org/CorpusID:267412607 SWE-Manager : Selecting and Synthesizing Golden Proposals Before Coding 21

    Pith/arXiv arXiv 2024