Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GitGoodBench presents a benchmark that measures AI agents on Git workflows—merge conflict resolution, interactive rebase, and iterative committing—and a baseline model solves only 21.11% of the prototype tasks.

desk verdict First real end-to-end Git-agent benchmark with released data; useful resource, but the LLM-judged solve rates for IR/ICC are provisional until validated. read the letter →

arxiv 2505.22583 v1 pith:Z5G3RV2U submitted 2025-05-28 cs.SE cs.AI

Tobias Lindenbauer , Egor Bogomolov , Yaroslav Zharov This is my paper · ORCID
classification cs.SEcs.AI
keywords benchmarkGitAIagentsversioncontrolmergeconflictresolutioninteractiverebaseLLM-as-a-judgesoftwareengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GitGoodBench is a benchmark that tests AI agents on version-control workflows rather than just code fixing. It covers three scenarios extracted from open-source Python, Java, and Kotlin repositories: resolving merge conflicts, rewriting commit history with interactive rebase, and turning scattered changes into a clean sequence of commits. The benchmark provides a 900-sample full evaluation set, a 120-sample rapid-prototyping set, and a 17,469-sample training corpus. A baseline agent built on a frontier language model with custom Git tools solves 21.11% of the prototype tasks, showing that Git competence is a largely unsolved part of software-engineering automation. If the benchmark is right, evaluating agents on Git tasks will be a necessary step toward comprehensive software-engineering agents.

What carries the argument

The central objects are the File-Commit Chain (a sequence of consecutive commits modifying a single file, used as a heuristic to find histories that could be improved) and the LLM-as-a-Judge evaluation protocol (an LLM compares agent-generated and ground-truth histories twice with positions swapped, scoring on commit-message quality, cohesion, progression, and commit size). These support the two FCC-based scenarios; Merge Conflict Resolution instead uses exact match against the ground-truth merge commit. The benchmark also provides three datasets and a difficulty heuristic for merges based on the number of conflicts and their spread across files.

What would settle it

Take a random sample of the 60 FCC samples from the Lite set and have the two histories (agent-generated and ground truth) rated by experienced developers; if the LLM judge agrees with the human majority in fewer than roughly 70% of cases, or if agreement varies strongly with the choice of judge model, the reported 26.67% and 23.33% solve rates are not a stable measure of Git competence.

Watch

Extended reading notes

Core claim

The paper claims to introduce the first benchmark that evaluates AI agents on multiple end-to-end Git scenarios, rather than on isolated sub-tasks. In the Merge Conflict Resolution scenario the agent must reproduce the ground-truth merge commit when given a set of conflicts; in Interactive Rebase it must produce an improved history from existing commits; in Iterative Committing of Changes it must turn a disorganized set of changes into a coherent history. The benchmark is built by mining repositories with permissive licenses, using File-Commit Chains as a heuristic for histories that could be improved, and stratifying samples by scenario type, language, and difficulty. Baseline results with a GPT-4o-based system equipped with custom tools show a 21.11% solve rate overall, with exact-match scoring for conflicts and LLM-as-a-judge scoring for histories; performance drops sharply on Merge Conflict Resolution samples as difficulty increases, with a 0% solve rate on the hardest stratum.

Load-bearing premise

The solve rates for Interactive Rebase and Iterative Committing rest on the assumption that an LLM judge can reliably and unbiasedly tell which of two Git histories is higher quality; the authors acknowledge in Section 7 that this may introduce bias.

Editorial extensions

If this is right

  • Git-specific tooling and evaluation become a recognized part of agentic software engineering, complementary to code-fixing benchmarks.
  • Current agents fail on interactive Git workflows such as rebase and partial staging, so new agent designs must handle delayed feedback and structured state.
  • The 21.11% baseline on the Lite set gives a concrete target: any proposed Git agent should report success on the same 120 samples.
  • The Train split (17,469 samples) is intended for collecting agent trajectories, enabling smaller models to be trained on Git tasks.
  • The MCR difficulty heuristic is validated by the baseline: easy samples solve at 22.58%, medium at 7.69%, and hard at 0%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the LLM-judge step could be a major source of variance; an ensemble of judges or human ratings would be a direct test.
  • A natural next benchmark axis would be diagnostic workflows such as git bisect, which the paper mentions only as future work.
  • A way to separate model reasoning from tool design would be to compare the custom scaffold against a generic Git protocol adapter, something the paper notes it did not investigate.
  • If Git proficiency becomes a standard axis, composite evaluations that pair code-fixing and history-quality scores would better reflect real developer workflows.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces GitGoodBench, a benchmark for evaluating AI agents on version-control (Git) tasks, covering three scenario types: Merge Conflict Resolution (MCR), Interactive Rebase (IR), and Iterative Committing of Changes (ICC). The benchmark is constructed by mining permissive open-source Python, Java, and Kotlin repositories, with a comprehensive suite (900 samples), a rapid-prototyping Lite set (120 samples), and a training corpus (17,469 samples). The authors present a baseline using GPT-4o with custom tools on the Lite set, achieving an 88% success rate and a 21.11% overall solve rate, with MCR evaluated by exact match and IR/ICC evaluated by an LLM-as-a-judge protocol that compares agent-generated vs. ground-truth Git histories. The paper also documents data filters, stratification, and a difficulty heuristic for MCR based on conflict counts and distribution, while acknowledging that the proposed FCC purity-based difficulty heuristic for IR/ICC does not correlate with empirical solve rates.

Significance. GitGoodBench addresses a real gap: existing SE agent benchmarks such as SWE-bench focus on issue resolution and do not evaluate VCS-specific workflows like interactive rebase, merge-conflict resolution, or iterative committing. The dataset construction is careful and externally grounded in real repositories, with transparent filters, stratification, and per-sample metadata; the release of a training corpus and a Lite/Full split is a useful contribution. If the evaluation methodology is validated, the benchmark could meaningfully advance research on VCS-aware agents. The baseline results, however, are only indicative: the solve rates for IR and ICC rest entirely on an unvalidated LLM judge, and the reported numbers come from a single run without error bars. These issues must be resolved before the quantitative claims can be taken as established reference points.

major comments (3)
  1. [§3.3, §7] The IR and ICC solve rates (26.67% and 23.33%) are computed via an LLM-as-a-judge protocol, but the judge is never validated against human judgment, no inter-judge agreement is reported, no sensitivity analysis with alternative judge models is given, and the paper does not even specify which LLM served as the judge. The paper's own Section 7 concedes that this 'evaluation methodology may introduce bias, as it is LLM-based.' Because these two scenario types constitute two-thirds of the benchmark's evaluations, the headline 21.11% overall solve rate is not yet an objective measure of Git competence. The authors should calibrate the judge against human ratings on a subsample, report agreement statistics, test at least one other judge model, and state the judge model and decoding parameters explicitly.
  2. [§3.2, §A.2.3, §5, Table 4] The FCC purity-based difficulty heuristic is explicitly acknowledged not to correlate with empirical solve rates: Table 4 shows easy IR samples with a 13.33% solve rate while medium and hard samples have 31.82% and 30.43% respectively, i.e., the 'easy' label corresponds to the worst performance. Since this heuristic is used for stratification and is included in the dataset metadata, the difficulty labels for FCC-based scenarios are not valid and should not be presented as meaningful. The authors should either revise the heuristic, relabel the samples as provisional, or remove the difficulty field from the released FCC samples.
  3. [§5] All baseline results are based on a single run per scenario, with no error bars or repeated trials. GPT-4o trajectories are stochastic; a single sample can be solved in one run and not another, so the reported 21.11% overall solve rate and the per-scenario rates in Tables 2–4 lack an estimate of variability. The authors should report aggregated results over at least a few repeated runs (e.g., 3 runs) with per-scenario and overall variance, or explain why the evaluation is deterministic despite the stochasticity of the underlying model.
minor comments (5)
  1. [§3.2] The command 'git show –remerge-diff' uses an en dash; it should be 'git show --remerge-diff'.
  2. [§3.3, §4] The paper does not state which LLM model was used as the judge in the LLM-as-a-judge protocol, nor the temperature/sampling parameters for either the agent or the judge; this information is needed for reproducibility.
  3. [§5, Table 2] The overall solve rate of 21.11% is computed over 180 scenario-evaluations (60 FCC samples evaluated once as IR and once as ICC, plus 60 MCR samples), while the Lite set contains 120 unique samples. The paper should clarify this double-counting of the FCC samples in the overall rate, or report a per-unique-sample rate in addition.
  4. [§5] The statement that 'Easy samples have a ≈3 times higher solve rate than hard samples' is inaccurate because the hard sample solve rate is 0%; the ratio is undefined, not three.
  5. [§1] There is a typo in 'a approximately 60% improvement' in the Introduction.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: benchmark construction and MCR evaluation are externally grounded; the FCC LLM-judge protocol is a disclosed measurement limitation, not a derivation from the paper's own inputs.

full rationale

GitGoodBench's derivation chain is: mine real permissive open-source repositories, apply metadata and scenario filters, construct MCR/IR/ICC scenarios, run a GPT-4o baseline, and report solve rates. The MCR solve rate is exact match against a real ground-truth merge commit, so it is externally grounded. The IR and ICC solve rates are defined in Section 3.3 as the LLM judge preferring the agent history over the original history in two position-swapped runs; this is the paper's own metric definition, not a hidden prediction of an independent quantity. Section 7 discloses the risk: 'for FCC samples our evaluation methodology may introduce bias, as it is LLM-based.' That is a measurement-validity limitation rather than a derivation-level circularity: no test-set parameter is fitted, no central claim is imported from author-only citations, and the dataset contribution is independent of the judge protocol. The only author self-citation (Eliseeva et al., 2023) appears in a related-work list about commit-message completion and is not load-bearing. The data splits are disjoint, and the baseline does not train on the benchmark. No step reduces by construction to its own input.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted in a mathematical sense; the listed numbers are hand-chosen dataset and difficulty thresholds. The benchmark relies on domain assumptions about the validity of mined Git histories, the uniqueness of merge resolutions, and the reliability of LLM-based evaluation.

free parameters (6)
  • Max merge conflicts per scenario = 8
    Hand-chosen threshold to keep scenarios solvable (Table 1b).
  • Max FCC length = 6
    Hand-chosen limit on commits in a File-Commit Chain (Table 1b).
  • Repository minimum stars = 1000
    Heuristic for repository quality (Table 1a).
  • Minimum branches and contributors = 5 and 5
    Heuristics for merge conflict availability (Table 1a).
  • MCR difficulty cutoffs = 1 conflict = easy; multiple in one file = medium; else hard
    Hand-defined difficulty stratification (Section 3.2).
  • FCC purity cutoffs = 0.5 and 0.75
    Hand-chosen boundaries for the FCC difficulty heuristic (Section A.2.3).
assumptions (3)
  • domain assumption Real-world Git histories mined from permissive open-source repositories, filtered by the metadata and scenario filters, constitute valid and representative tasks for evaluating VCS agent performance.
    The entire benchmark is built on this premise; the filters in Table 1 are heuristics, and the failure of the FCC difficulty heuristic shows the mapping is imperfect.
  • domain assumption The developer's actual merge commit is the unique correct conflict resolution.
    MCR solve rate is exact match against the ground-truth merge commit (Section 3.3). In practice multiple valid resolutions exist, so exact match may understate capability.
  • domain assumption An LLM judge's preference between two Git histories correlates with true history quality.
    IR and ICC solve rates are defined by LLM-as-a-Judge (Section 3.3); the authors flag bias risk in Section 7.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git." pith.science (2026). https://pith.science/paper/Z5G3RV2U

@misc{pith2026250522583,
  author       = {Pith},
  title        = {Pith review of: GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z5G3RV2U}},
  note         = {Machine review of arXiv:2505.22583}
}
read the original abstract

Benchmarks for Software Engineering (SE) AI agents, most notably SWE-bench, have catalyzed progress in programming capabilities of AI agents. However, they overlook critical developer workflows such as Version Control System (VCS) operations. To address this issue, we present GitGoodBench, a novel benchmark for evaluating AI agent performance on VCS tasks. GitGoodBench covers three core Git scenarios extracted from permissive open-source Python, Java, and Kotlin repositories. Our benchmark provides three datasets: a comprehensive evaluation suite (900 samples), a rapid prototyping version (120 samples), and a training corpus (17,469 samples). We establish baseline performance on the prototyping version of our benchmark using GPT-4o equipped with custom tools, achieving a 21.11% solve rate overall. We expect GitGoodBench to serve as a crucial stepping stone toward truly comprehensive SE agents that go beyond mere programming.

Figures

Figures reproduced from arXiv: 2505.22583 by the authors.

Figure 1
Figure 1. The three Git scenarios supported by GitGood [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Our MCR prompt [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Our MCR prompt continued [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Our MCR prompt continued. Interactive Rebase (IR) Prompt - Part 1 You are a staff software engineer with expertise in {programming_language} and Git. You are helping a junior team member who has been committing all day without pushing their commits to the remote. Help …
Figure 5
Figure 5. Figure 5: Our IR prompt [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Our IR prompt [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Our IR prompt. Iterative Committing of Changes (ICC) Prompt - Part 1 You are a staff software engineer with expertise in {programming_language} and Git. You are helping a junior team member who has been working all day without creating a commit to iteratively create co…
Figure 8
Figure 8. Figure 8: Our ICC prompt [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Our ICC prompt continued. LLM-as-a-Judge Evaluation Prompt - Part 1 Please act as an impartial judge and evaluate the quality of the two git histories that are displayed below. Your evaluation should consider the following aspects: - The quality of the commit messages …
Figure 10
Figure 10. Figure 10: Our LLM-as-a-Judge evaluation prompt. We use the same prompt for both evaluation runs, we simply [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Our LLM-as-a-Judge evaluation prompt continued. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BulkPR-Bench: Benchmarking Queue-Level Governance of Interacting Pull Requests

    cs.SE 2026-08 conditional novelty 7.0 of 10

    The paper introduces an execution-validated benchmark for pull-request queue governance and reports that top models reach 66.6% relational delivery yet only 8 of 324 runs deliver an entire queue exactly.

  2. SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments

    cs.SE 2025-07 conditional novelty 6.0 of 10

    SetupBench, a 93-instance environment-bootstrap benchmark, finds coding agents succeed on only 34.4-62.4% of setup tasks, with database configuration and repo setup being the hardest.

Reference graph

Works this paper leans on

31 extracted references · 15 canonical work pages · cited by 2 Pith papers

  1. [1]

    Anthropic. 2024. https://www.anthropic.com/news/model-context-protocol Introducing the model context protocol . Accessed on May 20, 2025

    work page 2024

  2. [2]

    Anthropic. 2025. https://www.anthropic.com/news/claude-3-7-sonnet Claude 3.7 sonnet and claude code . Accessed on February 27, 2025

    work page 2025

  3. [3]

    Shrestha Basu, Mallick, and Kathy Korevec. 2024. https://developers.googleblog.com/en/the-next-chapter-of-the-gemini-era-for-developers/ The next chapter of the gemini era for developers . Accessed on February 27, 2025

    work page 2024

  4. [4]

    Alexander Boll, Yael Van Dok, Manuel Ohrndorf, Alexander Schultheiß, and Timo Kehrer. 2024. https://doi.org/10.1145/3661167.3661197 Towards Semi - Automated Merge Conflict Resolution : Is It Easier Than We Expected ? In Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering , EASE '24, pages 282--292

    arXiv 2024

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    work page 2020

  6. [6]

    Embury, and Sukru Eraslan

    Julio César Cortés Ríos, Suzanne M. Embury, and Sukru Eraslan. 2022. https://doi.org/10.1016/j.infsof.2021.106811 A unifying framework for the systematic analysis of Git workflows . Information and Software Technology, 145(C)

    arXiv 2022

  7. [7]

    Cursor. 2024. https://www.cursor.com/changelog/new-composer-ui-agent-commit-messages New composer ui, agent, commit messages . Accessed on February 27, 2025

    work page 2024

  8. [8]

    Ozren Dabic, Emad Aghajani, and Gabriele Bavota. 2021. https://seart-ghs.si.usi.ch/ Sampling projects in github for MSR studies . In 18th IEEE/ACM International Conference on Mining Software Repositories, MSR 2021 , pages 560--564

    work page 2021

Show all 31 references
  1. [9]

    Aleksandra Eliseeva, Yaroslav Sokolov, Egor Bogomolov, Yaroslav Golubev, Danny Dig, and Timofey Bryksin. 2023. https://doi.org/10.1109/ASE56229.2023.00078 From Commit Message Generation to History - Aware Commit Message Completion . In 2023 38th IEEE / ACM International Confer...

    2023

  2. [10]

    S. R. P. van Hal, M. Post, and K. Wendel. 2019. https://doi.org/10.48550/arXiv.1911.11690 Generating Commit Messages from Git Diffs . arXiv preprint. ArXiv:1911.11690 [cs.SE]

    2019 doi arXiv

  3. [11]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring massive multitask language understanding . In International Conference on Learning Representations

    2021

  4. [12]

    Naman Jain, Jaskirat Singh, Manish Shetty, Liang Zheng, Koushik Sen, and Ion Stoica. 2025. R2E - Gym : Procedural Environments and Hybrid Verifiers for Scaling Open - Weights SWE Agents . arXiv preprint

    2025

  5. [13]

    Siyuan Jiang, Ameer Armaly, and Collin McMillan. 2017. https://doi.org/10.1109/ASE.2017.8115626 Automatically generating commit messages from diffs using neural machine translation . In 2017 32nd IEEE / ACM International Conference on Automated Software Engineering ( ASE ) , p...

    2017

  6. [14]

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. https://openreview.net/forum?id=VTF8yNQM66 SWE -bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Re...

    2024

  7. [15]

    Andy Konwinski, Christopher Rytting, Justin Fiedlerand Alex Shaw, Sohier Dane, Walter Reade, and Maggie Demkin. 2024. Konwinski prize. https://kaggle.com/competitions/konwinski-prize. Kaggle

    2024

  8. [16]

    Microsoft. 2025. https://code.visualstudio.com/blogs/2025/02/24/introducing-copilot-agent-mode Introducing github copilot agent mode for vscode . Accessed on February 27, 2025

    2025

  9. [17]

    OpenAI et al. 2024 a . https://doi.org/10.48550/arXiv.2303.08774 GPT -4 Technical Report . arXiv preprint. ArXiv:2303.08774 [cs.CL]

    2024 doi arXiv

  10. [18]

    OpenAI et al. 2024 b . https://openai.com/index/gpt-4o-system-card/ Openai gpt-4o system card . Accessed on March 6, 2025

    2024

  11. [19]

    Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. 2024. https://doi.org/10.48550/arXiv.2412.21139 Training Software Engineering Agents and Verifiers with SWE - Gym . arXiv preprint. ArXiv:2412.21139 [cs]

    2024 doi arXiv

  12. [20]

    Rangeet Pan, Vu Le, Nachiappan Nagappan, Sumit Gulwani, Shuvendu Lahiri, and Mike Kaufman. 2021. https://doi.org/10.1109/ICSE43902.2021.00077 Can Program Synthesis be Used to Learn Merge Conflict Resolutions ? An Empirical Analysis . In Proceedings of the 43rd International Co...

    2021

  13. [21]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. https://openreview.net/forum?id=Yacmpz84TH Toolformer: Language models can teach themselves to use tools . In Thirty-seventh C...

    2023

  14. [22]

    Chaochao Shen, Wenhua Yang, Minxue Pan, and Yu Zhou. 2023. https://doi.org/10.1109/QRS60937.2023.00031 Git Merge Conflict Resolution Leveraging Strategy Classification and LLM . In 2023 IEEE 23rd International Conference on Software Quality , Reliability , and Security ( QRS )...

    2023

  15. [23]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/1b44b878bb782e6954cd888628510e90-Paper-Conference.pdf Reflexion: language agents with verbal reinforcement learning . In Advances...

    2023

  16. [24]

    Alexey Svyatkovskiy, Sarah Fakhoury, Negar Ghorbani, Todd Mytkowicz, Elizabeth Dinella, Christian Bird, Jinu Jang, Neel Sundaresan, and Shuvendu K. Lahiri. 2022. https://doi.org/10.1145/3540250.3549163 Program merge conflict resolution via neural transformers . In Proceedings ...

    2022

  17. [25]

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. https://proceedings.mlr.press/v235/wang24h.html Executable Code Actions Elicit Better LLM Agents . In Proceedings of the 41st International Conference on Machine Learning , pages 50208--50232

    2024

  18. [26]

    Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang

    John Yang, Kilian Leret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. 2025. https://doi.org/10.48550/arXiv.2504.21798 SWE -smith: Scaling Data for Software Engineering Agents . arXiv preprint. ArXiv:...

    2025 doi arXiv

  19. [27]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. https://openreview.net/forum?id=WE_vluYUL-X React: Synergizing reasoning and acting in language models . In The Eleventh International Conference on Learning Representations

    2023

  20. [28]

    Andrew Zakonov. 2025. https://blog.jetbrains.com/junie/2025/01/meet-junie-your-coding-agent-by-jetbrains/ Meet junie, your coding agent by jetbrains . Accessed on February 27, 2025

    2025

  21. [29]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/91f18a1287b398d378ef22505bf41832-Pap...

    2023

  22. [30]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  23. [31]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.