Pith. sign in

REVIEW 3 major objections 5 minor 30 references

Pull Requests as a Training Signal for Repo-Level Code Editing

T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training on 2 million verified pull requests lets a 32-billion-parameter code model outperform larger agent-driven systems at repository-scale bug fixing, without complex inference loops.

desk verdict A well-engineered dataset and training recipe whose central internalisation claim currently hinges on an unproven disjointness between its SFT origins and the SWE-bench test sets. read the letter →

arxiv 2602.07457 v2 pith:VTPOX5ZN submitted 2026-02-07 cs.SE cs.AIcs.CL

Qinglin Zhu , Tianyu Chen , Shuai Lu , Lei Ji , Runcong Zhao , Murong Ma , Xiangxiang Dai , Yulan He
show 3 more authors
Lin Gui Peng cheng Yeyun Gong
This is my paper
classification cs.SEcs.AIcs.CL
keywords repository-levelcodeeditingpullrequestsmid-trainingSearch/ReplaceeditblocksSWE-benchsupervisedfine-tuningerror-drivenaugmentationgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the skill of editing an entire code repository—finding the right files, locating the precise lines, and generating a correctly applied patch—can be encoded directly into a model's weights by mid-training on a large, carefully verified corpus of real pull requests. The authors build Clean-PR, a 2-million-example corpus spanning 12 languages, by filtering 16.4 million raw GitHub pull requests and converting each merged diff into Search/Replace edit blocks that are verified to reproduce the post-merge code exactly through bit-wise round-trip application. They then mid-train a 32B code model on this corpus, followed by a supervised fine-tuning stage that teaches a three-step locate-navigate-edit protocol and hardens the model against distractor files using error-driven augmentation. On the standard SWE-bench benchmark, the model reaches 24.3% pass@1 on the Lite split and 30.6% on Verified—absolute gains of 13.6 and 12.3 points over the instruction-tuned baseline—and it also outperforms two 72-billion-parameter open systems. If correct, this shows that heavy agentic scaffolding is not required for strong repository-level editing, and that data quality and verification can substitute for test-time compute.

What carries the argument

The central object is the Clean-PR corpus in Search/Replace format: each edit is represented by a minimal, unique anchor of contiguous source text (the SEARCH block) and the replacement text (the REPLACE block), plus the linked GitHub issue description that supplies the original problem statement. The verification step is the load-bearing mechanism: a round-trip check applies the generated blocks back to the base repository and requires a bit-wise match to the reconstructed post-merge state, guaranteeing that every training example is deterministic and anchored to a unique context rather than to fragile line numbers. A second mechanism is error-driven augmentation during SFT, which injects d

What would settle it

Take the same Clean-PR-trained checkpoint and evaluate it under a full agentic or retrieval-heavy scaffold (for example, giving it all files or a search tool) versus the simplified protocol: if the 12–13 point gain over the instruct baseline shrinks or disappears when the simplified scaffold is removed or swapped, the capability is not internalised in the weights alone. A simpler companion check: apply the identical training recipe to a substantially smaller base model; if the gains do not appear, the claim that the data pipeline alone internalises the skill is weakened.

Watch

Extended reading notes

Core claim

The paper's central claim is that repository-level editing capability can be internalised into model weights through mid-training on verified pull-request data. Concretely, it shows that a 32-billion-parameter code model, after mid-training on 2 million verified Search/Replace pull requests and an agentless-aligned SFT with error-driven augmentation, achieves 24.3% pass@1 on SWE-bench Lite and 30.6% on SWE-bench Verified, outperforming the instruction-tuned baseline by 13.6 and 12.3 absolute points and surpassing two 72B open-source systems at 22.0/30.2. The authors argue this demonstrates that the locate-navigate-edit workflow can be baked into model weights rather than relying on iterative

Load-bearing premise

The central claim assumes that the simplified inference scaffold (top-3 file retrieval, fixed prompts, greedy decoding) contributes little to the measured gains, so the improvement can be attributed to the training data and the model's weights; the paper does not compare the same trained model under a stronger or more complex scaffold.

Editorial extensions

If this is right

  • Performing well on repository-level tasks does not require multi-turn agent loops; a linear locate-navigate-edit protocol is enough if the model has been mid-trained on verified editing data.
  • Training on verified Search/Replace edits improves general coding ability: the mid-trained model gains +5.7 on HumanEval and +3.6 on LiveCodeBench, while a diff-based baseline suffers catastrophic forgetting (-6.5 on HumanEval).
  • Multi-language mid-training adds value beyond Python alone: with similar token budgets, the all-language corpus improves Lite from 22.3 to 24.3 and Verified from 27.8 to 30.6 over the Python-only model.
  • Pass@1 substantially understates the model's capability: sampling 10 patches raises resolution to 37.5% Lite and 41.5% Verified, suggesting a lightweight reranker or verifier can unlock further gains without agentic training.
  • Error-driven augmentation with hard negatives is necessary for robustness: removing it drops pass@1 from 24.3 to 21.8 (Lite) and 30.6 to 27.4 (Verified).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: if these gains replicate on other base models, data construction and verification could substitute for model scale and test-time compute, making strong repository-editing capability substantially cheaper to produce.
  • My inference: the paper's comparison does not isolate the scaffold's contribution—the simplified inference protocol includes top-3 file retrieval and fixed prompts, so part of the measured gain may come from the harness rather than the weights; a same-model ablation with a richer scaffold would settle this.
  • My inference: the verified Search/Replace transformation is a general recipe: the same reconstruction-and-round-trip pipeline could turn other noisy edit corpora (commits, merge requests, downstream patches) into verifiable training signal for editing tasks beyond bug fixing.
  • My inference: a direct testable extension is to run the same trained weights inside a full agentic environment; the mid-training hypothesis predicts the model should outperform its instruct baseline there too, while the paper explicitly leaves that comparison to future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Clean-PR, a large-scale data-construction pipeline that converts raw GitHub pull requests into verified Search/Replace edit blocks, and a two-stage training recipe (mid-training on Clean-PR, then agentless-aligned SFT with error-driven augmentation). The authors report that a Qwen2.5-Coder-32B model trained with this recipe improves SWE-bench Lite pass@1 from 10.7% (instruct baseline) to 24.3% and SWE-bench Verified from 18.3% to 30.6%, and they interpret these gains as evidence that repository-level editing capability can be internalised into model weights under a simplified, agentless inference protocol.

Significance. If the results hold, the paper makes two useful contributions. First, the Clean-PR corpus is large, verified by round-trip patch application, and the decontamination protocol for the PR corpus is detailed; the Search/Replace format and issue-linkage ablations are informative. Second, the two-stage training recipe is reproducible in principle: exact prompts, hyperparameters, and dataset statistics are given. The central claim — that strong SWE-bench performance can be obtained without heavy agentic scaffolding — is practically important. However, the evidence is currently weakened by a gap in the SFT data provenance: the SFT set is built from benchmark-derived trajectories (SWE-rebench, SWE-Gym, SWE-bench-Live), and no decontamination is reported for those 53k+ instances. This must be resolved before the headline numbers can be interpreted as evidence of general internalisation rather than benchmark-adjacent memorisation.

major comments (3)
  1. [§2.2 / Table 5 / Appendix A.6] The SFT dataset is built from 'verified trajectories from SWE-rebench ... and SWE-Gym', and Table 5 lists the Origin sources as SWE-bench-Live, SWE-rebench, and SWE-Gym. The detailed decontamination protocol in Appendix A.6 (repo blocklist, SHA-256 exact-file matching, 15-gram overlap, 0.5 Jaccard filtering) is described only for the Clean-PR corpus. No comparable decontamination is reported for the 18,891 + 17,763 + 16,564 Origin SFT instances or the 21,864 error-augmented examples. Since SWE-rebench and SWE-Gym are derived from SWE-bench-style repositories and tasks, any overlap (by instance_id, repository, or gold patch) with SWE-bench Lite/Verified would let the SFT stage train directly on near-test problems, invalidating the 24.3%/30.6% numbers as evidence of internalised capability. The paper must either prove disjointness of the SFT origin data from the evaluation sets or retrain
  2. [§3.1 / §3.2 / Appendix D] The central claim is that improvements are 'internalised into model weights' under a simplified agentless protocol. The experiments compare training data settings within one fixed scaffold (three prompts, top-3 file retrieval, greedy decoding), so the relative gains over the instruct baseline are attributable to training. But the absolute-level interpretation ('without relying on heavy inference-time scaffolding') is not isolated: there is no comparison of the same trained model under a full Agentless-style scaffold or with retrieval/prompt components removed. For example, top-3 retrieval could be doing much of the localisation work. A concrete ablation — vary top-k (e.g., 1, 3, 5, full repo tree) and compare with a full Agentless baseline on the same checkpoint — would clarify how much of the final pass@1 is weight-internalised versus scaffold-assisted. Without this, the stronger wordin
  3. [Table 7 / §3.2] Table 7 compares Clean-PR against SWE-Gym, Lingma-SWE, and SWE-Fixer, with results 'copied from the original paper.' These systems use different base models (32B vs 72B), different scaffolds (OpenHands, SWESynInfer, SWE-Fixer), and possibly different sampling/decoding settings. As an external context this is fine, but the text draws a strong conclusion ('outperforms the 72B baselines ... confirms that rigorous mid-training bridges the scaling gap'). To support that claim, the comparison should be made under matched evaluation conditions, or at least the pass@1 settings (greedy vs sampling, number of samples, repository-state handling) should be reported for each external method. Otherwise this is not a controlled comparison and should be framed as indicative rather than confirmatory.
minor comments (5)
  1. [§3.2] The text says 'Base + SFT ... achieves only 10.3% on Lite and 17.6% on Verified', but Table 6 reports 11.3% on Lite and 17.6% on Verified. The Lite number is inconsistent; please correct.
  2. [Appendix A.6] The decontamination thresholds (15-gram overlap, 0.5 Jaccard) are stated but not justified. Since these are free parameters, a short sensitivity analysis or a citation to a standard protocol would strengthen the claim that partial-match leakage is excluded.
  3. [Table 7] Please clarify whether the external baseline numbers are greedy pass@1 or best-of-n pass@1. If they use different inference budgets, the comparison in the text should be adjusted.
  4. [Table 16] The field 'is use windows' appears to be a typo; presumably it should be 'is_use_windowing' or similar. Also, the release schema does not include the exact date/commit of the crawled base code, which would help reproducibility.
  5. [Abstract / Table 1] The abstract says 'largest publicly available corpus of 2 million pull requests', while Clean-PR-full contains 3,050,939 and Clean-PR-train 2,015,708. Please be explicit about which corpus is being released and released size.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Clean-PR mid-training signal is independent of SWE-bench and decontaminated; benchmark-derived SFT is a shared control, not a constructed prediction.

full rationale

The paper's claimed derivation chain is not circular. The mid-training corpus is built from raw GitHub PRs with an independent verification pipeline (Algorithm 1), not from SWE-bench labels; Section 2.1 and Appendix A.6 explicitly blocklist SWE-bench repositories and apply content-based decontamination (SHA-256 file matching, 15-gram overlap, Jaccard filtering) to the PR corpus. The SFT stage uses benchmark-derived trajectories (Table 5: SWE-bench-Live, SWE-rebench, SWE-Gym), but this SFT set is applied identically to the Base+SFT and StarCoder2-style baselines, so the reported 13.6/12.3 point gains isolate the Clean-PR mid-training variable rather than reducing to a fitted benchmark label. The simplified Agentless scaffold is a fixed, deterministic evaluation protocol, not a fitted parameter or a predicted quantity; aligning SFT to it is a standard train/evaluation distribution choice, not a circular reduction. No load-bearing self-citation was found: citations to Qwen-Coder, Agentless, SWE-Gym, and SWE-rebench are external works, not prior results by the same authors invoked to force the paper's conclusion. The only noteworthy caveat is an omitted proof rather than circularity: Appendix A.6 does not state that the 53,218 Origin SFT instances were decontaminated against SWE-bench Lite/Verified, and SWE-Gym/SWE-rebench are SWE-bench-adjacent. If such overlap existed it would threaten attribution of the absolute Pass@1 numbers, but it would not make the Clean-PR-versus-baseline comparison circular, and the central data-construction claim remains supported by the independently decontaminated open-source PR signal.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical or conceptual entities are invented. The paper introduces a dataset and a training recipe, not a new theoretical object. The main 'pulled-from-somewhere' elements are the hand-set filtering thresholds, the choice of simplified scaffold, and the single-model training assumption, all documented above.

free parameters (6)
  • Core-extension and Allowed-file rules (12 languages) = Defined in Table 12
    The choice of which file extensions count as 'core' and which are 'allowed' is a hand-specified design decision that determines the language distribution and the filtering of PRs. It is not fitted to SWE-bench but it is a manually chosen threshold that affects the final dataset composition.
  • PR length heuristics (title >= 10 chars, description >= 20 chars)
    These thresholds are chosen by hand to filter noisy PRs. They are not fitted to the benchmark, but they are free design choices that affect data quality and volume.
  • Repository sampling cap: max 2,000 PRs per repo = 2000
    This is an arbitrary balancing hyperparameter that determines the final corpus composition. It is not derived from the benchmark, but it affects the training distribution.
  • Max core files <= 5 for Clean-PR-train = 5
    A hand-chosen complexity-control threshold. Changing it changes the average modified-file count and thus the difficulty of the training signal.
  • Top-3 file retrieval in SFT/evaluation scaffold = 3
    The simplified agentless scaffold keeps only the top-3 ranked files for navigation and patch generation. This is a hand-set retrieval budget that directly affects both training and evaluation behavior. The paper does not ablate this number.
  • 15-gram overlap and 0.5 Jaccard decontamination thresholds = 15-gram, 0.5
    These are chosen rejection thresholds for content-based decontamination. They are not fitted to SWE-bench but are manually specified to balance contamination removal versus data retention.
assumptions (5)
  • domain assumption GitHub PR merge status and bot heuristics are reliable proxies for data quality.
    The pipeline discards unmerged/closed PRs and bot-created PRs based on username regexes. This assumes that merged status implies high-quality, learnable edits and that bot detection is accurate enough. Entered in Section 2.1 and Appendix A.2.
  • domain assumption The raw PR patch, when applied via git apply with fallback flags, correctly reconstructs the true 'after' repository state.
    The entire verification pipeline depends on FakeGitApply. If git apply succeeds due to fuzzy matching or whitespace fixes, the reconstructed target may differ from what the maintainers actually merged. Algorithm 1, Phase 1.
  • domain assumption Uniqueness of a Search/Replace block within a file is sufficient to guarantee unambiguous application during model training and evaluation.
    The paper verifies uniqueness in the base file, but during evaluation the model may output blocks that are unique in the retrieved context yet ambiguous in the full repository, or vice versa. The SWE-bench evaluator applies patches differently; the paper does not prove that their format guarantees apply-ability in the evaluation harness.
  • domain assumption Qwen2.5-Coder-32B-Base is a representative base model for repository-level editing.
    The paper only trains this one model and explicitly defers cross-model generalization to future work. All conclusions about 'internalising repo-level capabilities' are demonstrated for a single architecture. Acknowledged in Appendix D.
  • domain assumption SWE-bench Lite and Verified are contamination-free with respect to the Clean-PR corpus after the described decontamination.
    The decontamination protocol is strong but not provably perfect: it uses repository blocklists, file SHA-256 matching, 15-gram overlap, and Jaccard similarity. It cannot catch all conceivable leakage (e.g., semantic paraphrases of issue text or code transformations beyond 15-grams).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pull Requests as a Training Signal for Repo-Level Code Editing." pith.science (2026). https://pith.science/paper/VTPOX5ZN

@misc{pith2026260207457,
  author       = {Pith},
  title        = {Pith review of: Pull Requests as a Training Signal for Repo-Level Code Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VTPOX5ZN}},
  note         = {Machine review of arXiv:2602.07457}
}
read the original abstract

Repository-level code editing requires models to understand complex dependencies and execute precise multi-file modifications across a large codebase. While recent gains on SWE-bench rely heavily on complex agent scaffolding, it remains unclear how much of this capability can be internalised via high-quality training signals. To address this, we propose Clean Pull Request (Clean-PR), a mid-training paradigm that leverages real-world GitHub pull requests as a training signal for repository-level editing. We introduce a scalable pipeline that converts noisy pull request diffs into Search/Replace edit blocks through reconstruction and validation, resulting in the largest publicly available corpus of 2 million pull requests spanning 12 programming languages. Using this training signal, we perform a mid-training stage followed by an agentless-aligned supervised fine-tuning process with error-driven data augmentation. On SWE-bench, our model significantly outperforms the instruction-tuned baseline, achieving absolute improvements of 13.6% on SWE-bench Lite and 12.3% on SWE-bench Verified. These results demonstrate that repository-level code understanding and editing capabilities can be effectively internalised into model weights under a simplified, agentless protocol, without relying on heavy inference-time scaffolding.

Figures

Figures reproduced from arXiv: 2602.07457 by the authors.

Figure 1
Figure 1. Overview of the Clean-PR Framework. (a) Data Construction: Raw GitHub PRs undergo a rigorous filtering pipeline (bot detection, core language enforcement) and intent augmentation via linked Issues. The valid diffs are then converted into minimal unique Search/Replace blocks, verified through round-trip patch application to ensure correctness. (b) Two-Stage Training Pipeline: The base model first undergoes Mid-Traini… view at source ↗
Figure 2
Figure 2. Generalisation dynamics during mid-training. best-performing “All Languages” model, the augmentation boosts the Pass@1 rate from 21.8% to 24.3% on SWE￾bench Lite and from 27.4% to 30.6% on SWE-bench Ver￾ified. Crucially, we observe simultaneous improvements in Line accuracy, which confirms that explicitly training the model to discriminate against distracting context and reject irrelevant files significantly enhance… view at source ↗
Figure 3
Figure 3. Pass@k performance on SWE-bench Lite and Verified. We report the resolution rates of our model (Clean-PR, mid-trained on All Languages) as the number of samples k scales. 4. Related Work Inference Paradigms and System Complexity. The pur￾suit of automated repository-level engineering has spurred a diverse ecosystem of inference frameworks. Early dom￾inant approaches relied on Agentic frameworks, where models functio… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The Life of a Data Point: From Raw Noise to Verified Signal. Track A (Left) illustrates the aggressive pruning of noise, rejecting inputs due to bot activity, unmerged status, non-core language files, or missing history. Track B (Right) depicts the transformation of a …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 1 linked inside Pith

  1. [1]

    Automation and Bot Filtering.We exclude PRs cre- ated by automated tools or where the only activity comes from bots. A user is classified as a bot if their username matches any of the following regular expressions: •Suffix Patterns:bot$, bot$,-bot$ •Prefix Patterns:ˆbot • Specific Services: dependabot, renovate, github-actions, travis-ci, circleci, covera...

  2. [2]

    Jiang, Z., Ren, X., Yan, M., Jiang, W., Li, Y ., and Liu, Z

    URL https://openreview.net/forum? id=7evvwwdo3z. Jiang, Z., Ren, X., Yan, M., Jiang, W., Li, Y ., and Liu, Z. Issue localization via llm-driven iterative code graph searching, 2025. URL https://arxiv.org/abs/ 2503.22424. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. R. SWE-bench: Can language models resolve real-wor...

    arXiv 2025

  3. [3]

    We exclude PRs that: • Missing Base Code or Diff:PRs with empty base code files or missing diffs are filtered out

    Structural Integrity Checks.We strictly enforce that the PR represents a clean, in-place modification of existing code. We exclude PRs that: • Missing Base Code or Diff:PRs with empty base code files or missing diffs are filtered out. • Mismatched Files:Have a discrepancy between the set of files in the base state and the files in the diff (i.e., bijectiv...

  4. [4]

    The dividing line:=======

  5. [5]

    Wei, J., Durrett, G., and Dillig, I

    URL https://openreview.net/forum? id=OJd3ayDDoF. Wei, J., Durrett, G., and Dillig, I. Coeditor: Leveraging con- textual changes for multi-round code auto-editing, 2024a. URLhttps://arxiv.org/abs/2305.18584. 10 Pull Requests as a Training Signal for Repo-Level Code Editing Wei, Y ., Wang, Z., Liu, J., Ding, Y ., and Zhang, L. Magi- coder: Empowering code g...

    arXiv 2020

  6. [6]

    ISBN 979-8-89176-256-5

    Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025.findings-acl

    doi:10.18653/v1/2025.findings-acl 2025

  7. [8]

    src/models/attention.py

    URL https://openreview.net/forum? id=MhBZzkz4h9. Zeng, Y ., Cao, J., Li, Z., Yu, W., Ye, Z., Xiang, D., Hua, T., Liu, X., Gao, S., and Yu, T. Hyperedit: Un- locking instruction-based text editing in llms via hyper- networks, 2025. URL https://arxiv.org/abs/ 2512.12544. Zhang, J., Panthaplackel, S., Nie, P., Li, J. J., and Gligoric, M. Coditt5: Pretraining...

    arXiv 2025

  8. [10]

    • Title Blocklist:We remove maintenance PRs containing keywords: bump, dependencies, dependency,depend,release

    Metadata and Quality Constraints.We filter PRs based on status and textual content to ensure semantic rele- vance: • Status:The PR must be marked as MERGED or APPROVED. • Title Blocklist:We remove maintenance PRs containing keywords: bump, dependencies, dependency,depend,release. • Description Blocklist:We remove descriptions con- tainingqwiet(indicating ...

Show all 30 references
  1. [12]

    Ground-Truth Reconstruction (Fake Git Apply).We reconstruct the “After” state using a git sandbox to handle context fuzzy-matching

  2. [13]

    Initialise a temporary git repository with the base code

  3. [14]

    Added check

    Apply the diff hunk using git apply with fallback strategies: •--verbose 14 Pull Requests as a Training Signal for Repo-Level Code Editing RAW INPUT: PR #456 (Developer) Title: Fix null pointer exception Description: Fix issue #42. Added check. Files: src/main.py Comment: Agre...

  4. [15]

    If success- ful, the result defines theexpected content

    If the application fails, the PR is discarded. If success- ful, the result defines theexpected content

  5. [16]

    Minimal Unique Context Search.We identify edit spans by computing the difference between the base and the reconstructed target files. To generate SEARCH blocks that are both concise and unambiguous, we employ an iterative expansion strategy: • Edit Merging:Adjacent edits (sepa...

  6. [17]

    round-trip

    Round-Trip Verification.We validate the generated blocks by performing a strict “round-trip” application using simple string replacement, independent of git. A training instance is retained only if it passes three integrity checks:

  7. [18]

    Uniqueness:Each generated SEARCH block must be found exactly once in the base file

  8. [19]

    Non-Overlapping:Multiple edit blocks within the same file must not have overlapping search regions

  9. [20]

    Exact Reconstruction:Applying the Search/Replace blocks to the base file via string replacement must yield a file that isbit-wise identicalto the ground- truthtarget contentderived in Step 1. A.5. Context Windowing Strategy For files exceeding the token limit (e.g., 100k token...

  10. [21]

    Identify Ranges:Extract line ranges [start, end]cov- ered by verified Search/Replace blocks

  11. [22]

    Expand:Extend each range by N= 20 lines to cap- ture local definitions

  12. [23]

    This ensures the model sees the necessary context for the 15 Pull Requests as a Training Signal for Repo-Level Code Editing edit without processing the entire file

    Merge & Reconstruct:Merge overlapping ranges and concatenate them, inserting markers for omitted sections. This ensures the model sees the necessary context for the 15 Pull Requests as a Training Signal for Repo-Level Code Editing edit without processing the entire file. A.6. ...

  13. [24]

    Any Pull Request originating from or targeting these repositories is structurally discarded

    Repository-Level Exclusion.As a primary defence, we strictly blocklist all repositories present in the SWE-bench Lite and Verified metadata. Any Pull Request originating from or targeting these repositories is structurally discarded

  14. [25]

    depen- dency

    Content-Based Decontamination (Addressing Code Movement).Relying solely on repository names is insuf- ficient due to the prevalence of code cloning and vendored directories. To mitigate this, we implement content-aware filtering: • Exact File Matching:We compute SHA-256 hashes...

    2023

  15. [26]

    The start of search block:<<<<<<< SEARCH

  16. [27]

    A contiguous chunk of lines to search for in the existing source code

  17. [29]

    The lines to replace into the source code

  18. [30]

    If you would like to add the line ’ print(x)’, you must fully write that out, with all those spaces before the code! Wrap the *SEARCH/REPLACE* edit in blocks‘‘‘python...‘‘‘

    The end of the replace block:>>>>>>> REPLACE Here is an example: ‘‘‘python ### mathweb/flask/app.py <<<<<<< SEARCH from flask import Flask ======= import math from flask import Flask >>>>>>> REPLACE ‘‘‘ Please note that the *SEARCH/REPLACE* edit REQUIRES PROPER INDENTATION. If...

  19. [62]

    findings-acl.62/

    URL https://aclanthology.org/2025. findings-acl.62/. Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K. R., and Press, O. SWE-agent: Agent- computer interfaces enable automated software engineer- ing. InThe Thirty-eighth Annual Conference on Neural Infor...

    2025

  20. [2024]

    Kocetkov, D., Li, R., allal, L

    URL https://openreview.net/forum? id=VTF8yNQM66. Kocetkov, D., Li, R., allal, L. B., LI, J., Mou, C., Jernite, Y ., Mitchell, M., Ferrandis, C. M., Hughes, S., Wolf, T., Bahdanau, D., Werra, L. V ., and de Vries, H. The stack: 3 TB of permissively licensed source code.Transac-...

    2023

  21. [2025]

    Golzadeh, M., Legay, D., Decan, A., and Mens, T

    URL https://openreview.net/forum? id=nMpJoVmRy1. Golzadeh, M., Legay, D., Decan, A., and Mens, T. Bot or not? detecting bots in github pull request activ- ity based on comment similarity. InProceedings of the IEEE/ACM 42nd International Conference on Soft- ware Engineering Wor...

    2020

  22. [8856]

    Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., and Sundaresan, N

    URL https://openreview.net/forum? id=pxpbTdUEpD. Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., and Sundaresan, N. Automating code review activi- ties by large-scale pre-training. InProceedings of the 30th ACM Joint ...

    2022

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.