Computation and Language · arXiv:2611.21152 v2 · submitted 16 Nov 2026 · 71 references
From Repositories to Training Environments: Scaling Long-Horizon Coding Agents with Environment Evolution Loop
The paper says
“Training coding agents for long-horizon tasks requires environments with verifiable feedback. Existing pipelines often follow a linear process that ends with validation, establishing usability but providing little guidance on improving training value.”
more of the abstract
Recursive self-improvement (RSI) often requires model rollouts or model training followed by downstream evaluation, with costs and feedback delays amplified on long-horizon tasks. How can we use timely, reliable proxy signals to improve training value and synthesize environments at scale? We present RepoFactory, an automated pipeline that converts real repositories into verifiable environments where agents build complete projects from scratch given a language specification. Its Environment Evolution Loop optimizes a multiplicative objective combining validity, structural coupling, and implementation scope through proxy feedback. RepoFactory produces 10,159 environments across 17 software domains. Training Qwen3.5-35B-A3B on these environments yields RepoHorizen-35B-A3B, which achieves gains of 22.6 percentage points on NL2Repo-Bench and 8.5 points on average across five benchmarks covering long-horizon coding and agentic tasks in real-world production settings, compared with 17.5 and 6.4 points without environment evolution. These results highlight the training value of RepoFactory environments and the additional gains from the Environment Evolution Loop.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 18 Nov 2026 · read the full text (70,037 characters) · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. RepoFactory converts real GitHub repositories into verifiable training environments for long-horizon coding agents, and its evolution loop—using cheap proxy metrics—adds a clear 5-point gain on NL2Repo-Bench beyond training on static environments. This line is wrong
10 minor · 4 open questions
How often objections from reviews like this one are right: not yet measured
The paper, first character to last
The reviewer was given the whole text, 70,037 characters. Not checked: figures, code, what the 71 references say.
10 minor objections
The paper says, at §3.1, Eq. (3)
Pith says: minor objection 1 of 10
The definition of structural coupling C uses normalized propagation cost. It would be helpful to cite the source of this metric or provide a brief intuition for why R/N^2 measures coupling.
The paper says, at §3.1, Eq. (4)
Pith says: minor objection 2 of 10
The constant -2 in the definition of S appears to be a normalization offset. The paper should explain how this constant is chosen (e.g., to center the distribution or ensure S is non-negative for typical values).
show 8 more
The paper says, at §3.2, 'Environment Evolution Loop'
Pith says: minor objection 3 of 10
The thresholds 0.5 for S and 0.01 for C are introduced without explanation. Since these thresholds determine when the evolution loop switches from repair to optimization, a brief justification (e.g., based on empirical distributions) would improve clarity.
The paper says, at §5.2, Table 1
Pith says: minor objection 4 of 10
The table reports gains relative to the base model Qwen3.5-35B-A3B. It would be informative to also report the absolute scores of the base model in the same table (currently they are listed separately in the 'Open-Weight Models (<200B)' section). Consider adding a row for the base model within the 'Ours' section for direct comparison.
The paper says, at §5.4, Table 2
Pith says: minor objection 5 of 10
The comparison with prior work (Table 2) uses different harnesses for published results vs. the authors' runs. The paper acknowledges this ('varying harnesses') but does not discuss how harness differences might affect scores. A brief note on the sensitivity of results to harness choice would be appropriate.
The paper says, at §5.5, Figure 5 (EEL)
Pith says: minor objection 6 of 10
The caption of Figure 5 mentions 'wall-clock time' but the x-axis label is not visible in the text. The paper should ensure the figure is readable and the axes are clearly labeled.
The paper says, at §5.5, Figure 6 (level-overview)
Pith says: minor objection 7 of 10
The ablation of S and C uses 1,000 randomly sampled environments. The paper should report whether the random sample is representative of the full dataset (e.g., by comparing distributions of S, C, and A).
The paper says, at §A.2, Table 3
Pith says: minor objection 8 of 10
The SFT hyperparameters are reported in the appendix. It would be helpful to state the total number of training steps or tokens seen during training, to allow reproducibility.
The paper says, at §2, Related Work
Pith says: minor objection 9 of 10
The claim of 'first' is strong. The paper should ensure that no prior work (e.g., from the program synthesis or curriculum learning literature) has used similar proxy metrics for environment evolution. If the authors are confident, the statement is acceptable, but it may be safer to soften to 'to our knowledge, the first' (which they already do).
The paper says, at §6, Limitations
Pith says: minor objection 10 of 10
The limitation section is appropriately candid. The paper could also mention that the current pipeline relies on Docker and specific build tools, which may limit applicability to some software ecosystems.
Open questions (4)
Pith raised these and could not confirm them against the text of the paper it read. They are questions for the authors, not findings.
- §3.1, Eq. (1)-(4) What is the justification for the multiplicative combination Q = A*C*S over alternatives such as additive or thresholded combinations, and how sensitive are the results to the specific thresholds chosen for S (0.5) and C (0.01)?
- §5.5, Figure 5 (EEL) Does the 'no evolution' baseline in the recovery experiment (Figure 5) include any repair steps from the construction pipeline before evolution, such as the initial validity check and one repair step?
- §5.5, Figure 6 (level-overview) How does the combined objective Q = A*C*S compare against optimizing S alone or C alone in terms of downstream accuracy on the same 1,000 environments?
- §5.2, Table 1 How many environments and trajectories were used to train the 'w/o environment evolution' baseline, and is it controlled to match the dataset size of the full pipeline?
What would settle it
Train a model on RepoFactory environments without the evolution loop (using only the construction pipeline) and compare against the full evolution loop on NL2Repo-Bench with multiple random seeds and reported confidence intervals. If the 5.1-point evolution gain is not statistically significant (e.g., overlapping confidence intervals), the claim that environment evolution adds meaningful value would be weakened.
from Pith's summary of the paper
How the review went
5 passes by deepseek-v4-flash · 18 Nov 2026
- Reader holds up, with conditions
- Skeptic concern not confirmed: listed as an open question
- Referee 10 minor
- Check against the paper kept 0 of 5 objections
- Editor wrote the verdict above
Not run on this paper: the simulated authors' defence.
Pith's passes raised 5 objections. A check against the paper's text kept 0 and left 5 as questions.
The review in full
The review in brief, machine-written
This paper does something useful and does it carefully. The core idea is straightforward: take real Python repositories, turn each into a task where an agent must reconstruct the whole project from a spec, and then run an evolution loop that tweaks the task to increase three cheap-to-measure properties—validity, structural coupling, and implementation scope. The result is 10,159 environments across 17 domains, and training Qwen3.5-35B-A3B on them produces a model that jumps 22.6 points on NL2Repo-Bench and transfers to SWE-bench and Terminal-Bench.
The controlled experiments are the paper's strength. The same base model, scaffold, and training procedure are used across ablations. The evolution loop is ablated: training without it still gains 17.5 points on NL2Repo, but evolution adds another 5.1. The data-source comparison (Table 2) is fair—same harness, same teacher, same trajectory filtering—and RepoFactory outperforms six open datasets. The paper also shows that optimizing scope and coupling individually improves downstream accuracy, which supports the proxy-metric approach.
The soft spots are real but not fatal. The evolution thresholds (S ≥ 0.5, C ≥ 0.01) and the multiplicative objective Q = A·C·S are not ablated; we don't know how sensitive results are to these choices. The paper reports single-run results without variance, so we can't assess stability. The evaluation uses one scaffold (Claude Code) and one base model family; how much the gains depend on that scaffold is unclear. The paper acknowledges the Python-only limitation and the coarse granularity of the proxy metrics.
Who this is for: researchers working on synthetic training environments for coding agents, especially those interested in automated data generation pipelines. The paper deserves a serious referee. The claims are concrete, the experiments are controlled, and the released dataset and model are valuable community assets. The open questions about threshold sensitivity and evaluation variance should be addressed in review, but they don't undermine the central result.
The stress test's concern
Not confirmed. Pith could not check this concern against the text of the paper it was given; it is listed as an open question, not a finding.
The referee's report
Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 4 of its 4 major objections did not survive that check.
Pith's reading of the paper (a machine's summary, not the authors')
The paper presents RepoFactory, an automated pipeline that converts real-world Python repositories into verifiable training environments where coding agents must reconstruct the entire project from scratch given a language specification. The key innovation is an Environment Evolution Loop that uses low-cost proxy metrics—validity, structural coupling (how interconnected the code's functions are), and implementation scope (code size and dependency breadth)—to iteratively improve both the reliability and training value of these environments without requiring expensive model rollouts. The authors claim that training a Qwen3.5-35B-A3B model on RepoFactory environments yields RepoHorizen-35B-A3B, which achieves gains of 22.6 percentage points on NL2Repo-Bench and 8.5 points on average across five benchmarks, with environment evolution contributing 5.1 and 2.1 points respectively. If true, this would demonstrate that automated environment synthesis guided by cheap proxy signals can substantially improve long-horizon coding agent performance at fixed model scale, and that the evolution loop adds meaningful value beyond simply having more training environments.
What carries the argument. The central object is the multiplicative environment objective Q = A * C * S, where A is a binary validity check (oracle passes, empty workspace fails), C = R/$N^{2}$ is structural coupling measured as normalized propagation cost in the static call graph, and S = log10(LOC) + log10(1+D) - 2 is implementation scope combining code size and dependency breadth. The Environment Evolution Loop uses this objective to guide iterative revisions to the oracle implementation, task specification, and tests, with up to 12 rounds of low-cost feedback from verifier execution and code analysis (no model rollouts). The loop first ensures validity, then optimizes S and C against thresholds (0.5 and 0.01 respectively), retaining revisions only if the targeted score strictly increases.
Core claim. The paper claims that an automated pipeline (RepoFactory) can convert real-world Python repositories into verifiable full-repository generation environments, and that an Environment Evolution Loop optimizing a multiplicative objective Q = A * C * S (validity, structural coupling, implementation scope) using low-cost proxy feedback can improve both the yield and training value of these environments. Training Qwen3.5-35B-A3B on the resulting 10,159 environments yields RepoHorizen-35B-A3B, which achieves gains of 22.6 percentage points on NL2Repo-Bench and 8.5 points on average across five benchmarks, compared with 17.5 and 6.4 points without environment evolution. The evolution loop recovers 69.4% of initially unsuitable repositories (vs 33.8% without) and produces environments with longer trajectories and more interdependent code, which the authors argue provides richer supervision for long-horizon coding agents.
The premise it rests on. The proxy metrics (structural coupling C and implementation scope S) are assumed to correlate with training value for long-horizon coding agents, but the paper does not establish that the specific functional form Q = A*C*S or the chosen thresholds (0.5 for S, 0.01 for C) are optimal, and does not ablate alternative formulations.
If the paper is right (Pith's inferences, not the authors')
If the proxy metrics (structural coupling and implementation scope) reliably predict training value, other environment synthesis pipelines could adopt similar low-cost objectives to guide task generation without expensive model rollouts.
The 22.6-point gain on NL2Repo-Bench suggests that full-repository generation tasks provide supervision that transfers to other software engineering benchmarks (SWE-bench, Terminal-Bench), potentially reducing the need for benchmark-specific training data.
The continued improvement with more environments (no saturation up to 5,577 environments) implies that scaling repository-level environments could yield further gains for long-horizon coding agents.
The evolution loop's ability to recover 69.4% of initially unsuitable repositories (vs 33.8% without) means that the effective yield from a given repository pool can be doubled, reducing the need to find ever more repositories.
The paper's focus on Python limits the generality of the claim; extending to other languages (Java, TypeScript, C++) would test whether the proxy metrics and evolution loop transfer across language ecosystems with different build systems and dependency structures.
The proxy metrics S and C are coarse structural measures; they do not distinguish between different capability requirements (e.g., algorithmic reasoning vs API orchestration). A finer-grained decomposition of task content could enable more targeted environment evolution for specific agent weaknesses.
The evolution loop's reliance on fixed thresholds (0.5 for S, 0.01 for C) and the multiplicative combination Q = A*C*S is not ablated against alternative functional forms; the paper does not establish that this specific formulation is optimal, leaving room for future work on adaptive or learned objectives.
The paper does not report variance or confidence intervals for the key results, making it difficult to assess the statistical significance of the 5.1-point evolution gain on NL2Repo-Bench; replication with multiple seeds would strengthen the claim.
Check this review
What was read
The passes were given all 70,037 characters of the paper's LaTeX source.
Fingerprint of that text (SHA-256): c176be922f0d6fa9deaaec783c32c92fdacba4057edbb7fb1f40828af8787575
Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:
PYTHONPATH=. python scripts/review_input_check.py 2611.21152
The passes, in the order they ran
- Reader deepseek-v4-flash · medium effort · $0.0031 · instructions of the Reader, fingerprint f9ead7300382…
- Skeptic deepseek-v4-flash · medium effort · $0.0049 · instructions of the Skeptic, fingerprint 8f88c0f61488…
- Referee deepseek-v4-flash · medium effort · $0.0024 · instructions of the Referee, fingerprint a852d99aea0d…
- Check against the paper deepseek-v4-flash · medium effort · $0.0089 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
- Editor deepseek-v4-flash · medium effort · $0.0021 · instructions of the Editor, fingerprint e7211f6a3d27…
- Circularity check deepseek-v4-flash · medium effort · $0.0020 · instructions of the Circularity check, fingerprint e9946e39d67b…
- Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0025 · instructions of the Summary for the page, fingerprint e4f56bf568e7…
The passes cost $0.026 in model calls together.
Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00
Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.
curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum
prints the fingerprint of the reader's instructions. The same works for every pass.
The receipt
The review's fingerprint was taken on 18 Nov 2026. The review you are reading still matches it.
Fingerprint of the review (SHA-256): 9f6c8afd181a01428dd53fcb5d1f062b3dda6c02cd779ac0e4f8e1d1ac86b667
curl -s 'https://pith.science/api/v1/verdicts/2611.21152?form=canonical' | sha256sum
prints the fingerprint of the review as it stands now. The rule, in four steps.
A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, from the paper's LaTeX source.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
How Pith checks papers · The full audit of this paper · Review history (3 versions)
The paper
Abstract
Training coding agents for long-horizon tasks requires environments with verifiable feedback. Existing pipelines often follow a linear process that ends with validation, establishing usability but providing little guidance on improving training value. Recursive self-improvement (RSI) often requires model rollouts or model training followed by downstream evaluation, with costs and feedback delays amplified on long-horizon tasks. How can we use timely, reliable proxy signals to improve training value and synthesize environments at scale? We present RepoFactory, an automated pipeline that converts real repositories into verifiable environments where agents build complete projects from scratch given a language specification. Its Environment Evolution Loop optimizes a multiplicative objective combining validity, structural coupling, and implementation scope through proxy feedback. RepoFactory produces 10,159 environments across 17 software domains. Training Qwen3.5-35B-A3B on these environments yields RepoHorizen-35B-A3B, which achieves gains of 22.6 percentage points on NL2Repo-Bench and 8.5 points on average across five benchmarks covering long-horizon coding and agentic tasks in real-world production settings, compared with 17.5 and 6.4 points without environment evolution. These results highlight the training value of RepoFactory environments and the additional gains from the Environment Evolution Loop.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-17
The authors' main claim, as the classifier picked it out 0.95
“These results highlight the training value of RepoFactory environments and the additional gains from the Environment Evolution Loop.”
How the main claim is stated: asserted 0.95
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.12 · human participants 0.03 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.93 · mentions code or data availability 0.05
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Training coding agents for long-horizon tasks requires environments with verifiable feedback. | background 1.00 | 0.06 | 0.03 | asserted 1.00 |
| Existing pipelines often follow a linear process that ends with validation, establishing usability but providing little guidance on improving training value. | gap 0.65 | 0.06 | 0.02 | asserted 0.98 |
| Recursive self-improvement (RSI) often requires model rollouts or model training followed by downstream evaluation, with costs and feedback delays amplified on long-horizon tasks. | background 0.94 | 0.05 | 0.02 | asserted 0.98 |
| How can we use timely, reliable proxy signals to improve training value and synthesize environments at scale? | gap 0.94 | 0.03 | 0.02 | unclear 0.98 |
| We present RepoFactory, an automated pipeline that converts real repositories into verifiable environments where agents build complete projects from scratch given a language specification. | contribution 0.75 | 0.95 | 0.07 | asserted 1.00 |
| Its Environment Evolution Loop optimizes a multiplicative objective combining validity, structural coupling, and implementation scope through proxy feedback. | method 0.98 | 0.90 | 0.09 | asserted 1.00 |
| RepoFactory produces 10,159 environments across 17 software domains. | result 0.78 | 0.95 | 0.05 | asserted 1.00 |
| Training Qwen3.5-35B-A3B on these environments yields RepoHorizen-35B-A3B, which achieves gains of 22.6 percentage points on NL2Repo-Bench and 8.5 points on average across five benchmarks covering long-horizon coding and agentic tasks in real-world production settings, compared with 17.5 and 6.4 points without environment evolution. | result 1.00 | 0.96 | 0.54 | asserted 1.00 |
| These results highlight the training value of RepoFactory environments and the additional gains from the Environment Evolution Loop. | interpretation 1.00 | 0.88 | 0.67 | asserted 0.95 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-17: 1 request, 6,945 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (8)
Figures are the authors', reproduced from the paper.
References (71)
- [1] FirstName LastName , title = on Pith
- [2] FirstName Alpher , title = on Pith
- [3] FirstName Alpher and FirstName Fotheringham-Smythe , title =. Journal of Foo , volume = 13, number = 1, pages = on Pith
- [4] FirstName Alpher and FirstName Fotheringham-Smythe and FirstName Gamow , title =. Journal of Foo , volume = 14, number = 1, pages = on Pith
- [5] FirstName Alpher and FirstName Gamow , title = on Pith
- [6] AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks? , author=. arXiv preprint arXiv:2606.05080 , year= on PithPith's reviewarXiv
- [7] WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? , author=. 2024 , eprint= on Pith
- [8] WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks , author=. 2024 , eprint= on Pith
- [9] Deepseek-v4: Towards highly efficient million-token context intelligence, 2026 , author= on Pith
- [10] Recursive synthesis for long-horizon terminal tasks , author=. arXiv preprint arXiv:2608.05466 , year= on PithPith's reviewarXiv
- [11] From language models to agentic ai: A survey of autonomous, action-enabled, and collaborative llm agents , author=. Cognitive Computation , volume=. 2026 , publisher= on Pith
- [12] Beyond function calling: Benchmarking tool-using agents under tool-environment unreliability , author=. arXiv preprint arXiv:2606.25819 , year= on PithPith's reviewarXiv
- [13] Jian Yang and Wei Zhang and Jiajun Wu and Junhang Cheng and Shawn Guo and Haowen Wang and Weicheng Gu and Yaxin Du and Joseph Li and Fanglin Xu and Yizhi Li and Lin Jing and Yuanbo Wang and Yuhan Gao and Ruihao Gong and Chuan Hao and Ran Tao and Aishan Liu and Tuney Zheng and ... on Pithdoi:10.48550/arxiv.2603.16790
- [14] Deepseek-v4. 1-flash: Pushing the limits of kv cache compression , author=. arXiv preprint arXiv:2609.19969 , year= on PitharXiv
- [15] Kimi k3: Open frontier intelligence , author=. arXiv preprint arXiv:2607.24653 , year= on PitharXiv
- [16] Weili Cao and Xunjian Yin and Bhuwan Dhingra and Shuyan Zhou , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.20432 , eprinttype =. 2603.20432 , timestamp = on Pithdoi:10.48550/arxiv.2603.20432
- [17] Zhaoyang Wang and Canwen Xu and Boyi Liu and Yite Wang and Siwei Han and Zhewei Yao and Huaxiu Yao and Yuxiong He , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2602.10090 , eprinttype =. 2602.10090 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2602.10090
- [18] Yuxin Zuo and Zikai Xiao and Li Sheng and Fei Huang and Jianhong Tu and Yuxuan Liu and Tianyi Tang and Xiaomeng Hu and Yang Su and Qingfeng Lan and Yantao Liu and Qin Zhu and Yinger Zhang and Bowen Yu and Haiquan Zhao and Haiyang Xu and Jianxin Yang and Jiayang Cheng and Junya... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.24597
- [19] Yang Tian and Rui Wang and Xumeng Wen and Junjie Li and Shizhao Sun and Lei Song and Jiang Bian and Bo Zhao , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.09348 , eprinttype =. 2606.09348 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2606.09348
- [20] Chenxu Yang and Chuanyu Qin and Qingyi Si and Minghui Chen and Naibin Gu and Dingyu Yao and Zheng Lin and Weiping Wang and Jiaqi Wang and Nan Duan , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2604.03128 , eprinttype =. 2604.03128 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2604.03128
show 51 more
- [21] Xiaoshuai Song and Haofei Chang and Guanting Dong and Yutao Zhu and Ji. EnvScaler: Scaling Tool-Interactive Environments for. Findings of the Association for Computational Linguistics,. 2026 , url =. doi:10.18653/V1/2026.FINDINGS-ACL.407 , timestamp = on Pithdoi:10.18653/v1/2026.findings-acl.407
- [22] Dunwei Tu and Hongyan Hao and Hansi Yang and Yihao Chen and Yi. ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training , journal =. 2026 , url =. doi:10.48550/ARXIV.2602.06820 , eprinttype =. 2602.06820 , timestamp = on Pithdoi:10.48550/arxiv.2602.06820
- [23] Fei Bai and Huatong Song and Shuang Sun and Daixuan Cheng and Yike Yang and Chuan Hao and Renyuan Li and Feng Chang and Yuan Wei and Ran Tao and Bryan Dai and Jian Yang and Wayne Xin Zhao , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2604.26904 , eprinttype =. 2... on PithPith's reviewarXivdoi:10.48550/arxiv.2604.26904
- [24] GLM-5: from Vibe Coding to Agentic Engineering. arXiv preprint (2026) , author=. arXiv preprint arXiv:2602.15763 , year= on PithPith's reviewarXiv
- [25] Guanting Dong and Junting Lu and Junjie Huang and Wanjun Zhong and Longxiang Liu and Shijue Huang and Zhenyu Li and Yang Zhao and Xiaoshuai Song and Xiaoxi Li and Jiajie Jin and Yutao Zhu and Hanbin Wang and Fangyu Lei and Qinyu Luo and Mingyang Chen and Zehui Chen and Jiazhan... on PithPith's reviewarXivdoi:10.48550/arxiv.2604.18292
- [26] Zhiyuan Fan and Tinghao Yu and Yuanjun Cai and Jiangtao Guan and Yun Yang and Dingxin Hu and Jiang Zhou and Xing Wu and Zhuo Han and Feng Zhang and Lilin Wang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2604.25727 , eprinttype =. 2604.25727 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2604.25727
- [27] Hamish Ivison and Junjie Oscar Yin and Rulin Shao and Teng Xiao and Nathan Lambert and Hannaneh Hajishirzi , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.23321 , eprinttype =. 2606.23321 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2606.23321
- [28] Zhongzhi Li and Yucheng Shi and Zongxia Li and Ruhan Wang and Anhao Li and Zixun Huang and Junyao Yang and Lei Ke and Ninghao Liu and Haitao Mi and Leowei Liang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2608.05466 , eprinttype =. 2608.05466 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2608.05466
- [29] Mingguang Chen and Licheng Wang and Bo Qu , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2607.07663 , eprinttype =. 2607.07663 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.07663
- [30] Jenny Zhang and Shengran Hu and Cong Lu and Robert T. Lange and Jeff Clune , editor =. Darwin G. The Fourteenth International Conference on Learning Representations,. 2026 , url = on Pith
- [31] Jenny Zhang and Bingchen Zhao and Wannan Yang and Jakob N. Foerster and Jeff Clune and Minqi Jiang and Sam Devlin and Tatiana Shavrina , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.19461 , eprinttype =. 2603.19461 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2603.19461
- [32] A survey of agent memory in the second half: Towards self-evolving and long-horizon agents , author=. Transactions on Machine Learning Research , year= on Pith
- [33] Humanity's last exam , author=. arXiv preprint arXiv:2501.14249 , year= on PithPith's reviewarXiv
- [34] Agents’ last exam, 2026 , author=. URL https://arxiv. org/abs/2606.05405 , year= on PitharXiv
- [35] Tingyang Chen and Shuo Lu and Kang Zhao and Weicheng Meng and Hanlin Teng and Tianhao Li and Chao Li and Xule Liu and Jian Liang and Zhizhong Zhang and Yuan Xie and Heng Qu and Kun Shao and Jian Luan , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.14249 , epr... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.14249
- [36] Charles Packer and Vivian Fang and Shishir G. Patil and Kevin Lin and Sarah Wooders and Joseph E. Gonzalez , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2310.08560 , eprinttype =. 2310.08560 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2310.08560
- [37] Ilia Kulikov and Chenxi Whitehouse and Tianhao Wu and Yixin Nie and Swarnadeep Saha and Eryk Helenowski and Weizhe Yuan and Olga Golovneva and Jack Lanchantin and Yoram Bachrach and Jakob N. Foerster and Xian Li and Han Fang and Sainbayar Sukhbaatar and Jason Weston , title =.... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.25996
- [38] Jincheng Ren and Siwei Wu and Yizhi Li and Kang Zhu and Shu Xu and Boyu Feng and Ruibin Yuan and Wei Zhang and Riza Batista. A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression , journal =. 2026 , url =. doi:10.48550/ARXIV.2604.19572 ,... on PithPith's reviewarXivdoi:10.48550/arxiv.2604.19572
- [39] Yiqi Zhu and Apurva Gandhi and Graham Neubig , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2512.12216 , eprinttype =. 2512.12216 , timestamp = on Pithdoi:10.48550/arxiv.2512.12216
- [40] Kanishk Gandhi and Shivam Garg and Noah D. Goodman and Dimitris Papailiopoulos , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2601.16443 , eprinttype =. 2601.16443 , timestamp = on Pithdoi:10.48550/arxiv.2601.16443
- [41] Xiaoxuan Peng and Kaiqi Zhang and Xinyu Lu and Boxi Cao and Yaojie Lu and Hongyu Lin and Xianpei Han and Le Sun , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.29559 , eprinttype =. 2605.29559 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2605.29559
- [42] Zhanbo Hua and Yifan Yao and Weihao Xie and Yongchi Zhao and Minghao Liu and Ruizhi Qiu and Zhewei Huang and Zun Wang and Yiyan Ji and Yunhai Ye and Letian Zhu and Xinping Lei and Han Li and Zhiyuan Ma and Zili Wang and Zhaoxiang Zhang and Jiaheng Liu , title =. CoRR , volume ... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.22883
- [43] Zixiang Chen and Yihe Deng and Huizhuo Yuan and Kaixuan Ji and Quanquan Gu , editor =. Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models , booktitle =. 2024 , url = on Pith
- [44] Zongxia Li and Hongyang Du and Chengsong Huang and Xiyang Wu and Lantao Yu and Yicheng He and Jing Xie and Xiaomin Wu and Zhichao Liu and Jiarui Zhang and Fuxiao Liu , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.09206 , eprinttype =. 2603.09206 , timestamp = on Pithdoi:10.48550/arxiv.2603.09206
- [45] Yangzhen Wu and Aaron J. Li and Wenjie Ma and Li Cao and Ziheng Zhou and Mert Cemri and Shu Liu and Yuran Xiu and Chenxiao Yan and Haikun Zhao and Bin Yu and Ion Stoica and Dawn Song , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.01286 , eprinttype =. 2606.0... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.01286
- [46] Qijia Shen and Zhiqi Huang and Vamsidhar Kamanuru and Aznaur Aliev and Jay Rainton and Ahmed Awelkair and Zhichen Zeng and Jiajun Li and Shi Dong and Yueming Yuan and Boyuan Ma and Qizheng Zhang and Jiwei Fu and Yuzhen Mao and Wendong Fan and Ping Nie and Philip Torr and Berna... on PithPith's reviewarXivdoi:10.48550/arxiv.2607.10891
- [47] Jingzhe Ding and Shengda Long and Changxin Pu and Huan Zhou and Hongwan Gao and Xiang Gao and Chao He and Yue Hou and Fei Hu and Zhaojian Li and Weiran Shi and Zaiyuan Wang and Daoguang Zan and Chenchen Zhang and Xiaoxu Zhang and Qizhi Chen and Xianfu Cheng and Bo Deng and Qin... on Pithdoi:10.48550/arxiv.2512.12730
- [48] Xiang Deng and Jeff Da and Edwin Pan and Yannis Yiming He and Charles Ide and Kanak Garg and Niklas Lauffer and Andrew Park and Nitin Pasari and Chetan Rane and Karmini Sampath and Maya Krishnan and Srivatsa Kundurthy and Sean Hendryx and Zifan Wang and Chen Bo Calvin Zhang an... on PithPith's reviewarXivdoi:10.48550/arxiv.2509.16941
- [49] Daoguang Zan and Zhirong Huang and Wei Liu and Hanwu Chen and Shulin Xin and Linhao Zhang and Qi Liu and Aoyan Li and Lu Chen and Xiaojian Zhong and Siyao Liu and Yongsheng Xiao and Liangqiang Chen and Yuyu Zhang and Jing Su and Tianyu Liu and Rui Long and Ming Ding and Liang ... on Pith
- [50] Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces , author=. International Conference on Learning Representations , volume= on Pith
- [51] Carlos E. Jimenez and John Yang and Alexander Wettig and Shunyu Yao and Kexin Pei and Ofir Press and Karthik R. Narasimhan , title =. The Twelfth International Conference on Learning Representations,. 2024 , url = on Pith
- [52] Jiayi Pan and Xingyao Wang and Graham Neubig and Navdeep Jaitly and Heng Ji and Alane Suhr and Yizhe Zhang , editor =. Training Software Engineering Agents and Verifiers with SWE-Gym , booktitle =. 2025 , url = on Pith
- [53] Naman Jain and Manish Shetty and Tianjun Zhang and King Han and Koushik Sen and Ion Stoica , editor =. Forty-first International Conference on Machine Learning,. 2024 , url = on Pith
- [54] Minh V. T. Thai and Tue Le and Dung Nguyen Manh and Huy Phan Nhat and Nghi D. Q. Bui , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2512.18470 , eprinttype =. 2512.18470 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2512.18470
- [55] Jiarong Liang and Zhiheng Lyu and Zijie Liu and Xiangchao Chen and Ping Nie and Kai Zou and Wenhu Chen , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.20691 , eprinttype =. 2603.20691 , timestamp = on Pithdoi:10.48550/arxiv.2603.20691
- [56] Naman Jain and Jaskirat Singh and Manish Shetty and Liang Zheng and Koushik Sen and Ion Stoica , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2504.07164 , eprinttype =. 2504.07164 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2504.07164
- [57] Ibragim Badertdinov and Maksim Nekrashevich and Anton Shevtsov and Alexander Golubev , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2602.23866 , eprinttype =. 2602.23866 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2602.23866
- [58] Dayuan Fu and Shenyu Wu and Yunze Wu and Zerui Peng and Yaxing Huang and Jie Sun and Ji Zeng and Mohan Jiang and Lin Zhang and Yukun Li and Jiarui Hu and Liming Liu and Jinlong Hou and Pengfei Liu , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.13023 , eprint... on PithPith's reviewarXivdoi:10.48550/arxiv.2603.13023
- [59] John Yang and Kilian Lieret and Carlos E. Jimenez and Alexander Wettig and Kabir Khandpur and Yanzhe Zhang and Binyuan Hui and Ofir Press and Ludwig Schmidt and Diyi Yang , editor =. SWE-smith: Scaling Data for Software Engineering Agents , booktitle =. 2025 , url = on Pith
- [60] Jiale Zhao and Guoxin Chen and Fanzhe Meng and Minghao Li and Jie Chen and Hui Xu and Yongshuai Sun and Xin Zhao and Ruihua Song and Yuan Zhang and Peng Wang and Cheng Chen and Jirong Wen and Kai Jia , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2602.09892 , epr... on Pithdoi:10.48550/arxiv.2602.09892
- [61] AI4AI at scale: A full-pipeline system for enhancing LLM agentic capabilities , author=. 2026 , institution= on Pith
- [62] IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems , author=. arXiv preprint arXiv:2610.30949 , year= on PitharXiv
- [63] Swe-factory: Your automated factory for issue resolution training data and evaluation benchmarks , author=. arXiv preprint arXiv:2506.10954 , year= on PitharXiv
- [64] Swe-smith: Scaling data for software engineering agents , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [65] Swe-rebench: An automated pipeline for task collection and decontaminated evaluation of software engineering agents , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [66] DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratch , author=. arXiv preprint arXiv:2606.10728 , year= on PithPith's reviewarXiv
- [67] MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis , author=. arXiv preprint arXiv:2607.27146 , year= on PithPith's reviewarXiv
- [68] ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agents , author=. arXiv preprint arXiv:2610.13395 , year= on PithPith's reviewarXiv
- [69] Yaxin Du and Xiyuan Yang and Zhifan Zhou and Wanxu Liu and Zixing Lei and Zimeng Chen and Fenyi Liu and Haotian Wu and Yuzhu Cai and Zexi Liu and Xinyu Zhu and Wenhao Wang and Linfeng Zhang and Chen Qian and Siheng Chen , title =. CoRR , volume =. 2026 , url =. doi:10.48550/AR... on PithPith's reviewarXivdoi:10.48550/arxiv.2605.10906
- [70] Software Engineering LLMs: Why We Need Them and How Software Engineering Helps Build Them , author=. arXiv preprint arXiv:2610.33851 , year= on PithPith's reviewarXiv
- [71] TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training , author=. arXiv preprint arXiv:2610.04973 , year= on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment