Pith.

Software Engineering · arXiv:2610.35333 v1 · submitted 26 Oct 2026 · 32 references

RepoTrace: Improving Requirement Tracing in Multi-Round Agent Interactions

Huanyu Liu, Yezheng Cheng, Jinliang You, Tianxiao Huang, Linyu Li, Zheng Fang, Peng Cao, Ge Li

The paper says

“Code agents have become increasingly capable of performing software engineering tasks through interaction with software environments. However, agents tend to degrade as the number of interaction rounds increases, making it difficult to maintain existing software during iterative modification.”

more of the abstract

We analyze 2,656 agent trajectories in EvoCode-Bench and identify requirement focus misalignment as a recurring failure pattern, where agents struggle to distinguish the primary modification targets of a requirement from unrelated code. We propose Repotrace a requirement-driven repository understanding harness that combines repository structure with requirement semantics to construct a requirement-code correspondence and injects it into the agent context. This correspondence enables code agents to follow the repository architecture, trace requirements across multi-round interactions, and identify the core code elements corresponding to the current requirement. Experiments show that Repotrace improves the mean per-round reward (MPR) across different models (e.g., +19.99 on GLM-5.2) and agent harnesses (e.g., +10.22 on Claude Code). Across rounds, Repotrace increases the average per-round reward from 0.163 to 0.371 on Terminus-2 with DeepSeek-V4-Pro, while reducing average regression errors by 56.78%. We release the source code and agents' trajectories to support future research.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 27 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A plausible, honestly-reported harness for multi-round requirement tracing whose headline MPR gains rest on one run over 26 tasks; the regression-reduction results are the more convincing part. This line is wrong

3 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 32 references say.

The paper says, at §5.1, Table 1

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The central quantitative claims are not supported by uncertainty quantification. As reported, each condition appears to be a single run on 26 tasks, with no per-task breakdown, confidence intervals, or significance tests. Since a round is scored as passed only when all cumulative test cases pass and rounds within a task share a workspace and session, the effective independent sample size is at most 26 tasks per condition, not 227 rounds.

more of objection 1

One task moving from 0 to 1 changes MPR by roughly 3.85 points, so the +22.25 gain on DeepSeek-V4-Pro could be driven by about six tasks; consistent with this, Qwen3.7-Max is flat (11.87 to 11.36). The authors should report per-task deltas, bootstrap confidence intervals or a paired significance test, and, ideally, repeated runs.

The paper says, at §5.2, Table 1

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The efficiency and cross-model claims are not uniformly supported by Table 1. On MiniMax-M3, RepoTrace raises MPR by only +4.91 while input tokens more than double (726.66M to 1445.57M) and FCR drops from 23.08 to 15.38. The text in RQ1 states that IT decreases on the strongest performer, but this does not address the MiniMax cost increase or the FCR decrease. The authors should discuss this case explicitly and qualify the claim that the token overhead is acceptable across models.

The paper says, at §5 (RQ1-RQ4), Appendix A.3

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

There is no ablation separating the contribution of the structural representation from the contribution of the injected context in general. The message in Appendix A.3 contains generic instructions such as 'Keep edits inside the target components + impact scope. Do NOT modify unrelated components,' and the prior-requirement history is itself a substantial intervention.

more of objection 3

In the only fully traced example, the architecture snapshot reports 0 dependency edges (E7 = ∅), so PageRank and fan-in did no work in that case and the extracted targets could have come from LLM requirement parsing alone. I ask for ablations such as (a) the injected message without the structural section, (b) the message with only the generic preservation instructions, and (c) LLM extraction without repository structure, so that the claimed role of repository structure can be attributed.

4 minor objections

The paper says, at Abstract, §1

Pith says: minor objection 1 of 4

The name is spelled 'Repotrace' in the abstract and 'RepoTrace' elsewhere; please standardize the spelling throughout.

The paper says, at Table 2

Pith says: minor objection 2 of 4

The column header 'IO↓' appears to be a typo for 'IT↓'; the table should use the same token abbreviations as Table 1.

show 2 more

The paper says, at Appendix B

Pith says: minor objection 3 of 4

The 30-trajectory calibration reports a static estimate of 32.26% and a replay estimate of 34.29%, which are far below the full-corpus static estimate of 55.16%. Please explain the representativeness of the calibration subset or report a larger calibration sample so readers can interpret the discrepancy.

The paper says, at §4.1-4.2, Appendix A

Pith says: minor objection 4 of 4

The concrete values of the free parameters (n_top for the extraction prompt, truncation sizes, PageRank iterations and damping factor, and the keyword fallback rules) are not reported in the main text or appendix; please state them so the experiments are reproducible without inspecting the code.

What would settle it

From the released trajectories, recompute per-task MPR for Terminus-2 + DeepSeek-V4-Pro with and without RepoTrace across all 26 tasks, then run a paired bootstrap (10,000 resamples) on the task-level MPR deltas and a sign test over tasks. Report the 95% confidence interval for the mean delta and the number of tasks with positive versus negative deltas; do the same for GLM-5.2. If the confidence interval includes zero, or fewer than about 18 of 26 tasks improve, the headline +22.25/+19.99 gains are not established at task level. If the released trajectories do not cover both conditions on all 26 tasks, that missing coverage is itself a finding.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 27 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 1 min 26 s · 19,710 tokens read, 11,024 written
  3. Editor wrote the verdict above 27 s · 20,756 tokens read, 3,735 written
  4. Referee major revision: 3 major objections, 4 minor 1 min 14 s · 20,233 tokens read, 6,095 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

What you should know first: RepoTrace is a harness, not a new model or benchmark. It builds a file-level dependency graph (PageRank plus fan-in), extracts requirement entities with an LLM, keeps a cross-round requirement history, and injects the resulting requirement-code correspondence into the agent's context before each modification. The combination is a legitimate extension of existing localization and retrieval work, and the paper's failure-pattern analysis — requirement focus misalignment, with 55.16% of agent modifications falling outside the oracle's target files — is a useful addition to the multi-round-degradation literature.

The writing is honest. Appendix B calls the 55.16% a static estimate rather than replay ground truth, discloses the parser coverage, and reports a 30-trajectory calibration (32.26% static vs 34.29% replay). Appendix A.3 admits the one fully worked example had zero dependency edges, meaning the structural component did nothing in that case. That candor earns trust.

The soft spots are statistical rather than conceptual. The evidence is one run per condition on 26 tasks. Rounds share a session, so tasks — not rounds — are the independent units; one task moving from fail to pass shifts MPR by about 3.8 points, so the +22.25 delta could be five or six tasks. There are no confidence intervals, significance tests, or per-task breakdowns. Table 1 itself shows the effect is not uniform: Qwen is flat, and MiniMax gains +4.91 while input tokens roughly double and FCR drops. The abstract's "across different models" overstates that table. And there is no ablation separating the structural correspondence from the generic "preserve previous behavior, keep edits inside target components" instruction — which matters given the zero-edge worked example.

The regression story holds up better: the mean pass rate on earlier rounds' tests stays above 80 across all rounds with RepoTrace, versus a collapse to 14.62 without it, and the RQ4 case study replays both trajectories to confirm outcomes. That part is consistent with the mechanism.

This paper deserves a serious referee. For people building or evaluating code-agent harnesses, it is worth reading and citing. Ask for per-task distributions, some repeated runs or at least a bootstrap, and an ablation that drops the structural component — then the central claim should be testable at the right scale.

The strongest objection

The central claim is quantitative: RepoTrace improves MPR by +19.99 on GLM-5.2 and +22.25 on Terminus-2/DeepSeek-V4-Pro, and raises the average per-round reward from 0.163 to 0.371 while cutting regression errors by 56.78% (Table 1, Fig. 3, RQ3). The evidence behind these numbers is one run per condition on EvoCode-Bench, which has only 26 tasks and 227 rounds, and a round is binary: it counts as passed only when all test cases pass (Sec. 5.1). Because rounds within a task share a workspace and session, the effective independent units are tasks, not rounds. With 26 tasks, one task moving from 0% to 100% changes MPR by roughly 3.8 points, so the +22.25-point delta could be driven by about 5-6 tasks. Table 1 also shows the effect is not uniform: Qwen3.7-Max is flat (11.87 to 11.36), and MiniMax gains only +4.91 while input tokens roughly double and FCR drops from 23.08 to 15.38. The paper reports no per-task deltas, no confidence intervals, no significance tests, and no repeated runs. This does not disprove the method; it means the headline claim is not yet established at the only scale tested. A secondary attribution gap compounds this: there is no ablation separating the structural correspondence from the generic 'preserve previous behavior / keep edits inside target components' instruction in the injected message, and the one fully worked example reports E_7 = ∅, i.e., zero dependency edges, so the structural component did no work in that case (Appendix A.3).

The paper is honest and unusually self-aware: Appendix B discloses that the 55.16% off-oracle figure is a static estimate, calibrates it against a 30-trajectory replay, and states that graph snapshots and requirement records do not certify completion. Appendix A.3 even flags that an empty dependency graph makes the phrase 'leaf components' potentially misleading. Released trajectories and the replayed JobForge case are real supporting evidence. My concern is not about integrity or the plausibility of the method; it is about the amount of evidence behind the headline numbers. The benchmark has 26 tasks, the round-level reward is binary, and rounds within a task are correlated, so aggregating over 227 rounds without showing the task-level distribution can hide a result driven by a handful of tasks. The reported gains are large, and a paired per-task analysis could well confirm them; that is exactly why a paired bootstrap is the right check rather than a request for a new benchmark. The reader's stated weakest assumption was the oracle-scope and static-extraction issue; that concern is substantially addressed by the paper's own calibration and caveats. The more load-bearing gap is the missing per-task uncertainty analysis, which the reader noted in the rationale but did not make the central weakest assumption. I therefore keep the CONDITIONAL verdict: the method deserves conditional acceptance and artifact-based verification, but the abstract's across-model claim should be qualified given the flat Qwen result, and the headline gains should not be treated as established until per-task statistics are reported.

The referee's report

If the reported gains are robust, RepoTrace is a useful and lightweight contribution: it is model-agnostic, requires no model retraining, and the release of source code and agent trajectories supports reproducibility. The paper also handles several limitations of its observational analysis honestly, explicitly stating that the 55.16% off-oracle statistic is a static estimate, that the oracle is not the only valid implementation, and that graph snapshots do not certify completion. The main gap is evidentiary: the evaluation rests on a single 26-task benchmark, the quantitative claims include no uncertainty quantification, and no ablation isolates the structural component from generic context injection. The method is therefore plausible and directionally well supported, but the headline quantitative claims are not yet established at the reported scale.

Pith's reading of the paper (a machine's summary, not the authors')

The paper tries to establish that the main reason code agents get worse the longer they work on a repository is requirement focus misalignment: agents either edit code that has nothing to do with the current requirement, or they update one file and miss the file that must change together with it. Supporting this diagnosis, the authors analyze 2,656 agent trajectories from EvoCode-Bench and report that across failed rounds 55.16% of agent code modifications fall outside the files the human-written reference solution changed. To fix this, they propose RepoTrace, a harness that builds a structural map of the repository, extracts the requirement's semantic targets, and injects a requirement-code correspondence into the agent's context before each action. They report that this raises mean per-round reward across models and harnesses, for example by +19.99 points on GLM-5.2 and +10.22 on Claude Code, and that on Terminus-2 with DeepSeek-V4-Pro it lifts average per-round reward from 0.163 to 0.371 while cutting average regression errors by 56.78%. If these numbers hold, a large share of multi-round agent failure is not missing model capability but missing context, and a cheap harness-level intervention can restore it.

What carries the argument. The load-bearing object is the requirement-code correspondence $M_t$, a text message composed from two ingredients before the agent acts in round $t$. The first ingredient is the structural representation $G_t$: a directed file graph built from imports and matched call sites, a symbol-to-file index, structural weights from PageRank and fan-in, and a structural-change summary $\Delta_t$ listing files and relations added or removed since the last retained snapshot. The second is the extracted requirement entity record $Q_t$: candidate target files and symbols, behavior constraints, prior code elements the new requirement may affect, and a summary, obtained by an LLM prompt (or a keyword fallback) that consults the current requirement, the structure, and records from earlier requirements. Composing $G_t$ and $Q_t$ yields a message that names the files to create or modify, distinguishes new files from existing ones, lists prior components that must keep working, and gives verification guidance; RepoTrace then saves the graph snapshot and appends the requirement record to history so tracing persists across rounds. The mechanism's work is to draw an explicit edit boundary in advance, so the agent does not have to rediscover the repository's architecture during a long interaction.

Core claim. On the paper's own terms, the central discovery is a failure pattern plus a fix. The failure pattern: in multi-round software evolution, agents lose track of which code a requirement actually targets, so they either modify unrelated code (as when an agent rewrote a shared command dispatcher and accidentally dropped an existing command) or they make a partial update (as when an agent added an output field but not the code that fills it). The fix is to make requirement tracing explicit: before each round, RepoTrace constructs a structural representation of the repository from import and call relations with PageRank and fan-in weights, extracts candidate target files, symbols, and behavior constraints from the requirement while carrying prior requirement records across rounds, and composes these into a requirement-code correspondence message the agent receives alongside the requirement. The message tells the agent which files are new versus existing, which prior components must keep working, and where the edit boundary lies. On EvoCode-Bench, the paper reports that this raises mean per-round reward by +19.99 on GLM-5.2 and +4.91 on MiniMax-M3 across models, by +22.25 on Terminus-2 and +10.22 on Claude Code across harnesses, and across rounds keeps earlier rounds' tests above 80% accuracy on the main configuration while the unhooked agent falls to 14.62%, a 56.78% relative reduction in average regression errors.

The premise it rests on. The central claim rests on EvoCode-Bench's reference solutions: if the oracle-modified file sets and cumulative tests are not a fair measure of what an agent should change, then both the 55.16% off-oracle diagnosis and the measured reward gains are computed against the wrong yardstick.

If the paper is right (Pith's inferences, not the authors')

Multi-round degradation is partly an input-context failure rather than pure model failure: models as different as GLM-5.2, MiniMax-M3, DeepSeek-V4-Pro, and Qwen3.7-Max run in the same harness, yet the ones that can exploit repository-level guidance show large mean-per-round-reward gains from an added message.

Iterative maintenance becomes safer: on the main configuration, regression accuracy on earlier rounds' tests stays above 80% in every round with RepoTrace, versus falling from 84.68 to 14.62 without it, so earlier achievements survive later edits.

The intervention can pay for itself in tokens: on DeepSeek-V4-Pro with Terminus-2, input tokens drop by 27.61% while mean per-round reward nearly triples, because the guidance prunes redundant codebase exploration.

Carrying requirement records and structural-change summaries across rounds is what lets the agent distinguish already-implemented behavior from newly demanded behavior, which is what keeps cumulative test cases passing rather than just one-shot localization working.

The harness-level design means the method transfers without modifying the agent's core loop: both a weaker harness (Terminus-2) and a stronger one (Claude Code) improve, suggesting the correspondence complements rather than replaces the harness's own context management.

My extension: if the diagnosis generalizes, the off-oracle rate becomes a cheap scope-control metric for other multi-round benchmarks, and harnesses whose injected context constrains edit boundaries should measurably lower it without needing full test replays.

My extension: RepoTrace's structural weights double as a change-impact predictor, since high-PageRank, high-fan-in files are the ones whose editing should break dependents; comparing predicted dependents against actual per-round test failures would validate or refine the weighting scheme.

My extension: because the correspondence is plain text in the prompt, it acts as external memory that survives context truncation; a testable prediction is that the per-round benefit grows with total conversation length, since the harness is compensating for exactly the memory the agent loses.

My extension: the motivating examples suggest the method matters most in tasks with shared hub files such as a command dispatcher, where off-target edits are silent regressions; removing the structural section of the message in an ablation should hurt most on such tasks, isolating which part of the correspondence carries the gain.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Code agents have become increasingly capable of performing software engineering tasks through interaction with software environments. However, agents tend to degrade as the number of interaction rounds increases, making it difficult to maintain existing software during iterative modification. We analyze 2,656 agent trajectories in EvoCode-Bench and identify requirement focus misalignment as a recurring failure pattern, where agents struggle to distinguish the primary modification targets of a requirement from unrelated code. We propose Repotrace a requirement-driven repository understanding harness that combines repository structure with requirement semantics to construct a requirement-code correspondence and injects it into the agent context. This correspondence enables code agents to follow the repository architecture, trace requirements across multi-round interactions, and identify the core code elements corresponding to the current requirement. Experiments show that Repotrace improves the mean per-round reward (MPR) across different models (e.g., +19.99 on GLM-5.2) and agent harnesses (e.g., +10.22 on Claude Code). Across rounds, Repotrace increases the average per-round reward from 0.163 to 0.371 on Terminus-2 with DeepSeek-V4-Pro, while reducing average regression errors by 56.78%. We release the source code and agents' trajectories to support future research.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-27

The authors' main claim, as the classifier picked it out 0.86

“Experiments show that Repotrace improves the mean per-round reward (MPR) across different models (e.g., +19.99 on GLM-5.2) and agent harnesses (e.g., +10.22 on Claude Code).”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.11 · human participants 0.04 · controlled comparison 0.76 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.74 · mentions code or data availability 0.98

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Code agents have become increasingly capable of performing software engineering tasks through interaction with software environments. background 1.00 0.04 0.02 asserted 0.99
However, agents tend to degrade as the number of interaction rounds increases, making it difficult to maintain existing software during iterative modification. background 0.79 0.11 0.03 asserted 0.88
We analyze 2,656 agent trajectories in EvoCode-Bench and identify requirement focus misalignment as a recurring failure pattern, where agents struggle to distinguish the primary modification targets of a requirement from unrelated code. result 0.47 0.97 0.08 asserted 1.00
We propose Repotrace a requirement-driven repository understanding harness that combines repository structure with requirement semantics to construct a requirement-code correspondence and injects it into the agent context. method 0.74 0.93 0.11 asserted 1.00
This correspondence enables code agents to follow the repository architecture, trace requirements across multi-round interactions, and identify the core code elements corresponding to the current requirement. method 0.48 0.54 0.20 asserted 0.98
Experiments show that Repotrace improves the mean per-round reward (MPR) across different models (e.g., +19.99 on GLM-5.2) and agent harnesses (e.g., +10.22 on Claude Code). result 1.00 0.96 0.35 asserted 1.00
Across rounds, Repotrace increases the average per-round reward from 0.163 to 0.371 on Terminus-2 with DeepSeek-V4-Pro, while reducing average regression errors by 56.78%. result 1.00 0.96 0.22 asserted 1.00
We release the source code and agents' trajectories to support future research. contribution 0.65 0.90 0.02 asserted 0.98

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-27: 1 request, 6,352 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (12)
Figure 1 of the paper
Figure 1. Two examples of requirement focus misalignment. (a) Unrelated code modification: the agent rewrites a shared dispatcher and omits the existing validate() command. (b) Missing update: the agent adds the output field but omits the code that collects its values. ❷ Missing update. In
Figure 2 of the paper
Figure 2. Overview of RepoTrace in multi-round interactions. Left: an illustrative sequence of re￾quirements for the same code repository. Right: before each code modification, RepoTrace analyzes repository structure, extracts requirement semantics, and constructs a requirement-code correspon￾dence message Mt for the code agent. each file, defined as the number of distinct files that refer to it. PageRank captures the structural im￾portance of files in the dependency graph, while fan-in measures how many files directly depend on each file. We use PageRank to prioritize files included in the…
Figure 3 of the paper
Figure 3. Results of DeepSeek-V4-Pro on EvoCode-Bench using Terminus-2 and Claude Code,
Figure 4 of the paper
Figure 4. Heatmap of the per-round mean reward for each harness-
Figure 5 of the paper
Figure 5. Case study of RepoTrace on JobForge. In the Vanilla setting, adding the critical subcommand breaks the existing schedule dispatch. In the RepoTrace setting, the requirement￾code correspondence guides the agent to add an independent critical branch while preserving the schedule function, satisfying both the current and prior requirements. requirement chain, model, and execution environment. We independently replayed both trajectories and confirmed their outcomes
Figure 6 of the paper
Figure 6. Repository structure and selected requirement entities for JobForge Round 7. Panel (a)
Figure 7 of the paper
Figure 7. Reference prompt for requirement semantics extraction in Section 4.2. The upper box or
Figure 8 of the paper
Figure 8. Excerpts from the requirement-code correspondence message
Figure 9 of the paper
Figure 9. Cropped screenshots of three consecutive records in the supplementary HTML report. The
Figure 10 of the paper
Figure 10. The schedule.go region of the Round 7 HTML diagram. The screenshot shows scheduleCmd and functions for parsing worker settings, computing the schedule, and formatting output. This file corresponds to the added-file entry in
Figure 11 of the paper
Figure 11. The critical.go region of the Round 8 HTML diagram. This file corresponds to the added-file entry in
Figure 12 of the paper
Figure 12. The overall repository structure and function-call graph for Round 7 in the supplementary

Figures are the authors', reproduced from the paper.

References (32)
  1. [1] Anthropic. 2026 agentic coding trends report. Technical report, Anthropic, on Pith
  2. [3] URLhttps://www.anthropic.com/research/ zero-days. Nicholas Carlini, Keane Lucas, and Evyatar Ben Asher contributed equally. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond ´e de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Broc... on PithPith's reviewarXiv
  3. [5] Mandatory workflow for this round [...] - Keep edits inside the target components + impact scope. Do NOT modify unrelated components. - After editing, verify the impacted dependents still build and behave as before (regression check). Source: round_7_message.md; exact text als... on Pith
  4. [6] Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu, and Brad Ken- stler. ... on PithPith's reviewarXiv
  5. [7] 10 Preprint, October 2026 Romina Etezadi, Sallam Abualhaija, Chetan Arora, and Lionel C. Briand. Llm-driven cost-effective requirements change impact analysis.CoRR, abs/2511.00262, on PithPith's reviewarXiv
  6. [8] GLM. GLM-5: from vibe coding to agentic engineering.CoRR, abs/2602.15763, on PithPith's reviewarXiv
  7. [10] Yonghui Huang, Lin Ma, Amjed Tahir, Qian Zhang, Liwen Xiao, and Lysa Xiao. Regression accu- mulation in multi-turn LLM programming conversations.CoRR, abs/2607.01855, on PithPith's reviewarXiv
  8. [11] Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. Llms get lost in multi-turn conversation.CoRR, abs/2505.06120, on PithPith's reviewarXiv
  9. [12] URLhttps://arxiv.org/abs/ 2606.13392. Meir M Lehman. Programs, life cycles, and laws of software evolution.Proceedings of the IEEE, 68(9):1060–1076, on PithPith's reviewarXiv
  10. [14] Zhongxin Liu, Zhonghao Jiang, Zhifan Ye, Haoye Wang, Jiakun Liu, and Xiaoxue Ren. Effective and efficient context retrieval via partial dependency graph for repository-level code generation. CoRR, abs/2608.01927, 2026b. Jane Luo, Xin Zhang, Steven Liu, Jie Wu, Yiming Huang, Ya... on PithPith's reviewarXiv
  11. [15] 11 Preprint, October 2026 Muhammad Maaz, Liam DeV oe, Zac Hatfield-Dodds, and Nicholas Carlini. Finding bugs across the python ecosystem with claude and property-based testing. Technical report, Anthropic, on Pith
  12. [16] URLhttps://www.anthropic.com/research/property-based-testing. Mike A. Merrill, Alexander Glenn Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, Estefany Kelly Buchanan, Junhong Shen, Guang- hao Ye, Haowei Lin, Jason Poulos,... on PithPith's reviewarXiv
  13. [17] URLhttps://kimi.com/code. Accessed: 2026-09-07. OpenAI. Introducing SWE-bench verified.https://openai.com/index/ introducing-swe-bench-verified/, August on Pith
  14. [18] URLhttps://openai.com/codex. Gabriel Orlanski, Devjeet Roy, Alexander Yun, Changho Shin, Alex Gu, Albert Ge, Dyah Adila, Frederic Sala, and Aws Albarghouthi. Slopcodebench: Benchmarking how coding agents degrade over long-horizon iterative tasks.CoRR, abs/2603.24755, on PithPith's reviewarXiv
  15. [19] URLhttps://qwen.ai/blog?id= qwen3.7. Ruchit Rawal, Jeffrey Yang Fan Chiang, Chihao Shen, Jeffery Siyuan Tian, Aastha Mahajan, Tom Goldstein, and Yizheng Chen. Benchmarking correctness and security in multi-turn code genera- tion.CoRR, abs/2510.13859, on PitharXiv
  16. [20] Pratik Shah, Rajat Ghosh, Aryan Singhal, and Debojyoti Dutta. RANGER - repository-level agent for graph-enhanced retrieval.CoRR, abs/2509.25257, on PitharXiv
  17. [21] Haiyang Shen, Xuanzhong Chen, Wendong Xu, Yun Ma, Liang Chen, and Kuan Li. Evocode-bench: Evaluating coding agents in multi-turn iterative interactions.CoRR, abs/2605.24110, on PithPith's reviewarXiv
  18. [22] Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive archi- tectures for language agents.Trans. Mach. Learn. Res., 2024, on Pith
  19. [24] Peiding Wang, Li Zhang, Fang Liu, Lin Shi, Minxiao Li, Bo Shen, and An Fu. Codeif-bench: Eval- uating instruction-following capabilities of large language models in interactive code generation. CoRR, abs/2503.22688, on PitharXiv
  20. [25] 12 Preprint, October 2026 Lu Yan, Xuan Chen, and Xiangyu Zhang. When the specification emerges: Benchmarking faithful- ness loss in long-horizon coding agents.CoRR, abs/2603.17104, on PitharXiv
show 12 more
  1. [27] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin g ¨odel machine: open- ended evolution of self-improving agents. InInternational Conference on Learning Representa- tions, volume 2026, pp. 104223–104294, on Pith
  2. [28] Tailin Zhou. Hierarchical self-improvement: A framework for task-specific evolvable agent har- nesses.CoRR, abs/2608.08466, on PithPith's reviewarXiv
  3. [29] 13 Preprint, October 2026 A SPECIFICEXAMPLES OFREPOTRACE We use JobForge Round 7 to illustrate the three steps in Section 4: repository structure analysis, requirement semantics extraction, and agent context augmentation across rounds. JobForge is a Go command-line tool develo... on Pith
  4. [30] These records distinguish the existingschedule.gofile from the candidate new filecritical.go. A.2 REQUIREMENTSEMANTICSEXTRACTION To identify code elements associated withr 7, the extraction step uses the inputs defined in Sec- tion 4.2: the current requirement, repository stru... on Pith
  5. [32] In Round 6,schedule.goappears as a candidate target. In Round 7,schedule.goappears in the added-file record, andcritical.gobecomes a candidate target. In Round 8,critical.go appears in the added-file record. These entries distinguish the targets of a current requirement from f... on Pith
  6. [1978] Changzhi Liu, Yilun Liu, Sikuan Yan, V olker Tresp, and Yunpu Ma. Mendel g¨odel machine: Recur- sive self-improving coding agents via comparative evolution.CoRR, abs/2608.07645, 2026a. Wei Liu, Ailun Yu, Daoguang Zan, Bo Shen, Wei Zhang, Haiyan Zhao, Zhi Jin, and Qianxiang Wan... on PithPith's reviewarXiv
  7. [1994] doi: 10.1109/ICRE.1994.292398. Fei Gu, Zi Liang, Jiahao Ma, and Hongzong Li. The matthew effect of ai programming assistants: A hidden bias in software evolution. InInternational Conference on Learning Representations, volume 2026, pp. 68445–68470, on Pithdoi:10.1109/icre.1994.292398
  8. [2005] Zexun Zhan, Shuzheng Gao, Ruida Hu, and Cuiyun Gao. Sr-eval: Evaluating llms on code genera- tion under stepwise requirement refinement.CoRR, abs/2509.18808, on PitharXiv
  9. [2021] Mingyu Chen, Yakun Zhang, Zihao Xie, Yixing Luo, Jinrui Xu, Cuiyun Gao, Kaiqi Zhao, and Yunming Ye. Tracedev: A traceability-driven multi-agent framework for requirement-to-code development.CoRR, abs/2607.18886, on PithPith's reviewarXiv
  10. [2024] Ningzhi Tang, Chaoran Chen, Gelei Xu, Yiyu Shi, Yu Huang, Collin McMillan, Tao Dong, and Toby Jia-Jun Li. How coding agents fail their users: A large-scale analysis of developer-agent misalignment in 20,574 real-world sessions.CoRR, abs/2605.29442, on PithPith's reviewarXiv
  11. [2025] DeepSeek-AI. Deepseek-v4: Towards highly efficient million-token context intelligence.CoRR, abs/2606.19348, on PitharXiv
  12. [2026] URL https://resources.anthropic.com/hubfs/2026%20Agentic%20Coding% 20Trends%20Report.pdf. Anthropic. Claude code. Product documentation, on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment