Pith. sign in

REVIEW 3 major objections 4 minor 35 references

OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read OpenCodeReview claims that injecting determinism at three pipeline points—rule-guided dispatch, grounded file review, and independent reflection—makes LLM code-review agents both deeper and cheaper than general-purpose coding agents.

desk verdict A well-engineered review agent with plausible gains, but the paper's determinism thesis is never actually measured — single-run results and a same-group benchmark leave the headline open. read the letter →

arxiv 2608.09290 v2 pith:OVHPONIR submitted 2026-08-10 cs.SE

Zhengfeng Li , Lei Zhang , Xianwei Wu , Zhengqi Zhuang , Yingjie Xu , Boge Wang , Shaofei Zhu , Chuan Wang
show 3 more authors
Peng Zhao Xinyu Zheng Guoping Rong
This is my paper · ORCID
classification cs.SE
keywords LLM-basedcodereviewdeterministicengineeringmulti-agentsystemsReActhallucinationfilteringreflectiontokenefficiencySEM-F1
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that for LLM-based code review, deliberately constraining the agent—rather than giving it maximal freedom—produces better, more trustworthy comments at far lower cost. It presents OpenCodeReview, an open-source review agent with three forced design choices: deterministic file and criterion selection through rules, a curated six-tool exploration loop with bounded outputs, and a reflection stage that filters comments by attempting to falsify them against the diff alone. On the AACR-Bench benchmark, it reports that OpenCodeReview beats Claude Code and Codex across all six tested model backends, reaching 25.10% SEM-F1 with Claude-4.6-Opus versus 11.57% for the same model inside Claude Code, a 2.17× gap, while consuming 5–15× fewer tokens. If true, the result matters because it reframes agent reliability as an engineering property that can be designed in, rather than something that scales with model size or autonomy.

What carries the argument

The load-bearing mechanism is what the paper calls deterministic engineering for uncertain agents: a design stance that deliberately replaces agent discretion with rules at the points where variability hurts. It is carried by three concrete components. Rule-Guided Dispatch removes agent-driven file triage by resolving each changed file to a rule through a four-tier priority chain with first-match-wins glob matching. Grounded File Review replaces free-form shell access with six curated tools whose outputs are bounded (file_read capped at 500 lines, code_search capped at 100 matches with a 10-second timeout, and so on), wrapped in a ReAct loop with a 30-iteration ceiling and context compression triggered at 60% and 80% utilization. Independent Reflection is the counterintuitive core: a separate pass in which the same LLM, seeing only the diff, may delete a comment if the diff contradicts it but cannot add or rewrite comments, so the filter is conservative and recall-preserving. Together these components convert the review pipeline from an open-ended agent problem into a deterministic dispatch-plus-filter problem.

What would settle it

Re-running OpenCodeReview and Claude Code on AACR-Bench with a different semantic judge, or with human reviewers adjudicating matched and unmatched comments, and measuring run-to-run variance, would settle the claim: if a judge swap reverses the ranking, or if the standard deviation across repeated runs of one configuration exceeds the 2.17× gap, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the two weaknesses of current LLM review agents—non-determinism and context locality—share one root cause: nothing constrains agent behavior along axes validated against what reviewers actually need. OpenCodeReview therefore injects determinism at three pipeline points. Rule-Guided Dispatch resolves a four-tier rule chain (built-in, user-global, project-level, and ad-hoc) so the same PR always maps to the same files and review criteria. Grounded File Review runs one parallel SubAgent per file in a ReAct loop with six bounded tools (file_read, file_find, code_search, file_read_diff, code_comment, and task_done), preventing context bloat while still tracing cross-file dependencies. Independent Reflection applies a falsification-first filter under an asymmetric information boundary: the reflector sees only the diff, not the agent's tool-augmented exploration, so it removes only comments the diff directly contradicts. On AACR-Bench, the system achieves the highest SEM-F1 across all six backends, with the top configuration (Claude-4.6-Opus) reaching 25.10% SEM-F1, 33.90% precision, and 20.00% recall, while the same model under Claude Code reaches 11.57% SEM-F1 at 5–15× the token cost.

Load-bearing premise

The whole comparison rests on the assumption that the LLM judge used to decide whether comments match the benchmark's ground truth gives an unbiased ranking, and that the numbers reported for each system are representative despite LLM agents being non-deterministic.

Editorial extensions

If this is right

  • The same model, Claude-4.6-Opus, scores 25.10% SEM-F1 under OpenCodeReview but 11.57% under Claude Code, so the reported gain is attributed to system design rather than model capability.
  • Across all tested backends, OpenCodeReview's lowest SEM-F1 (17.90%) exceeds Claude Code's highest (14.13%), so the ordering is reported to be robust within this benchmark.
  • OpenCodeReview consumes 5–15× fewer tokens than Claude Code while scoring 1.3–2.2× higher SEM-F1, so constrained exploration is claimed to shift the cost-quality frontier rather than trade one for the other.
  • Because the reflector only filters and never generates, the design predicts precision can rise while issue breadth, and therefore recall, is preserved.
  • The asymmetric information boundary is proposed as a transferable principle for other agent tasks where outputs can be checked against a subset of the evidence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not ablate its three modules separately; running each module alone would reveal how much of the 2.17× gap comes from dispatch, from bounded tools, and from reflection.
  • Because the reflector improves precision while seeing strictly less context than the reviewing agent, the same move could transfer to other agent tasks whose outputs can be checked against a small evidence slice, though the paper does not demonstrate that transfer.
  • The rule sets are hand-authored, so the open question the paper leaves is whether automatically mined rules from historical reviews would preserve the reported gains.
  • A judge-swap test—scoring the same outputs with a different LLM judge or with human raters—would clarify whether the benchmark's semantic-equality judgments favor the system's comment style.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes OpenCodeReview, an LLM-based code-review agent whose design injects determinism at three pipeline points: rule-guided dispatch of files and criteria, a grounded ReAct loop with six bounded output tools and per-file subagents, and an independent reflection filter that sees only the diff. The system is evaluated on AACR-Bench (200 PRs, 10 languages, 1,505 expert-verified comments) against Claude Code and Codex under six LLM backends. The central reported result is that OpenCodeReview achieves the highest SEM-F1 in every configuration, with the top configuration (Claude-4.6-Opus) reaching 25.10% SEM-F1 versus 11.57% for the same model under Claude Code, while consuming 5–15x fewer tokens. The paper concludes that structured, deterministic engineering yields deeper, more reliable, and cheaper code review than less constrained agents.

Significance. If the reported effects are real, the paper makes a useful empirical contribution: it articulates a concrete design principle, ships an open-source implementation, evaluates across several LLM backends, and reports token costs transparently. The architecture is clearly described and the individual modules are plausible, and the paper does not claim that rule dispatch makes comment generation itself deterministic. The main empirical claims, however, currently rest on a single run per configuration and on semantic-equivalence judgments produced by a benchmark and judge developed within the same author group. Before the stronger conclusions about consistent outperformance and determinism improving reliability can be accepted, the evaluation needs repeated runs, variance or confidence-interval reporting, significance tests, and independent validation of the judge. Conditional on those fixes, this would be a solid empirical contribution to agent-based code review.

major comments (3)
  1. [§4.2, Table 3; §6] The headline comparison rests on a single execution of each system/backend. Table 3 reports one row per configuration with integer matched/generated counts (e.g., 301/889), and the determinism discussion in §6 covers only the five-run averaging of the LLM judge, not repeated runs of the reviewed systems. Because LLM agent loops are acknowledged in the paper to be non-deterministic, a single run cannot establish the 25.10% vs. 11.57% SEM-F1 gap, the weaker 17.90% vs. 14.13% comparison, or the title's determinism claim; no reproducibility metric (e.g., comment-set overlap, variance of SEM-F1, or distribution of per-PR scores) is reported anywhere. Please add repeated runs for each configuration, report means with bootstrap confidence intervals over the 200 PRs, run a paired significance test for the headline comparisons, and report at least one determinism measure, such as overlap of comments across repeated runs or the standard deviation of SEM-F1.
  2. [§4.1.1, §6, and AACR-Bench [30]] The semantic-equivalence judge (Qwen3-235B-A22B-Instruct) and the benchmark itself come from the same author group, yet no human-validated accuracy or agreement statistics for the judge are reported. If the judge systematically favors the comment style produced by OpenCodeReview, the 2.17x ratio and the 'consistently outperforms' conclusion could be partly an evaluation artifact, even though the system's numbers are measured rather than derived. Please report (i) a human-rated sample of judge match decisions with inter-rater agreement, (ii) results with at least one independent judge, and (iii) per-configuration judge stability across the five runs, not just the mean.
  3. [§4.4 and §3.4] The causal attribution to independent reflection is not tested. Section 4.4 states that OpenCodeReview's favorable precision-recall position is 'a direct consequence of the independent reflection module,' and Section 5 generalizes the asymmetric-information-boundary design principle, but Table 3 contains no ablation: there is no run with reflection disabled, no run without rule-guided dispatch, and no run with the curated tools replaced by a general-purpose tool. Because the system differs from both baselines along several dimensions simultaneously, the reported experiments cannot attribute the SEM-F1 advantage to any one design decision. Please add ablations on at least two backends—minimum reflection on/off, and ideally dispatch and tool-set variants—and report whether recall is preserved as claimed.
minor comments (4)
  1. [§4.2] The text says Claude Code's highest-recall configuration generates 4,580 comments across 200 PRs, but Table 3's Claude-4.6-Opus row reports 435/5980 generated comments; please correct the number or clarify what 4,580 refers to.
  2. [§4.2 and Table 3] The match counts are integers (e.g., 301/889) while the percentages are described as means over five judge runs; please state explicitly how the counts and the averaged percentages are reconciled, or whether the counts come from a single judge run.
  3. [§3.2.3] The file-filtering stage (binary exclusion, include/exclude patterns, extension allowlist, and the 80% context-window filter) determines the reviewed file set; please report how many files and pull requests are affected by each filter in the evaluation.
  4. [§3.3.1] The default ReAct iteration bound and compression thresholds are free parameters; please state explicitly whether identical values were used for all backends and whether the main results are sensitive to these bounds.

Circularity Check

0 steps flagged · score 0.0 of 10

No construction-level circularity; the benchmark/judge author overlap is a validity caveat, not a reduction.

full rationale

OpenCodeReview's headline results are empirical measurements on a fixed, public benchmark. The paper does not fit any parameter to AACR-Bench's ground truth or to the SEM-F1 judge, and no equation or component is defined in terms of the outcome it is claimed to predict. Rule-Guided Dispatch, Grounded File Review, and Independent Reflection are engineering designs, not quantities derived from the evaluation metric. The only self-citation of note is AACR-Bench [30], whose author list overlaps with this paper, and the semantic-equivalence judge is part of that benchmark. Under the stated rules, a cited resource that is public and externally falsifiable counts as independent support: the ground truth is described as 'AI-assisted, expert-verified' with three rounds of cross-validation by over 80 senior engineers, and any external system can be run on the same 200 PRs. The lack of human validation for the LLM judge's equivalence decisions and the absence of repeated system runs (Section 6 only averages the judge five times, not the systems) are real threats to validity and to the determinism thesis, but they are evidentiary weaknesses, not circular derivations. No step reduces to its input by construction, so the circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim depends on hand-set engineering constants and strong evaluation assumptions. The free parameters are ordinary design choices for an agent system, but their values are not justified by sensitivity analysis, and the evaluation assumes the author-built benchmark and LLM judge are unbiased. No new physical or conceptual entities are introduced beyond the software system itself.

free parameters (6)
  • Max ReAct iterations = 30
    Hand-set ceiling for the SubAgent loop; affects how much cross-file evidence can be gathered and how high the token cost can grow (Section 3.3.1).
  • Context compression thresholds = 60% and 80% of context window
    Hand-set triggers for asynchronous and synchronous compression; affect context preservation and cost behavior (Section 3.3.1).
  • Tool output caps = 500 lines file_read, 100 results file_find and code_search, 10s timeout
    Hand-set bounds intended to prevent context bloat; they determine how much evidence a SubAgent can see per tool call (Table 2, Section 3.3.2).
  • File diff size filter = 80% of model context window
    Hand-set threshold that excludes large diffs from review entirely, shaping which files are covered (Section 3.2.3).
  • Judge repetition count = 5
    Hand-set number of LLM judge runs averaged to compute SEM-F1; no variance or significance is reported (Section 6).
  • Built-in rule set = not enumerated in paper
    Hand-authored natural-language rule documents determine file and criterion assignment; their exact content is not specified, so they act as an undeclared parameter of the system (Section 3.2).
assumptions (4)
  • domain assumption LLM judge semantic equivalence is a valid proxy for ground-truth matching.
    SEM-F1 is computed from Qwen3-235B judge decisions, with no human or external validation of the judge's match decisions (Section 4.1.1).
  • domain assumption AACR-Bench ground truth is accurate and unbiased.
    The benchmark reports expert verification by over 80 engineers, but its authors overlap with this paper's authors and no independent audit is cited (Section 4.1.1).
  • domain assumption The six bounded curation tools can recover the cross-file context needed for deep review.
    The tool set with capped outputs is assumed sufficient for tracing callers, callees, and dependencies; no information-completeness analysis is given (Section 3.3.2).
  • domain assumption Diff-only reflection removes hallucinations without suppressing true positives.
    The falsification-first filter is assumed conservative and recall-preserving, but no ablation or counterfactual analysis demonstrates this (Sections 3.4 and 4.4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review." pith.science (2026). https://pith.science/paper/OVHPONIR

@misc{pith2026260809290,
  author       = {Pith},
  title        = {Pith review of: OpenCodeReview: Determinism over Non-Determinism for Cost-Effective Agent-Based Code Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OVHPONIR}},
  note         = {Machine review of arXiv:2608.09290}
}
read the original abstract

LLM-based code review agents promise scalable, always-on review, yet current systems suffer from two intertwined weaknesses: (1) non-determinism--unbounded tool use makes review outcomes unstable, and (2) context locality--the reviewer's access remains bounded to the diff, capping discoverable issue depth. Both give rise to three challenges: misaligned context retrieval, a coherence-efficiency trade-off in multi-file pull requests, and hallucinated comments that erode trust. To address these, we introduce OpenCodeReview, built on deterministic engineering for uncertain agents: rather than granting maximal freedom, we inject determinism at three deliberate pipeline points. Rule-Guided Dispatch uses a multi-layer rule system to deterministically select files and review criteria, eliminating variability of agent-driven triage. Grounded File Review replaces free-form exploration with a curated tool set exposed through a ReAct loop, while file-level parallel SubAgents balance context coherence against efficiency and recover cross-file dependencies on demand. Independent Reflection introduces a falsification-first filter under an asymmetric information boundary--the reflector sees only the diff, not the agent's tool-augmented exploration--removing hallucinated comments without self-reinforcing bias, improving precision while preserving recall. On AACR-Bench (200 real-world PRs, 10 languages, 1,505 expert-verified comments), OpenCodeReview outperforms mainstream coding agents (e.g., Claude Code and Codex) across six LLM backends, achieving up to 2.17x higher SEM-F1 (25.10% vs. 11.57%) while consuming 5-15x fewer tokens. We open-source OpenCodeReview at https://github.com/alibaba/open-code-review.

Figures

Figures reproduced from arXiv: 2608.09290 by the authors.

Figure 1
Figure 1. Banner of OpenCodeReview. Abstract LLM-based code review agents promise scalable, always-on review, yet current systems suffer from two intertwined weaknesses: (1) non-determinism—unbounded tool use and sprawling action spaces make review outcomes unstable across runs, and (2) context local￾ity—the reviewer’s effective access remains bounded to the diff itself, capping the depth of discoverable issues. Both weakness… view at source ↗
Figure 2
Figure 2. Overview of OpenCodeReview. The pipeline proceeds in three stages. (1) Rule-Guided Dispatch: given a PR, the system resolves applicable rules from a four-tier chain (built-in, user-global, project-level, and ad-hoc) for each changed file, filters files by extension, user include/exclude patterns, and size, and dispatches one SubAgent per file in parallel. (2) Grounded File Review: each SubAgent executes a ReAct loop… view at source ↗
Figure 3
Figure 3. Precision-recall trade-off across all 12 configura [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 1 canonical work pages

  1. [1]

    Alberto Bacchelli and Christian Bird. 2013. Expectations, Outcomes, and Chal- lenges of Modern Code Review. InProceedings of the 35th International Conference on Software Engineering (ICSE). 712–721

    2013

  2. [2]

    Kowshik Chowdhury, Dipayan Banik, K M Ferdous, and Shazibul Islam Shamim

  3. [3]

    Xueying Du, Han Wu, Xiaobing Sun, Jingzhi Gong, Yu Zhang, and Xin Ye. 2022. AUGER: Automatically Generating Review Comments with Pre-trained Models. InProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). 1442– 1454

    2022

  4. [4]

    Zhang, Pinjia He, and Ahmed E

    Zhiyu Fan, Kirill Vasilevski, Dayi Lin, Boyuan Chen, Yihao Chen, Zhiqing Zhong, Jie M. Zhang, Pinjia He, and Ahmed E. Hassan. 2025. SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints.arXiv preprint arXiv:2509.09853(2025)

    arXiv 2025

  5. [5]

    Jinyao Guo, Chengpeng Wang, Xiangzhe Xu, Zian Su, and Xiangyu Zhang. 2025. RepoAudit: An Autonomous LLM-Agent for Repository-Level Code Auditing. arXiv preprint arXiv:2501.18160(2025)

    arXiv 2025

  6. [6]

    Asif Haider, Ayesha Binte Mostofa, Sk

    Md. Asif Haider, Ayesha Binte Mostofa, Sk. Sabit Bin Mosaddek, Anindya Iqbal, and Toufique Ahmed. 2024. Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation.arXiv preprint arXiv:2411.10129(2024)

    arXiv 2024

  7. [7]

    Hyunsun Hong and Jongmoon Baik. 2025. Retrieval-Augmented Code Review Comment Generation.arXiv preprint arXiv:2506.11591(2025)

    arXiv 2025

  8. [8]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Schürmannand Josef

Show all 35 references
  1. [9]

    Li, Emmanuel Candès, and Jure Leskovec

    Kexin Huang, Ying Jin, Ryan Li, Michael Y. Li, Emmanuel Candès, and Jure Leskovec. 2025. Automated Hypothesis Validation with Agentic Sequential Falsifications.arXiv preprint arXiv:2502.09858(2025)

    2025 arXiv

  2. [10]

    Osman İçöz and Asaf Biricik. 2026. Context-Aware Code Review Automation: A Retrieval-Augmented Approach.Applied Sciences16, 4 (2026), 1875

    2026

  3. [11]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation.Comput. Surveys55, 12 (2023), 1–38

    2023

  4. [12]

    Fang Liu, Ge Li, Yusi Fu, Fei Jin, Wenlin Liu, Xiangxin Tan, and Xin Xia. 2022. Automating Code Review Activities by Large-Scale Pre-training. InProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering ...

    2022

  5. [13]

    Junyi Lu, Lei Yu, Xiaojia Li, Li Yang, and Chun Zuo. 2023. LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning. InProceedings of the 34th IEEE International Symposium on Software Reliability Engineering (ISSRE)

    2023

  6. [14]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Shayne Longpre, Stephen Pulman, Suhas Patil, Sharan Narang, Gagan Bansa...

    2023

  7. [15]

    Phattharawin Namprasert, Chanathip Thongtan, Pongsakorn Tantikitti, Vishnu Tudupala, Patanamon Thongtanunam, Alaknantha Surendra, Hao Li, and Narayan Rangaraj. 2024. Fine-Tuning Large Language Models to Improve Accu- racy and Consistency of Code Review Comment Generation.ACM T...

    2024 doi

  8. [16]

    Sai Sharanarthi Polineni et al. 2025. Multi-Agent LLM Collaboration for Adaptive Code Review, Debugging, and Security Analysis. In2025 International Conference on Mechatronics, Robotics and Artificial Intelligence (ICMRAI)

    2025

  9. [17]

    Chen Qian, Xin Cong, Wei Liu, Cheng Yang, Weize Chen, Yusheng Su, Yufan Dang, Jiahao Li, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative Agents for Software Development. InProceedings of the 41st International Conference on Machine Learning (ICML)

    2024

  10. [18]

    Rigby and Margaret-Anne Storey

    Peter C. Rigby and Margaret-Anne Storey. 2013. Understanding Broadcast Based Peer Review on Open Source Software Projects.Empirical Software Engineering 18, 2 (2013), 225–261

    2013

  11. [19]

    Caitlin Sadowski, Emma Söderberg, Luke Church, Michal Sipko, and Alberto Bacchelli. 2018. Modern Code Review: A Case Study at Google. InProceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 181–190

    2018

  12. [20]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. InAdvances in Neural Information Processing Systems (NeurIPS). OpenCodeReview: Determinism over Non-Determinism for Cost-Effe...

    2023

  13. [21]

    Kexin Sun, Yucong Guan, Jiaqi Sun, Hongyu Kuang, Guoping Rong, Dong Shao, He Zhang, Xiaoxing Ma, and Christoph Treude. 2026. Improving LLM-Based Go Code Review through Issue-List Generation and Context Augmentation.arXiv preprint arXiv:2606.01859(2026)

    2026 arXiv

  14. [22]

    Bissyandé

    Xunzhu Tang, Kisub Kim, Yewei Song, Cedric Lothritz, Bei Li, Saad Ezzini, Haoye Tian, Jacques Klein, and Tegawendé F. Bissyandé. 2024. CodeAgent: Autonomous Communicative Agents for Code Review. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Proc...

    2024

  15. [23]

    Ningning Wang, Xavier Hu, Pai Liu, He Zhu, Yue Hou, Heyuan Huang, Shengyu Zhang, Jian Yang, Jiaheng Liu, Ge Zhang, Changwang Zhang, Jun Wang, Yuchen Eleanor Jiang, and Wangchunshu Zhou. 2025. Efficient Agents: Building Effective Agents While Reducing Cost.arXiv preprint arXiv:...

    2025 arXiv

  16. [24]

    Abel Yagubyan. 2026. How Consistent Are LLM Agents? Measuring Behav- ioral Reproducibility in Multi-Step Tool-Calling Pipelines.arXiv preprint arXiv:2605.28840(2026)

    2026 arXiv

  17. [25]

    Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2025. SWE-agent: Agent-Computer In- terfaces Enable Automated Software Engineering. InProceedings of the 47th International Conference on Software Engineering (ICSE)

    2025

  18. [26]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Mod- els. InProceedings of the 11th International Conference on Learning Representations (ICLR)

    2023

  19. [27]

    Yongda Yu, Guoping Rong, Haifeng Shen, He Zhang, Dong Shao, Min Wang, Zhao Wei, Yong Xu, and Juhong Wang. 2025. Fine-Tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review. ACM Transactions on Software Engineering Methodology34, 1 (2025...

    2025

  20. [28]

    Yongda Yu, Guohao Shi, Xianwei Wu, Haochuan He, XueMing Gu, Qianqian Zhao, Kui Liu, Qiushi Wang, Zhao Tian, Haifeng Shen, and Guoping Rong

  21. [29]

    Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. 2023. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. InProceedings of the 2023 Conference on Empirical Methods in Natural La...

    2023

  22. [30]

    Lei Zhang, Yongda Yu, Minghui Yu, Xinxin Guo, Zhengqi Zhuang, Guoping Rong, Dong Shao, Haifeng Shen, Hongyu Kuang, Zhengfeng Li, Boge Wang, Guoan Zhang, Bangyu Xiang, and Xiaobin Xu. 2026. AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Context.arXi...

    2026

  23. [31]

    Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. AutoCodeRover: Autonomous Program Improvement. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)

    2024

  24. [32]

    Yuxin Zhang, Yuxia Zhang, Zeyu Sun, Yanjie Jiang, and Hui Liu. 2025. LAURA: En- hancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM. InProceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE)

    2025

  25. [2024]

    InProceedings of the 12th International Conference on Learning Representations (ICLR)

    MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework. InProceedings of the 12th International Conference on Learning Representations (ICLR)

  26. [2025]

    Fine-Tuning LLMs to Analyze Multiple Dimensions of Code Review: A Maximum Entropy Regulated Long Chain-of-Thought Approach.arXiv preprint arXiv:2509.21170(2025)

    2025

  27. [2026]

    InProceedings of the 23rd International Conference on Mining Software Repositories (MSR)

    From Industry Claims to Empirical Reality: An Empirical Study of Code Review Agents in Pull Requests. InProceedings of the 23rd International Conference on Mining Software Repositories (MSR)

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.