Pith. sign in

REVIEW 4 major objections 6 minor 22 references

Coding-agent strength is limited less by model size than by missing executable environments, verifiable rewards, and high-value trajectories; an end-to-end post-training stack closes that gap.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 13:34 UTC pith:454KHC6B

load-bearing objection Solid industrial systems report: real environment/trajectory infrastructure and strong external numbers, with one real soft spot in single-harness ranking claims. the 4 major comments →

arxiv 2607.05471 v1 pith:454KHC6B submitted 2026-07-06 cs.SE cs.AI

KAT-Coder-V2.5 Technical Report

Bo Huang , Fengxiang Li , Hao Xu , Haoyang Huang , Hongyi Fu , Jinhua Hao , Kun Yuan , Minglei Zhang
show 45 more authors
Pengcheng Xu Shiyang Liu Wenhao Zhuang Yuze Shi Zongxian Feng Chao Wang Cheng He Chongling Rao Deyu Cao Fan Yang Gang Xiong Haochen Liu Jiabao Li Jian Liang Jinghui Jia Jingwen Chang Jun Du Junyu Shi Min Li Mingqi Wu Qiang Gao Shangpeng Yan Shaotong Qi Shu Xu Shuo Zhou Tiankuo Xu Tong Zheng Weilun Zhao Xiancheng Meng Xianda Sun Xiaoyu Jiang Xunhao Jia Yao Xia Yimeng Xu Yinghan Cui Yingpeng Chen Yiwen Ning Yong Wang Yuxuan Sun Zhongsheng Liu Ming Sun Cheng Luo Chen Yang Han Li Kun Gai
This is my paper
classification cs.SE cs.AI
keywords coding agentssoftware engineeringagentic reinforcement learningexecutable environmentstool useprocess-aware trajectorieson-policy distillationharness randomization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This report argues that autonomous coding agents are held back mainly by infrastructure: real repositories are hard to rebuild as reproducible, testable sandboxes; issue text is noisy; and long-horizon reinforcement learning is unstable under sparse, corrupted rewards and fixed tool harnesses. The authors answer with a full post-training pipeline. AutoBuilder turns multilingual repositories into sandboxed environments with fail-to-pass and pass-to-pass tests, regenerates clear task specs from patches, and recovers near-miss rollouts while filtering for process quality rather than final pass alone. KwaiClawEnv synthesizes large tool-use trajectories from executable services and real task seeds. Reinforcement learning then uses harness randomization, a reliability-hardened sandbox, asymmetric actor-critic PPO with hindsight value estimation, and multi-layer rewards that credit partial progress and engineering discipline. Domain experts are fused by Multi-Teacher On-Policy Distillation so one student keeps software-engineering, tool-use, and related skills without a severe see-saw. On six benchmarks the resulting model leads PinchBench tool use and ranks second only to a frontier general model on repository-level software engineering. A sympathetic reader cares because the claim reframes progress: better agents may come from better environments and trajectories, not only larger base models.

Core claim

The paper's central claim is that agentic coding capability is bottlenecked less by model scale than by scarcity of reproducible executable environments, verifiable rewards, and high-value trajectories, and that an end-to-end post-training framework—AutoBuilder environments and process-aware trajectories, KwaiClawEnv tool-use data, harness-randomized asymmetric PPO with hardened sandboxes and harness-oriented rewards, plus Multi-Teacher On-Policy Distillation—can produce a unified coding agent that is best-in-class on agentic tool use and near-frontier on repository-level software engineering.

What carries the argument

The end-to-end agentic post-training framework: AutoBuilder for multilingual fail-to-pass/pass-to-pass environments and process-aware trajectory recovery; KwaiClawEnv for scalable tool-use trajectories; harness randomization with reliability-hardened sandboxes; asymmetric actor-critic PPO with hindsight-augmented value estimation and harness-oriented rewards; Multi-Teacher On-Policy Distillation to fuse domain experts into one student.

Load-bearing premise

That scoring every model under one fixed evaluation harness cleanly isolates intrinsic model skill rather than how well each model matches that harness's interface, context layout, and control flow.

What would settle it

Re-run the same six-benchmark suite with multiple harnesses (or with each model under its preferred production harness) and check whether the PinchBench lead and the second-place SWE ranking versus the frontier peer reverse or shrink once harness match is equalized.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Building larger pools of reproducible, fail-to-pass/pass-to-pass repository environments should raise coding-agent pass rates more efficiently than scale alone.
  • Process-aware filtering and near-miss recovery can turn many failed rollouts into useful training signal instead of discarding them.
  • Randomizing tool protocols and control flow during RL should improve cross-harness generalization in real deployments.
  • Hardening sandbox feedback (cutting environment-induced reward errors) is a first-class lever for stable long-horizon agent RL.
  • On-policy multi-teacher distillation can merge specialized SWE and tool-use experts into one model without the severe see-saw of weight averaging.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If environment and trajectory quality dominate, open efforts that publish verifiable multi-language repo sandboxes may close capability gaps faster than closed model-size races.
  • The same process-score and near-miss recovery loop could transfer to other long-horizon domains where final success is sparse (e.g., multi-step data analysis or ops automation).
  • Harness randomization during training suggests evaluation leaderboards should report multi-harness averages to avoid ranking models by scaffold fit.
  • Cutting sandbox reward corruption from ~16% to under 2% implies many past agent-RL plateaus may have been infrastructure noise rather than algorithmic limits.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. KAT-Coder-V2.5 is presented as a coding-focused agentic model whose capability is limited less by scale than by scarce reproducible environments, verifiable rewards, and high-value trajectories. The authors address this with AutoBuilder (multilingual sandboxed SWE environments with F2P/P2P verification; construction success 16.5%→57.2%, 100K+ envs), process-aware trajectory recovery/filtering, KwaiClawEnv for tool-use synthesis, harness-randomized RL with a reliability-hardened sandbox (feedback error ~16%→<2%), asymmetric actor–critic PPO with hindsight-augmented value estimation, a harness-oriented rule+GRM reward stack, and Multi-Teacher On-Policy Distillation (MOPD) to fuse domain experts. Under a unified Claude Code harness, the model reports best PinchBench tool-use (94.9) and second place behind Opus 4.8 on SWE-Bench Pro (65.2) and KAT Code Bench (53.1), with competitive but weaker Terminal-Bench 2.1 and SciCode results.

Significance. If the infrastructure thesis holds, the work is a substantial systems contribution to agentic software engineering: it operationalizes verifiable environment construction at scale, process-aware trajectory quality beyond final pass rates, sandbox reliability as a first-class RL concern, and multi-expert fusion via on-policy reverse-KL distillation with drift-aware truncation. Concrete diagnostics (sandbox error rates, environment success rates, reward curve stability in Fig. 5) and external benchmarks (SWE-Bench Pro, PinchBench, Terminal-Bench 2.1, SciCode) give the claims more empirical grounding than typical closed technical reports. The open service link and internal KAT Code/Claw benches further support reproducibility of evaluation design, even if training data and code are not fully released. The main significance is methodological: treating harness, sandbox, and trajectory quality as the bottleneck rather than parameter count alone.

major comments (4)
  1. [§6 Evaluation / §4.1 Harness Scaling] §6 and Table 4 evaluate all models under a single fixed Claude Code harness so that gaps are “attributable to model capability rather than to scaffold.” §4.1 simultaneously argues that fixed-harness training causes format, context-structure, and control-flow overfitting, and that black-box harnesses including Claude Code are the deployment distribution deliberately used in RL. The decisive ranking (best PinchBench; second on SWE-Bench Pro / KAT Code Bench) is therefore reported only under a harness the system was trained to match. Without multi-harness numbers (e.g., mini-swe-agent, OpenHands, Codex CLI) for KAT and peers, the ranking can partly reflect differential Claude Code fit rather than pure evidence for the post-training stack. This is load-bearing for the central claim; please add multi-harness evaluation or a controlled harness-swap study.
  2. [§2–§5 / Table 4] The paper attributes gains to AutoBuilder, process filtering, harness randomization, asymmetric hindsight PPO, the rule+GRM reward framework, and MOPD, but provides almost no component ablations. Fig. 5 shows a stable SWE reward curve and §4.2.2 quantifies sandbox error reduction, yet Table 4 cannot isolate which pieces move SWE-Bench Pro or PinchBench. At minimum, ablate (i) process-aware filtering vs final-pass-only, (ii) white-box-only vs white+black harness RL, (iii) standard PPO critic vs hindsight-augmented critic, and (iv) sequential multi-expert SFT/merge vs MOPD on the same student. Without these, the infrastructure-bottleneck thesis remains under-supported relative to the strength of the claim.
  3. [Table 4 / §6.2] Table 4 reports single point estimates with no error bars, multi-seed variance, or run counts. For agentic SWE and long-horizon tool use, trajectory stochasticity and environment flakiness are material; a 65.2 vs 62.1 (GLM-5.2) or 94.9 vs 93.5 (Opus) gap may not be robust. Please report mean±std over seeds or independent evaluation runs, and clarify decoding temperature, max turns, and number of attempts per task under the fixed Claude Code protocol.
  4. [§6.1 Benchmarks] Internal KAT Code Bench and KAT Claw Bench are valuable for authenticity, but §6.1 describes curation from Kuaishou internal workloads without a public contamination audit relative to AutoBuilder/KwaiClawEnv training sources. Mild circularity risk remains if internal tasks overlap training distributions while external suites do not fully. Please state decontamination procedures (repo/commit/task overlap checks) and, if possible, release task IDs or a public subset so the second-place SWE claim can be independently stress-tested.
minor comments (6)
  1. [§4.3] Eqs. (1)–(4) define PPO/GAE and the asymmetric critic, but clip ε, γ, λ, and the hindsight context schema for c_t are not numerically specified; a short hyperparameter table would aid reproduction.
  2. [§4.4 / Eq. (5)] Eq. (5) for GRM reward uses λ as a false-positive penalty without a reported value or sensitivity study; Table 2/3 rubrics are partial—state how many criteria enter the live reward and their relative weights vs rule-based tiers in Table 1.
  3. [§5] Eqs. (6)–(8) introduce MOPD with top-k overlap ρ_t and truncation after m low-ρ tokens; k, m, and the monotonic map from ρ_t to w_t are unspecified. Also clarify whether all five experts or only SWE/Claw/WebCoding teachers are used in the reported student.
  4. [Figure 1 / Table 4] Figure 1 and Table 4: PinchBench notes Kimi as K2.6 in the figure but K2.7-Code with * in the table caption—align naming and retrieval date consistently.
  5. [§2.1] §2.1: “>90% tests collected” acceptance threshold is central to AutoBuilder quality; discuss false-accept rate when tests are collected but not the intended suite, and whether multilingual frameworks share the same parser reliability.
  6. [§3 / References] Minor typos/consistency: “T ask Layer” spacing in §3.2.2 headings; “see-saw” vs “seesaw”; arXiv IDs in references appear future-dated relative to typical citation practice—verify metadata.

Circularity Check

0 steps flagged

No derivation circularity: empirical systems report whose rankings rest on external suites, not self-defined predictions.

full rationale

KAT-Coder-V2.5 is an empirical post-training / systems paper. Its load-bearing claims are (i) that agentic coding is bottlenecked by environments, verifiable rewards, and trajectories, and (ii) that AutoBuilder, KwaiClawEnv, harness-randomized asymmetric PPO, and Multi-Teacher On-Policy Distillation yield strong scores on named benchmarks. There is no algebraic derivation chain in which a fitted constant or self-defined quantity is relabeled as a first-principles prediction. PPO/GAE objectives (Eqs. 1–4), the reverse-KL MOPD loss (Eq. 6), and the GRM reward (Eq. 5) are standard training objectives, not closed-form predictions forced by construction from the evaluation metrics. Self-citations to prior KAT-Coder reports describe infrastructure reuse and do not substitute for the external results (SWE-Bench Pro, PinchBench, Terminal-Bench 2.1, SciCode). Internal KAT Code/Claw benches raise evaluation-contamination risk, and single-harness Claude Code evaluation raises fairness risk, but those are validity concerns, not circular reductions of claimed predictions to their inputs. Per the analyzer rules, that yields score 0 with no circular steps.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 4 invented entities

As an engineering technical report, the load-bearing content is mostly domain assumptions about verifiable SWE tasks and RL stability, plus many operational thresholds and reward weights chosen for training. There is no first-principles derivation; free parameters are training/design knobs, and invented entities are named systems rather than physical objects.

free parameters (5)
  • AutoBuilder test-collection acceptance threshold = >90%
    Environments are accepted only when more than 90% of expected tests are collected and outcomes are reproducible; this cutoff is a design choice that defines the training distribution.
  • PPO clip epsilon, discount gamma, GAE lambda
    Standard PPO/GAE hyperparameters control the policy update and advantage estimation; values are not fully specified but the central RL claims depend on them.
  • Rule-based reward tier weights and tool-parallelism threshold
    Core task score, behavior penalties, and failed-trajectory incentives are weighted by hand-designed rules (Table 1); relative weights shape the optimized policy.
  • GRM false-positive penalty lambda and rubric criteria set
    Model-based process rewards depend on a human-derived rubric and the lambda penalty in the GRM reward (Eq. 5), both chosen from failure analysis rather than derived.
  • Drift-aware top-k overlap threshold and consecutive-token truncation m
    MOPD stability uses top-k teacher/student overlap rho_t and truncates after m low-compatibility tokens; these knobs gate which tokens train the student.
axioms (5)
  • domain assumption A patch is correct iff it passes the task’s fail-to-pass and pass-to-pass validation tests in the reconstructed environment.
    Stated in §2.1 as the definition of a verifiable SWE task; all environment scaling and rewards rest on this operational correctness criterion.
  • domain assumption Regenerated structured task descriptions from golden/test patches are more reliable training targets than raw issue/PR text.
    Core of verifiable task mining in §2.1; if regenerated specs still misalign with real user intent, process-filtered trajectories optimize the wrong objective.
  • domain assumption PPO with GAE and an asymmetric hindsight critic is a valid long-horizon credit-assignment method when the actor never sees hindsight at deployment.
    §4.3 adopts asymmetric actor–critic paradigms; soundness of reported RL gains assumes this training-time information asymmetry does not induce untransferable value artifacts.
  • ad hoc to paper Reverse KL multi-teacher on-policy distillation with drift-aware truncation fuses experts without severe see-saw degradation.
    §5 presents MOPD as solving multi-expert interference; the paper offers limited quantitative fusion ablations relative to the strength of this claim.
  • standard math Standard clipped PPO objective and GAE recursions (Eqs. 1–2) are mathematically well-posed for token-level agent trajectories.
    Uses textbook PPO/GAE forms from Schulman et al.; not re-proved here.
invented entities (4)
  • AutoBuilder no independent evidence
    purpose: Agent-driven multilingual reconstruction of repositories into sandboxed, test-verified SWE environments at scale.
    Named pipeline introduced as the environment-scaling engine; evidence is internal success-rate lift (16.5%→57.2%) and environment count, not an external independent artifact.
  • KwaiClawEnv no independent evidence
    purpose: Three-layer Service/Task/Eval synthesis of large-scale multi-tool agent trajectories from skills and real task seeds.
    Framework introduced for general agentic data; validation is internal filtering and scale claims.
  • Multi-Teacher On-Policy Distillation (MOPD) no independent evidence
    purpose: Fuse SWE, Claw, terminal, web, and knowledge experts via on-policy reverse-KL with cold start and drift-aware truncation.
    Named method combining known distillation ideas with paper-specific stabilization; no external formal verification or third-party reimplementation cited.
  • Harness-oriented reward framework + GRM judge no independent evidence
    purpose: Dense rule- and rubric-based rewards for long-horizon coding agents, including failed-trajectory incentives.
    Custom reward stack and specialized judge model trained on annotated trajectories; criteria come from authors’ failure analysis.

pith-pipeline@v1.1.0-grok45 · 22412 in / 4041 out tokens · 41535 ms · 2026-07-11T13:34:25.506212+00:00 · methodology

0 comments
read the original abstract

We present KAT-Coder-V2.5, a coding-focused agentic model trained to act autonomously inside real, executable repositories rather than as a single-turn code generator. Its capability is bottlenecked less by model scale than by the scarcity of reproducible environments, verifiable rewards, and high-value trajectories, which we address with an end-to-end agentic post-training framework. AutoBuilder reconstructs multilingual repositories into sandboxed environments with fail-to-pass and pass-to-pass verification at scale, from which we regenerate self-contained task specifications, recover near-miss trajectories, and distill supervision through process-aware filtering, while KwaiClawEnv synthesizes large-scale tool-use trajectories from executable services and real task seeds. We further scale reinforcement learning with harness randomization, a reliability-hardened sandbox, an asymmetric actor--critic PPO with hindsight-augmented value estimation, and a harness-oriented reward framework, and unify SWE, Agent-Claw, and WebCoding experts via Multi-Teacher On-Policy Distillation. Across six software-engineering and agentic benchmarks, KAT-Coder-V2.5 delivers the best agentic tool-use result on PinchBench and ranks second only to the frontier Opus 4.8 on repository-level software engineering. Our service is available at https://streamlake.com/product/kat-coder.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 13 linked inside Pith

  1. [1]

    Kat-coder-v2 technical report

    Fengxiang Li, Han Zhang, Haoyang Huang, Jinghui Wang, Jinhua Hao, Kun Yuan, Meng- tong Li, Minglei Zhang, Pengcheng Xu, Wenhao Zhuang, Yizhen Shao, Zongxian Feng, Can Tang, Chao Wang, Chengxiao Tong, Fan Yang, Gang Xiong, Haixuan Gao, Han Gao, Hao Wang, Haochen Liu, Hongliang Sun, Jiabao Li, Jingwen Chang, Jun Du, Junyi Peng, Leizhen Cui, Meimei Jing, Min...

    arXiv 2026

  2. [2]

    SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024

    2024

  3. [3]

    Swe-bench pro: Can ai agents solve long-horizon software engineering tasks? arXiv preprint arXiv:2509.16941, 2025

    Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, et al. Swe-bench pro: Can ai agents solve long-horizon software engineering tasks? arXiv preprint arXiv:2509.16941, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Swe-smith: Scaling data for software engineering agents

    John Yang, Kilian Lieret, Carlos E Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. Swe-smith: Scaling data for software engineering agents. arXiv preprint arXiv:2504.21798, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Swe- rebench v2: Language-agnostic swe task collection at scale.arXiv preprint arXiv:2602.23866, 2026

    Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, and Alexander Golubev. Swe- rebench v2: Language-agnostic swe task collection at scale.arXiv preprint arXiv:2602.23866, 2026

    Pith/arXiv arXiv 2026

  6. [6]

    Docarena: Turning raw documents into control- lable training environments for document search agents

    Jiamian Wang, Ruiyi Zhang, Tong Yu, et al. Docarena: Turning raw documents into control- lable training environments for document search agents. arXiv preprint arXiv:2606.26122, 2026

    Pith/arXiv arXiv 2026

  7. [7]

    Openclaw-rl: Train any agent simply by talking

    Yinjie Wang, Xuyang Chen, Xiaolong Jin, et al. Openclaw-rl: Train any agent simply by talking. arXiv preprint arXiv:2603.10165, 2026

    Pith/arXiv arXiv 2026

  8. [8]

    Skill-as-pseudocode: Refactoring skill libraries to pseudocode for llm agents

    Xinze Li, Yuhang Zang, Yixin Cao, et al. Skill-as-pseudocode: Refactoring skill libraries to pseudocode for llm agents. arXiv preprint arXiv:2605.27955, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    Clawenvkit: Automatic environment generation for claw-like agents

    Xirui Li, Ming Li, Ion Stoica, et al. Clawenvkit: Automatic environment generation for claw-like agents. arXiv preprint arXiv:2604.18543, 2026

    Pith/arXiv arXiv 2026

  10. [10]

    Clawgym: A scalable framework for building effective claw agents

    Fei Bai, Huatong Song, Shuang Sun, Daixuan Cheng, Yike Yang, Chuan Hao, Renyuan Li, Feng Chang, Yuan Wei, Ran Tao, et al. Clawgym: A scalable framework for building effective claw agents. arXiv preprint arXiv:2604.26904, 2026

    Pith/arXiv arXiv 2026

  11. [11]

    Large-scale terminal agentic trajectory generation from dockerized environments

    Siwei Wu, Yizhi Li, Yuyang Song, et al. Large-scale terminal agentic trajectory generation from dockerized environments. arXiv preprint arXiv:2602.01244, 2026

    arXiv 2026

  12. [12]

    Claw-eval: Towards trustworthy evaluation of autonomous agents

    Bowen Ye, Rang Li, Qibin Yang, et al. Claw-eval: Towards trustworthy evaluation of autonomous agents. arXiv preprint arXiv:2604.06132, 2026

    Pith/arXiv arXiv 2026

  13. [13]

    Pinchbench: Openclaw benchmark.https://pinchbench.com/, 2026

    PinchBench. Pinchbench: Openclaw benchmark.https://pinchbench.com/, 2026

    2026

  14. [14]

    Kat-coder technical report

    Zizheng Zhan, Ken Deng, Jinghui Wang, Xiaojiang Zhang, Huaixi Tang, Minglei Zhang, Zhiyi Lai, Haoyang Huang, Wen Xiang, Kun Wu, et al. Kat-coder technical report. arXiv preprint arXiv:2510.18779, 2025. 21

    arXiv 2025

  15. [15]

    Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Group-in-group policy optimization for llm agent training

    Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for llm agent training. Advances in Neural Information Processing Systems, 38:46375– 46408, 2026

    2026

  17. [17]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  18. [18]

    Asymmetric proximal policy optimization: mini-critics boost llm reasoning

    Jiashun Liu, Johan Obando-Ceron, Han Lu, Yancheng He, Weixun Wang, Wenbo Su, Bo Zheng, Pablo Samuel Castro, Aaron Courville, and Ling Pan. Asymmetric proximal policy optimization: mini-critics boost llm reasoning. arXiv preprint arXiv:2510.01656, 2025

    arXiv 2025

  19. [19]

    Asymmetric actor critic for image-based robot learning

    Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, and Pieter Abbeel. Asymmetric actor critic for image-based robot learning. arXiv preprint arXiv:1710.06542, 2017

    Pith/arXiv arXiv 2017

  20. [20]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Ha- jishirzi, and Ali Farhadi. Editing models with task arithmetic. In International Conference on Learning Representations (ICLR), 2023

    2023

  21. [21]

    On-policy distillation of language models: Learning from self-generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In International Conference on Learning Representations (ICLR), 2024

    2024

  22. [22]

    function

    Zhicheng Yang, Zhijiang Guo, Yifan Song, Minrui Xu, Yongxin Wang, Yiwei Wang, Xiaodan Liang, and Jing Tang. Prune-opd: Efficient and reliable on-policy distillation for long- horizon reasoning. arXiv preprint arXiv:2605.07804, 2026. 22 Table 1|Rule-based reward components. The rewards are organized into a primary task objective, standard behavioral constr...

    Pith/arXiv arXiv 2026