EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents

Bingxuan Li; Dongqi Fu; Hanghang Tong; Hanqing Zeng; Hong Li; Jiayi Liu; Jingrui He; Qifan Wang; Tianxin Wei; Xiangjun Fan

REVIEW 5 major objections 5 minor 18 references

EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training an agent to decide when to read and write external state lifts ALFWorld success from 47.9% to 96.9%.

desk verdict A strong empirical system paper whose headline gain is not yet cleanly attributed to learned harness coordination; worth refereeing with requests for a trained-policy ablation and matched baselines. read the letter →

arxiv 2608.05446 v1 pith:PYMPGEDG submitted 2026-08-05 cs.LG cs.CL

Xuying Ning , Dongqi Fu , Tianxin Wei , Hanqing Zeng , Yuanchen Bei , Bingxuan Li , Zihao Li , Qifan Wang

show 8 more authors

Xiang Shen Yifan Wu Jiayi Liu Hong Li Yinglong Xia Xiangjun Fan Hanghang Tong Jingrui He

This is my paper · ORCID

classification cs.LGcs.CL

keywords LLMagentslong-horizontasksharnesspolicylearningexternalstateBeliefProgressExperienceGRPOreinforcementALFWorld

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EvoHarness-RL claims that the missing ingredient in long-horizon LLM agents is not more tools or bigger memories, but a trainable policy that decides when to form and consult external state. The paper exposes that state as Belief, Progress, and Experience (BPE), lets the agent act on it with four compact meta-actions—track, commit, recall, note—and trains the policy in two stages: supervised fine-tuning on teacher trajectories, then cost-aware GRPO that pays for each harness call out of the same step budget as environment actions. On the ALFWorld benchmark with a Qwen3-8B backbone, the resulting agent reaches 96.9% success on seen tasks and 86.6% on unseen tasks, a 49-point absolute jump over base ReAct and roughly the level of frontier models. The paper also reports two training dynamics: harness annealing, where the policy internalizes routine scaffold usage and becomes selective, and harness evolution, where the external experience store is consolidated into a compact, task-adaptive skill bank. If correct, this makes harness use a learned decision rather than a hand-engineered scaffolding convention across long-horizon tasks.

What carries the argument

The load-bearing object is the BPE workspace (Belief, Progress, Experience), a unified harness state rendered at each step as $H_t = (B_t, P_t, E_t)$: Belief tracks what is true in the environment, Progress records committed subgoals and their status, and Experience holds cross-episode skills and search priors. The agent interacts with this workspace through four harness meta-actions—track to read belief, commit to write progress, recall to retrieve experience, and note to record new insight—and both environment and harness actions draw from the same step budget. The second mechanism is a two-stage training recipe: supervised harness fine-tuning bootstraps the action protocol from teacher demonstrations, and cost-aware GRPO with a reward combining success, efficiency, action diversity, repetition penalties, and format validity learns selective, cost-aware coordination. An environment adapter grounds BPE in ALFWorld as a rule-based world-state parser, a bounded subgoal list, and a capacity-bound skill store with LFU eviction and periodic consolidation by an external summarizer model.

What would settle it

Train EvoHarness-RL on ALFWorld with an open-weight teacher and consolidation model in place of the proprietary frontier model, keeping everything else fixed; if the seen-split success rate falls back toward the 68.6% SFT-only level—or below the best frozen memory baselines—then the frontier teacher, not the learned cost-aware coordination policy, is carrying the reported 49-point gain.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that external execution support becomes substantially more valuable when the agent is trained to control it. EvoHarness-RL abstracts heterogeneous runtime components—state trackers, subgoal records, skill banks—behind a single policy-facing interface $H_t = (B_t, P_t, E_t)$, navigated by four harness actions that consume the same interaction budget as environment actions. Supervised harness fine-tuning teaches Qwen3-8B the semantics of the action protocol, and cost-aware GRPO then optimizes when to read, write, and consolidate the workspace, with task success as the gatekeeping reward and efficiency, diversity, spam, and formatting terms shaping harness usage. Trained this way, EvoHarness-RL reports 96.9% average success on the ALFWorld seen split and 86.6% on the unseen split, outperforming both frozen memory baselines and trainable skill-learning agents, and matching or exceeding prompting-only frontier models. The paper interprets the trajectory of training as evidence that harness use anneals into selective access while the experience store evolves into a compact task-adaptive substrate.

Load-bearing premise

The load-bearing premise is that the method's gains come from the learned coordination policy rather than from the quality of the proprietary frontier teacher that supplies demonstrations and consolidates the skill store; the paper does not vary either model.

Editorial extensions

If this is right

Harness use becomes a learned policy decision: after GRPO, the agent stabilizes near one harness call per episode instead of scaffold-like frequent calls, showing the policy internalizes recurring patterns.
The BPE interface provides gains across model scales at inference time: prompt-time harness adds +22.1 for GPT-4.1, +25.7 for GPT-5, and pushes Claude Opus 4.5 from 96.4% to 98.5%.
Training on top of SFT matters: EvoHarness-SFT reaches 68.6% seen and 69.4% unseen, while adding cost-aware GRPO reaches 96.9% seen and 86.6% unseen, so the coordination optimization, not just imitation, carries most of the final gain.
The three BPE components are synergistic: ablating any one component at inference time drops average success by roughly 6 to 8 points, with the largest drop when Experience is removed.
The learned policy generalizes beyond training tasks: the RL-optimized agent outperforms both the prompt-time harness and the SFT policy on the unseen split.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the BPE abstraction is what enables transfer, the same interface could be adapted to web navigation or software engineering with a different environment adapter; the paper only instantiates ALFWorld, so this is an extension, not a claim.
Editorial inference: the annealing dynamic suggests a general curriculum—start with abundant scaffolding, then let RL prune it—that may apply to other forms of tool use, but the paper does not test this outside ALFWorld.
Editorial inference: because the teacher and consolidation model are both a proprietary frontier model, the paper does not establish whether the method's gains survive with open-weight teachers or rule-based consolidation; an ablation varying those components would decide this.
Editorial inference: the persistence of recall over commit and note after annealing suggests that cross-episode experience is the component whose value survives internalization, which would predict that environments with strong reusable priors benefit most from this method.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

EvoHarness-RL is a clear, well-executed empirical paper on a real problem: how an agent learns when to write and read external state instead of relying on hand-crafted prompts. The BPE abstraction (Belief, Progress, Experience) and the four meta-actions (track, commit, recall, note) are a sensible compact interface, and the two-stage SFT+GRPO recipe is a natural way to train it. The headline result—96.9% on ALFWorld seen, 86.6% unseen with Qwen3-8B—is strong, and the paper is careful to separate prompt-time, SFT, and RL variants.

The main weaknesses are about attribution and comparison. Table 2 ablates the BPE components only on the frozen inference-time harness, not on the trained policy that produces the headline gain. The standard-GRPO baseline (65.6) is not matched on SFT initialization, reward shaping, or the harness action space, so the +49.0 cannot be cleanly credited to learned harness coordination. The reward schedule in Eq. 6 includes a cosine-decayed diversity bonus with an explicit annealing horizon U; the 'harness annealing' dynamic in Figure 3 may be largely a consequence of that design rather than an emergent discovery. Both SFT demonstrations and experience consolidation use Claude Opus, and there is no ablation varying the teacher or replacing the consolidator with an open-weight model. The SkillRL comparison is on a smaller backbone (Qwen2.5-7B versus Qwen3-8B), which weakens the SOTA claim. And there are no error bars or released code.

None of these are fatal to the paper's value. The BPE interface and the training recipe are a genuine new combination, and the direction—learning when to use external state—is likely to be useful beyond ALFWorld. But the scientific claim about the mechanism needs a trained-policy ablation and a matched GRPO baseline to be solid. As is, it reads like a strong system paper with a slightly oversold interpretation.

This paper is worth a serious referee. I'd send it to review, and I'd ask for the missing ablations before acceptance. For my own reading group, it's a maybe: good for a discussion of agent harnesses, but not a must-read.

Referee Report

5 major / 5 minor

Summary. The paper proposes EvoHarness-RL, a two-stage training method that teaches an LLM agent to construct and selectively access an external harness state organized as Belief, Progress, and Experience (BPE). The agent interacts with this state through four meta-actions (track, commit, recall, note), and is trained first by supervised fine-tuning on teacher trajectories and then by cost-aware GRPO with a reward that combines task success, efficiency, diversity, and format penalties. On ALFWorld with Qwen3-8B, the method reportedly reaches 96.9% average success on the seen split and 86.6% on the unseen split, outperforming several memory-augmented and trainable baselines. The paper also reports two training dynamics: harness annealing (decreasing harness calls over RL) and harness evolution (growth and consolidation of the experience store). The central claim is that training a policy to coordinate with an external harness workspace yields large gains for long-horizon agents, beyond simply providing a fixed scaffold.

Significance. If validated, the central claim is significant: it suggests that the runtime policy for using memory, state trackers, and skill stores should itself be learned, rather than fixed by prompts or heuristics. The BPE abstraction is clean and well motivated, and the two-stage SFT-plus-cost-aware-GRPO recipe is a concrete, transferable design. The paper also provides useful analysis of how harness usage changes during training, and the ALFWorld instantiation is fully specified with hyperparameters in Table 4. However, the current evidence is not sufficient to isolate the claimed mechanism: the headline gain is demonstrated on a full system, while the component ablations are only run on the frozen inference-time harness, not on the trained policy, and the comparison baselines are not matched on initialization, reward terms, or backbone. The dependence on a proprietary teacher and consolidation model further limits the generalizability of the claim as stated.

major comments (5)

[Table 2 and Section 3.3] Table 2 ablates the three BPE components only on the frozen inference-time harness (EvoHarness-Base), while the top block merely lists the learned policies without any learned-policy ablation. Section 3.3 concludes that 'the absence of any single component significantly harms execution,' but this is established only for the prompt-time harness, not for the trained EvoHarness-RL policy whose gains are the headline result. Without ablations that remove Belief, Progress, or Experience from the trained policy, the +49.0 improvement in Table 1 cannot be attributed to learned harness coordination rather than to SFT initialization, reward shaping, or the teacher-curated skill bank.
[Table 1 and Section 2.4] The standard GRPO baseline (65.6%) is not matched to EvoHarness-RL: it is not initialized from the same SFT checkpoint, it does not receive the auxiliary reward terms in Eqs. (5)-(6), and it does not have the BPE action space. Therefore the comparison does not isolate the contribution of the harness policy: the gap could be driven by the extra reward shaping (efficiency, diversity, spam/format penalties) or by the SFT initialization. A controlled GRPO baseline that uses the same initialization and the same auxiliary rewards but without harness actions or harness state is needed to support the claim that harness coordination, rather than reward engineering, drives the gains.
[Section 4.1 and Eq. (6)] The reported harness-annealing dynamic may be largely a consequence of the reward design. The diversity weight lambda_div(u) in Eq. (6) is cosine-annealed over horizon U, and the efficiency bonus in Eq. (5) is described as 'naturally penalizing redundant harness queries.' Figure 3's decline in harness calls during GRPO is therefore expected from the curriculum, not necessarily an emergent internalization of harness use. The paper should either train with a fixed diversity weight or otherwise separate the effect of the annealing schedule from the policy's learned behavior.
[Table 1] The state-of-the-art comparison against SkillRL is not controlled: SkillRL is reported on Qwen2.5-7B while EvoHarness-RL uses Qwen3-8B, so the 7.0-point gap could reflect the larger or more recent backbone rather than the method. In addition, no error bars or multiple-seed variances are reported for any method, so it is unclear whether the reported differences are statistically stable. The SOTA claim requires a matched-backbone comparison and at least a small number of seeds.
[Appendix C] The method depends on a proprietary teacher and consolidation model: Claude Opus is used for SFT trajectory collection and for experience-store consolidation. This is a boundary condition on the central claim: the gains may not transfer to settings where only open-weight models are available, and no ablation varies the teacher or consolidation model. The paper should either provide such an ablation or explicitly qualify the claim that EvoHarness-RL teaches an open-weight agent to self-evolve its harness.

minor comments (5)

[Conclusion] The conclusion begins with 'We introduced We introduced EvoHarness-RL'; the duplicated phrase should be removed.
[Figures 3 and 4] The plots show a single training run with no error bars or confidence intervals, despite being used to support the annealing and evolution dynamics; reporting multiple seeds would strengthen these analyses.
[Section 3.1] The paper does not state how many seeds or runs the main results in Tables 1 and 3 are based on; adding this information is important for reproducibility.
[Appendix C] The SFT dataset is small (87 trajectories, 1,153 conversation pairs), and the paper does not report whether the 500 training games used for collection were randomly sampled or whether the distribution over the six task families affects the results; a brief sensitivity check would help.
[General] No code or data are released, which makes it difficult to independently verify the reported harness dynamics and the exact reward normalization in Eq. (5); a public release or a detailed pseudocode would improve reproducibility.

Circularity Check

2 steps flagged · score 6.0 of 10

The two reported 'dynamics'—harness annealing and harness evolution—are encoded in the reward schedule and consolidation prompt by construction; the main success numbers remain empirical but these discovery claims are circular.

self definitional [Section 2.4, Eq. (6); Section 4.1, Figure 3]
"To prevent policy collapse... we introduce a time-dependent vocabulary diversity bonus: R_div(τ)=|{verb(a_t):a_t∈τ}|/|τ|, λ_div(u)=λmax_div/2(1+cos πu/U). This curriculum encourages broad exploration of harness actions early in training, before gracefully decaying to force specialization and efficient task resolution. ... Figure 3 shows a clear annealing pattern during GRPO... usage drops quickly and stabilizes near one call per episode."

The reward function literally contains an annealing curriculum: the cosine-decayed diversity weight λ_div(u) is designed to 'force specialization' over U epochs, and the efficiency bonus R_eff penalizes every step, including every harness call, since harness actions consume the same interaction budget. The paper then reports the resulting decline in harness calls as an emergent 'harness annealing' dynamic in which 'GRPO gradually internalizes routine scaffolded behaviors.' The observed pattern is a stated consequence of the reward schedule, not an independent discovery.
self definitional [Appendix D.2, Note-consolidation prompt; Section 4.2, Figure 4]
"Your job is to maintain a clean, deduplicated, conflict-free skill bank... For each note, decide ONE of: 1.ADD... 2.UPDATE... 3.REMOVE... 4.SKIP... Rules: prefer UPDATE over ADD on overlap; use REMOVE only when a note clearly invalidates an existing skill; keep principles short and concrete... Return ONLY the JSON object. ... The skill bank expands rapidly early in training... later, growth becomes selective as redundant entries are merged, rarely useful skills are evicted..."

The 'harness evolution' dynamic—selective growth followed by compaction into a stable, task-adaptive store—is explicitly programmed into the consolidation prompt's add/update/remove/skip decision rules, together with Table 4's capacity caps (K_max=80 per category) and LFU eviction. Reporting this as an emergent co-adaptive loop is a restatement of the consolidation instructions and store-management hyperparameters, not a separately observed phenomenon.

full rationale

The main empirical claim—that SFT plus cost-aware GRPO raises ALFWorld success from 56.4% to 96.9% on Qwen3-8B—is an independent experimental result: the policy is trained on task success and compared against ReAct, GRPO, SkillOS, and SkillRL. That comparison is not circular by construction. However, the paper presents 'harness annealing' and 'harness evolution' as key dynamics revealed by training, and both are direct outputs of the designed reward terms and consolidation rules. The efficiency bonus (Eq. 5) penalizes long trajectories, and since harness actions consume steps, it penalizes harness calls; the diversity bonus (Eq. 6) is explicitly a cosine-decayed annealing curriculum meant to 'force specialization.' The consolidation prompt in Appendix D.2 instructs the summarizer to prefer UPDATE over ADD, REMOVE stale skills, and SKIP redundancy, while Table 4 enforces finite capacity and LFU eviction. The observed curves in Figures 3 and 4 therefore follow from the setup rather than from unanticipated policy behavior. The Table 2 ablation gap—removing BPE components only from the frozen inference-time harness, not from the trained RL policy—is a real confound for attributing the 96.9% result to learned harness coordination, but it is an experimental omission rather than a circular derivation. No load-bearing self-citation chain or imported uniqueness theorem was found; the self-citations to the authors' prior harness/memory work are contextual. Overall, the central success numbers stand on their own, but the two headline 'dynamics' reduce to the reward and consolidation design, giving a partial-circularity score of 6.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The training recipe relies on a handful of hand-chosen reward weights and several domain assumptions, most notably that ALFWorld represents long-horizon agent tasks and that the proprietary teacher and consolidation model are reliable. The BPE abstraction is an invented software construct with empirical support, while the harness dynamics are partly induced by the reward schedule.

free parameters (6)

Efficiency reward weight lambda_eff = 1.0
Hand-chosen in Eq. 5; scales the success-gated trajectory length penalty that incentivizes selective harness use.
Max diversity weight lambda_max_div = 0.5
Hand-chosen in Eq. 6; controls the early-training exploration bonus that decays over the annealing horizon.
Spam penalty weight lambda_spam = 0.1
Hand-chosen in Eq. 5; penalizes repetitive action sequences.
Invalid-action penalty weight lambda_inv = 0.1
Hand-chosen in Eq. 5; penalizes malformed or out-of-space actions.
Annealing horizon U = 150
Hand-chosen total RL epochs; defines the cosine decay of the diversity bonus and is the same as the training length, so the annealing dynamic is scheduled.
Success reward scale = 10.0
Hand-chosen; sets the relative strength of the sparse success signal against shaping terms.

assumptions (5)

standard math GRPO is a valid optimization algorithm for this trajectory-level reward
Uses Shao et al. (2024) GRPO without modification beyond the custom reward.
domain assumption BPE (Belief, Progress, Experience) covers the recurring failure modes of long-horizon agents
Section 2.1 asserts these three roles capture state loss, progress loss, and experience reuse without empirical proof that other roles are unnecessary.
domain assumption ALFWorld task success is a meaningful proxy for long-horizon agent capability
All experiments are on one text-based household benchmark; the general conclusion in Section 6 extrapolates beyond this environment.
domain assumption Teacher and consolidation model demonstrations are reliable and do not leak privileged information
Appendix C uses Claude Opus to collect SFT trajectories and to merge notes into the skill store; no ablation shows the method works with an open-weight teacher.
ad hoc to paper The reward auxiliary terms shape rather than predetermine the reported dynamics
Equation 5 includes an efficiency bonus explicitly described as penalizing redundant harness queries, and Eq. 6 schedules the diversity bonus to decay; the observed harness annealing is partly a consequence of the chosen rewards.

invented entities (2)

BPE (Belief, Progress, Experience) external workspace independent evidence
purpose: Policy-facing harness state that the agent reads and writes via track, commit, recall, note meta-actions.
It is a software abstraction directly implemented and measured in the ALFWorld adapter; its value is evidenced by ablations in Table 2, so it has empirical falsifiable support within the paper.
Harness annealing and harness evolution dynamics
purpose: Labeled training-time phenomena claimed to explain how the policy and skill store co-adapt.
These are post-hoc characterizations of the training curves in Figures 3 and 4; the annealing pattern is partly induced by the designed reward schedule, so there is no independent evidence outside the paper's own setup.

how reviews work

0 comments

Cite this review

Pith. "Pith review of EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents." pith.science (2026). https://pith.science/paper/PYMPGEDG

@misc{pith2026260805446,
  author       = {Pith},
  title        = {Pith review of: EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PYMPGEDG}},
  note         = {Machine review of arXiv:2608.05446}
}

read the original abstract

Long-horizon LLM agents increasingly rely on external execution support to maintain state, track progress, invoke tools, verify outcomes, and reuse experience across interactions. However, effective harness use raises two coupled challenges: state formation from noisy interaction traces and runtime control over external-state access. Existing agents usually handle both through prompts, heuristics, or domain-specific conventions, leaving the external workspace and its usage policy manually engineered. To address this, we study the problem of harness policy learning, where agents learn harness policies offline and deploy them to construct and update external harness state online during runtime task execution. We introduce EvoHarness-RL, which exposes Belief, Progress, and Experience (BPE) as policy-facing harness state. Supervised harness fine-tuning teaches the base agent the harness action space and how to construct useful external state, while cost-aware GRPO explores coordination policies to selectively read, update, and consolidate that state during long-horizon interaction. Instantiated on ALFWorld with a Qwen3-8B LLM, EvoHarness-RL reaches 96.9% success and reveals two key dynamics: harness annealing, where training internalizes recurring harness-use patterns into the model policy and shifts the agent from frequent harness calls toward selective external-state access, and harness evolution, where progress updates and experience consolidation refine the harness into a compact, task-adaptive state substrate. These results suggest that long-horizon agents benefit from trainable policies for constructing and coordinating with external harness workspaces, beyond simply adding stronger tools or larger memories.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 4 canonical work pages

[1]

Harnessx: A composable, adaptive, and evolvable agent harness foundry.arXiv preprint arXiv:2606.14249,

Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, et al. Harnessx: A composable, adaptive, and evolvable agent harness foundry.arXiv preprint arXiv:2606.14249,

arXiv
[3]

Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

Pengcheng Jiang, Zhiyi Shi, Kelly Hong, Xueqiang Xu, Jiashuo Sun, Jimeng Sun, Hammad Bashir, and Jiawei Han. Harness-1: Reinforcement learning for search agents with state-externalizing harnesses.arXiv preprint arXiv:2606.02373,

work page Pith review arXiv
[4]

Meta-harness: End-to- end optimization of model harnesses.arXiv preprint arXiv:2603.28052,

Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-harness: End-to- end optimization of model harnesses.arXiv preprint arXiv:2603.28052,

arXiv
[5]

A survey of rag-reasoning systems in large language models

Yangning Li, Weizhi Zhang, Yuyao Yang, Wei-Chieh Huang, Yaozu Wu, Junyu Luo, Yuanchen Bei, Henry Peng Zou, Xiao Luo, Yusheng Zhao, et al. A survey of rag-reasoning systems in large language models. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 12120–12145,

work page 2025
[7]

Code as agent harness.arXiv preprint arXiv:2605.18747,

Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai, Zhining Liu, Ting-Wei Li, et al. Code as agent harness.arXiv preprint arXiv:2605.18747,

arXiv
[8]

Reasoningbank: Scaling agent self-evolving with reasoning memory.arXiv preprint arXiv:2509.25140,

Siru Ouyang, Jun Yan, I Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T Le, Samira Daruki, Xiangru Tang, et al. Reasoningbank: Scaling agent self-evolving with reasoning memory.arXiv preprint arXiv:2509.25140,

arXiv
[9]

Skillos: Learning skill curation for self-evolving agents.arXiv preprint arXiv:2605.06614,

9 Siru Ouyang, Jun Yan, Yanfei Chen, Rujun Han, Zifeng Wang, Bhavana Dalvi Mishra, Rui Meng, Chun-Liang Li, Yizhu Jiao, Kaiwen Zha, et al. Skillos: Learning skill curation for self-evolving agents.arXiv preprint arXiv:2605.06614,

arXiv
[11]

Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty

Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALF- World: Aligning text and embodied environments for interactive learning. InInternational Conference on Learning Representations, 2021.https://openreview.net/forum?id=0IOX0YcCdTn. Joykirat Singh, Zaid Khan, Archiki Prasad, Justin Chih-Yao Chen, Aksh...

work page Pith review arXiv 2021

Show all 18 references

[13]

A subgoal-driven framework for improving long-horizon llm agents.arXiv preprint arXiv:2603.19685,

Taiyi Wang, Sian Gooding, Florian Hartmann, Oriana Riva, and Edward Grefenstette. A subgoal-driven framework for improving long-horizon llm agents.arXiv preprint arXiv:2603.19685,
[14]

Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory

Tianxin Wei, Noveen Sachdeva, Benjamin Coleman, Zhankui He, Yuanchen Bei, Xuying Ning, Mengting Ai, Yunzhe Li, Jingrui He, Ed H Chi, et al. Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory. arXiv preprint arXiv:2511.20857,

arXiv
[15]

Skillrl: Evolving agents via recursive skill-augmented reinforcement learning.arXiv preprint arXiv:2602.08234,

Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, et al. Skillrl: Evolving agents via recursive skill-augmented reinforcement learning.arXiv preprint arXiv:2602.08234,

arXiv
[16]

React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629,

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629,

arXiv
[18]

Webarena: A realistic web environment for building autonomous agents

Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. InInternational Conference on Learning Representations, volume 2024, pag...

2024
[2022]

Effective harnesses for long-running agents

Justin Young. Effective harnesses for long-running agents. Anthropic Engineering Blog, November 2025.https: //www.anthropic.com/engineering/effective-harnesses-for-long-running-agents. Accessed: 2026-05-11. Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong,...

2025 arXiv
[2023]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

arXiv
[2024]

Jiayu Wang, Yifei Ming, Riya Dulepet, Qinglin Chen, Austin Xu, Zixuan Ke, Frederic Sala, Aws Albarghouthi, Caiming Xiong, and Shafiq Joty

ISSN 2835-8856.https://openreview.net/forum?id=ehfRiF0R3a. Jiayu Wang, Yifei Ming, Riya Dulepet, Qinglin Chen, Austin Xu, Zixuan Ke, Frederic Sala, Aws Albarghouthi, Caiming Xiong, and Shafiq Joty. Liveresearchbench: A live benchmark for user-centric deep research in the wild....

arXiv
[2025]

Autoharness: improving llm agents by automatically synthesizing a code harness.arXiv preprint arXiv:2603.03329,

Xinghua Lou, Miguel Lázaro-Gredilla, Antoine Dedieu, Carter Wendelken, Wolfgang Lehrach, and Kevin P Murphy. Autoharness: improving llm agents by automatically synthesizing a code harness.arXiv preprint arXiv:2603.03329,
[2026]

Memp: Exploring agent procedural memory

Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. Memp: Exploring agent procedural memory. InFindings of the Association for Computational Linguistics: ACL 2026, pages 17490–17502,

2026

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Harnessx: A composable, adaptive, and evolvable agent harness foundry.arXiv preprint arXiv:2606.14249,

Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, et al. Harnessx: A composable, adaptive, and evolvable agent harness foundry.arXiv preprint arXiv:2606.14249,

arXiv

[2] [3]

Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

Pengcheng Jiang, Zhiyi Shi, Kelly Hong, Xueqiang Xu, Jiashuo Sun, Jimeng Sun, Hammad Bashir, and Jiawei Han. Harness-1: Reinforcement learning for search agents with state-externalizing harnesses.arXiv preprint arXiv:2606.02373,

work page Pith review arXiv

[3] [4]

Meta-harness: End-to- end optimization of model harnesses.arXiv preprint arXiv:2603.28052,

Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-harness: End-to- end optimization of model harnesses.arXiv preprint arXiv:2603.28052,

arXiv

[4] [5]

A survey of rag-reasoning systems in large language models

Yangning Li, Weizhi Zhang, Yuyao Yang, Wei-Chieh Huang, Yaozu Wu, Junyu Luo, Yuanchen Bei, Henry Peng Zou, Xiao Luo, Yusheng Zhao, et al. A survey of rag-reasoning systems in large language models. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 12120–12145,

work page 2025

[5] [7]

Code as agent harness.arXiv preprint arXiv:2605.18747,

Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai, Zhining Liu, Ting-Wei Li, et al. Code as agent harness.arXiv preprint arXiv:2605.18747,

arXiv

[6] [8]

Reasoningbank: Scaling agent self-evolving with reasoning memory.arXiv preprint arXiv:2509.25140,

Siru Ouyang, Jun Yan, I Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T Le, Samira Daruki, Xiangru Tang, et al. Reasoningbank: Scaling agent self-evolving with reasoning memory.arXiv preprint arXiv:2509.25140,

arXiv

[7] [9]

Skillos: Learning skill curation for self-evolving agents.arXiv preprint arXiv:2605.06614,

9 Siru Ouyang, Jun Yan, Yanfei Chen, Rujun Han, Zifeng Wang, Bhavana Dalvi Mishra, Rui Meng, Chun-Liang Li, Yizhu Jiao, Kaiwen Zha, et al. Skillos: Learning skill curation for self-evolving agents.arXiv preprint arXiv:2605.06614,

arXiv

[8] [11]

Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty

Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALF- World: Aligning text and embodied environments for interactive learning. InInternational Conference on Learning Representations, 2021.https://openreview.net/forum?id=0IOX0YcCdTn. Joykirat Singh, Zaid Khan, Archiki Prasad, Justin Chih-Yao Chen, Aksh...

work page Pith review arXiv 2021