Artificial Intelligence · arXiv:2611.18533 v1 · submitted 13 Nov 2026 · 25 references
ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL
The paper says
“Agentic reinforcement learning (RL) increasingly relies on a harness, the scaffold around the model that decides what it sees of its own interaction and what guidance it receives. Most existing approaches train under a hand-written harness that stays fixed throughout.”
more of the abstract
However, an effective harness depends on the domain, the data and the model, and keeps changing as the model trains. We introduce ModHarness-RL, which recasts the harness as programs that are learned from data and evolve as the model trains with RL. ModHarness-RL records the interaction as structured State, a set of typed blocks, and two core components of the harness are programs over these blocks, a Compression program that assembles the accumulated history into the model's context and a Steering program that supplies guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. We study CodeSearch in depth and further analyze ALFWorld and BIRD SQL, with models from 1.5B to 8B. The proposed Compression and Steering mechanisms improve held-out performance over the fixed-harness baseline in most settings. These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 16 Nov 2026 · read the full text (114,619 characters) · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A principled method for learning and evolving the harness (Compression and Steering programs over typed interaction blocks) alongside RL training, with thorough CodeSearch results and honest limitations. This line is wrong
8 minor · 1 open question
How often objections from reviews like this one are right: not yet measured
The paper, first character to last
The reviewer was given the whole text, 114,619 characters. Not checked: figures, code, what the 25 references say.
8 minor objections
The paper says, at §1
Pith says: minor objection 1 of 8
The paper uses many custom macros (e.g., CsOursLo, BenchCsSteer) that expand to numbers. While this is common in LaTeX, it makes the text hard to read without the macro definitions. Consider using explicit numbers or a table for clarity.
The paper says, at §4.1, Table 1
Pith says: minor objection 2 of 8
The table caption says 'F1 averaged over the three granularities' but the table shows 'Overall' which is the average. It would be helpful to explicitly state that the F1 is the mean of file, module, and entity F1 scores.
show 6 more
The paper says, at §4.2, Figure 3
Pith says: minor objection 3 of 8
The caption says 'Prefill tokens are the prompt tokens not served from the prefix cache, summed over an episode and measured by replaying the recorded episodes'. This is a good definition, but it would be helpful to also report the raw prompt token counts for completeness.
The paper says, at §5
Pith says: minor objection 4 of 8
The Limitations section acknowledges the proprietary creator and the scope of comparison. However, it does not discuss the potential for the LLM creator (GPT-5-mini) to introduce biases or the reproducibility concerns of using a proprietary model. The paper should mention that the results may not be fully reproducible if the creator model is not publicly available.
The paper says, at §A.2, Table 5
Pith says: minor objection 5 of 8
The ablation table shows that 'w/o generation' and 'w/o judging' both reduce performance. However, the paper does not explain what 'w/o generation' means in detail. From context, it means the creator proposes no new triggers during training, but the paper should state this explicitly.
The paper says, at §A.1
Pith says: minor objection 6 of 8
The paper states that 'every method that learns from rollouts uses GPT-5-mini as its creator'. This is good for fairness, but it also means that the baselines are not necessarily the original implementations. The paper should clarify whether the baselines are re-implemented or adapted, and whether any hyperparameter tuning was done for them.
The paper says, at §3.2
Pith says: minor objection 7 of 8
The algorithm (Algorithm 1) is presented in a wrapfigure environment, which may cause formatting issues. The pseudocode is clear, but the use of exttt{Call} functions without definitions makes it hard to follow. Consider providing more detailed pseudocode or a clearer description of each subroutine.
The paper says, at §4.2, Figure 4
Pith says: minor objection 8 of 8
The figure shows 'F1 against inference time' but the inference time is measured in seconds per episode. It would be helpful to also report the number of turns per episode, as inference time depends on episode length.
Open questions (1)
Pith raised these and could not confirm them against the text of the paper it read. They are questions for the authors, not findings.
- §3.2, §4.3 Pith raised a concern here that it could not check against the text of the paper it was given, so it is not published as an objection.
What would settle it
If one trained a model with ModHarness-RL on a structured environment like CodeSearch and found that the learned Compression and Steering programs, when applied to a held-out set, consistently perform worse than a simple append-only baseline with the same context budget, the central claim would be falsified. The paper already reports such comparisons and finds improvements, so a falsifier would be a reproduction where the improvements fail to replicate, or where the programs learned are consistently inert (do not change the context) or unfaithful (remove task-relevant information).
from Pith's summary of the paper
How the review went
5 passes by deepseek-v4-flash · 16 Nov 2026
- Reader holds up, with conditions
- Skeptic concern withdrawn: the paper already says this
- Referee 8 minor
- Check against the paper kept 0 of 5 objections
- Editor wrote the verdict above
Not run on this paper: the simulated authors' defence.
Pith's passes raised 5 objections. A check against the paper's text kept 0, withdrew 4 (4 the paper already says) and left 1 as a question.
The review in full
The review in brief, machine-written
This paper introduces ModHarness-RL, which treats the harness of an agentic RL system as programs over typed interaction blocks and learns/evolves these programs during training. The core idea is clean: represent Compression (how history is assembled into context) and Steering (when to supply guidance) as deterministic programs over a structured State, propose candidates from training trajectories, verify by execution, and gate on held-out tasks. The method is evaluated on CodeSearch (in depth), ALFWorld, and BIRD SQL, with models from 1.5B to 8B under two context budgets.
The paper does several things well. The representation of harness components as deterministic programs is genuinely novel and addresses a real bottleneck: harness design is currently manual and task-specific. The replay mechanism that checks whether a failure persists before installing guidance is sensible. The CodeSearch results are strong: ModHarness-RL improves over the fixed-harness baseline by 14-24% and outperforms memory, skill, prompt, and harness evolution baselines. The ablation study and the analysis of the judge's decisions are informative. The paper is clearly written and the limitations section is unusually honest.
The soft spots are real but acknowledged. The full method (Compression + Steering) is only evaluated on CodeSearch; ALFWorld and BIRD SQL test individual components. The programs are written by a proprietary model (GPT-5-mini), which is a reproducibility concern—though the paper notes this and uses the same creator for all baselines. The cost comparison with GPT compaction uses estimated figures. The gate's held-out set is small, which the authors flag. These are not fatal flaws, but they mean the central claim is not fully supported across all claimed settings.
This paper is for researchers working on LLM agent training, particularly those frustrated with manual harness design. It deserves a serious referee. The method is well-motivated, the experiments are thorough for the primary environment, and the limitations are stated rather than hidden. A referee should ask for the full method on at least one more environment and for the generated programs to be released, but the core contribution is solid.
The stress test's concern
Withdrawn: the paper already says this.
The paper says
“The programs are written by a proprietary model, GPT-5-mini. To enable a fair comparison, every learning baseline uses the same creator. GPT-5-mini writes the \Compression\ and \Steering\ programs of \ours, the memories, skills, prompts and harness edits of the baselines, and the summaries of the compaction baseline.”
The referee's report
Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 4 of its 4 major objections did not survive that check.
Pith's reading of the paper (a machine's summary, not the authors')
The paper introduces ModHarness-RL, a method that recasts the harness (the scaffold around an LLM agent that decides what it sees and what guidance it receives) as deterministic programs over a structured State of typed interaction blocks. Two core components are learned and evolved: a Compression program that assembles the accumulated history into the model's context under a budget, and a Steering program that supplies guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. The paper reports that on CodeSearch, ModHarness-RL improves over the fixed-harness baseline by 14 to 24% in every setting and scores highest against memory, skill, prompt and harness evolution baselines. On ALFWorld and BIRD SQL, each program alone improves over the base harness in most settings.
What carries the argument. The central object is the harness represented as deterministic programs over a structured State (s_t), which records each turn as typed blocks (reasoning, tool call, tool result). Two programs are learned: Compression, which reads the blocks and returns a new view that fits the context budget while keeping key information, and Steering, which is a set of triggers (small programs over s_t) that fire on detectable failure modes and return guidance. The key mechanism is the Replay process: because each trigger is a program, it can be executed over all stored trajectories from past training steps to determine whether a failure is persisting or being learned away, and a judge decides whether to install the guidance. The programs are proposed by an LLM creator (GPT-5-mini) from training trajectories, verified by execution, and gated on held-out training tasks.
Core claim. The central claim is that the harness for agentic RL can be represented as deterministic programs over typed interaction blocks (State), and that these programs can be learned from data and evolved with the model during RL training. The paper proposes two such programs: Compression, which re-renders earlier turns to fit a context budget while keeping key information, and Steering, which detects specific failure patterns and appends guidance only when they fire. The key insight is that the harness should guide only what the model cannot learn on its own, so Steering programs are replayed over past training trajectories and installed only for failures that persist. The paper demonstrates that this approach improves held-out performance over a fixed hand-written harness in most settings across three environments (CodeSearch, ALFWorld, BIRD SQL) with models from 1.5B to 8B parameters.
The premise it rests on. The harness can be effectively represented as deterministic programs over typed interaction blocks, and the patterns worth steering (e.g., repeated tool calls) can be detected by simple rules written by an LLM from training trajectories.
If the paper is right (Pith's inferences, not the authors')
If the harness can be learned from data and evolved with the model, then agentic RL systems can automatically adapt their scaffolding to the task, data, model, and context budget without manual tuning.
The Compression program's ability to reduce prefill tokens while maintaining or improving performance suggests that learned context management can be more efficient than append-only history, even under sufficient budgets.
The Steering program's replay mechanism provides a principled way to decide when to intervene: only for failures that the policy does not learn to fix on its own, avoiding unnecessary guidance that could interfere with learning.
The approach outperforms methods that learn memory, skills, prompts, or harness edits from the same creator model, suggesting that representing harness components as deterministic programs over typed blocks is a more effective representation for learnable scaffolding.
The method beats zero-shot GPT-5-mini (the creator of its harness) in every setting, indicating that the gain is not simply distillation from a stronger model.
The reliance on a proprietary model (GPT-5-mini) to write the programs is a practical limitation, but the framework itself is model-agnostic: any sufficiently capable LLM could serve as the creator, and open-source models may eventually match this capability.
The deterministic program representation limits the harness to patterns detectable by explicit rules over typed interaction blocks. For open-ended domains like conversation where relevant signals are semantic, a learned program representation may need to incorporate learned embeddings or neural detectors, which would sacrifice the computational efficiency and replayability that the current approac
The paper's ablation showing that evolving Compression (rather than fitting it once) reduces performance suggests a trade-off: context stability during policy training may be more important than adapting the compression to the changing policy. This could be tested by varying the frequency of Compression updates.
The replay mechanism for Steering programs could be extended to automatically propose new triggers by mining trajectories for recurring patterns, rather than relying on an LLM to hypothesize failures from mixed-outcome groups.
Check this review
What was read
The passes were given all 114,619 characters of the paper's LaTeX source.
Fingerprint of that text (SHA-256): 2cdda69d7f1101c345a040c88435edad0153b0916efe6632f0fb584463394805
Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:
PYTHONPATH=. python scripts/review_input_check.py 2611.18533
The passes, in the order they ran
- Reader deepseek-v4-flash · medium effort · $0.0043 · instructions of the Reader, fingerprint f9ead7300382…
- Skeptic deepseek-v4-flash · medium effort · $0.0038 · instructions of the Skeptic, fingerprint 8f88c0f61488…
- Referee deepseek-v4-flash · medium effort · $0.0040 · instructions of the Referee, fingerprint a852d99aea0d…
- Check against the paper deepseek-v4-flash · medium effort · $0.0054 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
- Editor deepseek-v4-flash · medium effort · $0.0038 · instructions of the Editor, fingerprint e7211f6a3d27…
- Circularity check deepseek-v4-flash · medium effort · $0.0037 · instructions of the Circularity check, fingerprint e9946e39d67b…
- Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0041 · instructions of the Summary for the page, fingerprint e4f56bf568e7…
The passes cost $0.029 in model calls together.
Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00
Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.
curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum
prints the fingerprint of the reader's instructions. The same works for every pass.
The receipt
The review's fingerprint was taken on 16 Nov 2026. The review you are reading still matches it.
Fingerprint of the review (SHA-256): 6224e5a71e52f50dad09c8fae905ac32d63338ccf6e5724f9402bf61ba6c36ec
curl -s 'https://pith.science/api/v1/verdicts/2611.18533?form=canonical' | sha256sum
prints the fingerprint of the review as it stands now. The rule, in four steps.
A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, from the paper's LaTeX source.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Agentic reinforcement learning (RL) increasingly relies on a harness, the scaffold around the model that decides what it sees of its own interaction and what guidance it receives. Most existing approaches train under a hand-written harness that stays fixed throughout. However, an effective harness depends on the domain, the data and the model, and keeps changing as the model trains. We introduce ModHarness-RL, which recasts the harness as programs that are learned from data and evolve as the model trains with RL. ModHarness-RL records the interaction as structured State, a set of typed blocks, and two core components of the harness are programs over these blocks, a Compression program that assembles the accumulated history into the model's context and a Steering program that supplies guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. We study CodeSearch in depth and further analyze ALFWorld and BIRD SQL, with models from 1.5B to 8B. The proposed Compression and Steering mechanisms improve held-out performance over the fixed-harness baseline in most settings. These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-16
The authors' main claim, as the classifier picked it out 1.00
“These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model.”
How the main claim is stated: hedged possibility 1.00
What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.14 · human participants 0.03 · controlled comparison 0.96 · randomized assignment 0.01 · states quantitative result 0.19 · compares with prior results 0.96 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Agentic reinforcement learning (RL) increasingly relies on a harness, the scaffold around the model that decides what it sees of its own interaction and what guidance it receives. | background 1.00 | 0.05 | 0.02 | asserted 0.99 |
| Most existing approaches train under a hand-written harness that stays fixed throughout. | background 0.85 | 0.05 | 0.02 | asserted 1.00 |
| However, an effective harness depends on the domain, the data and the model, and keeps changing as the model trains. | gap 0.62 | 0.08 | 0.06 | asserted 1.00 |
| We introduce ModHarness-RL, which recasts the harness as programs that are learned from data and evolve as the model trains with RL. | contribution 0.99 | 0.90 | 0.15 | asserted 1.00 |
| ModHarness-RL records the interaction as structured State, a set of typed blocks, and two core components of the harness are programs over these blocks, a Compression program that assembles the accumulated history into the model's context and a Steering program that supplies guidance when behavior deviates. | method 0.97 | 0.82 | 0.04 | asserted 0.99 |
| During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. | method 1.00 | 0.89 | 0.07 | asserted 0.99 |
| We study CodeSearch in depth and further analyze ALFWorld and BIRD SQL, with models from 1.5B to 8B. | method 0.70 | 0.92 | 0.02 | asserted 0.90 |
| The proposed Compression and Steering mechanisms improve held-out performance over the fixed-harness baseline in most settings. | result 1.00 | 0.95 | 0.23 | asserted 1.00 |
| These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model. | interpretation 1.00 | 0.77 | 0.90 | hedged possibility 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.94
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-16: 1 request, 6,922 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (14)
Showing 13 of the paper's 14 figures. Figures are the authors', reproduced from the paper.
References (25)
- [3] URL https://arxiv.org/ abs/2610.09832. 11 Prannay Hebbar, Yogendra Manawat, Samuel Verboomen, Alesia Ivanova, Selvam Palanimalai, Kunal Bhatia, and Vignesh Baskaran. SIA: Self improving AI with harness & weight updates, on PithPith's reviewarXiv
- [4] URL https://arxiv.org/abs/2605.27276. Kelly Hong, Anton Troynikov, and Jeff Huber. Context rot: How increasing input tokens impacts LLM performance. Technical report, Chroma, July on PithPith's reviewarXiv
- [5] URLhttps://arxiv.org/abs/2609.00196. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. InProceedings of the 29th Sympo... on PithPith's reviewarXiv
- [7] URLhttps://arxiv.org/abs/2609.37105. 12 Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. ToolRL: Reward is all tool learning needs. InAdvances in Neural Information Processing Systems, volume 38, pages 105523–105553, on PithPith's reviewarXiv
- [8] URLhttps://arxiv.org/abs/2609.37725. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models, on PithPith's reviewarXiv
- [10] URL https://arxiv.org/abs/ 2607.21419. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning text and embodied environments for interactive learning. In International Conference on Learning Representations, on PithPith's reviewarXiv
- [11] URL https: //arxiv.org/abs/2603.17829. Lev S. Vygotsky.Mind in Society: Development of Higher Psychological Processes. Harvard University Press, Cambridge, MA, on PitharXiv
- [12] doi: 10.18653/v1/2026.acl-long.69. Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Fei-Fei Li, Lijuan Wang, Yejin Choi, and Manling Li. RAGEN:... on Pithdoi:10.18653/v1/2026.acl-long.69
- [13] URLhttps://arxiv.org/abs/2504.20073. Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li. WebAgent-R1: Training web agents via end-to- end multi-turn reinforcement learning. InProceedings of... on PithPith's reviewarXiv
- [14] doi: 10.18653/v1/2025.emnlp-main.401. David Wood, Jerome S. Bruner, and Gail Ross. The role of tutoring in problem solving.Journal of Child Psychology and Psychiatry, 17(2):89–100, on Pithdoi:10.18653/v1/2025.emnlp-main.401
- [16] URLhttps://arxiv.org/abs/2509.13313. Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Demystifying LLM-based software engineering agents.Proceedings of the ACM on Software Engineering, 2(FSE):801–824, on PitharXiv
- [17] URLhttps://arxiv.org/abs/2602.08234. Can Xie, Yuyi Zhou, Wen Yang, Ziyi Zhang, Siyao Song, Yingzhuo Deng, Shuo Ren, and Jiajun Zhang. EDGE: Experience-distillation for guided exploration in agentic reinforcement learning, on PithPith's reviewarXiv
- [18] URL https://arxiv.org/abs/2608.21946. Zekun Xu, Siyu Xia, Chuhuai Yue, Jiajun Chai, Mingxue Tian, Xiaohan Wang, Wei Lin, Haoxuan Li, and Guojun Yin. MTIR-SQL: Multi-turn tool-integrated reasoning reinforcement learning for text-to-SQL, on PithPith's reviewarXiv
- [19] URL https://arxiv. org/abs/2512.24601. Han Zhou, Xingchen Wan, Ivan Vuli´ c, and Anna Korhonen. Agentic policy optimization via instruction- policy co-evolution, on PithPith's reviewarXiv
- [20] URLhttps://arxiv.org/abs/2512.01945. Yang Zhou, Sunzhu Li, Shunyu Liu, Wenkai Fang, Kongcheng Zhang, Jiale Zhao, Jingwen Yang, Yihe Zhou, Jianwei Lv, Tongya Zheng, Hengtong Lu, Chen Wei, Xie Yan, and Mingli Song. Breaking the exploration bottleneck: Rubric-scaffolded reinforce... on PitharXiv
- [21] in verl (Sheng et al., 2025), and vLLM (Kwon et al., on Pith
- [22] samples a group of G trajectories {τi}Gi=1 from the current policy and normalizes their rewardsR i =r(τ i)within the group to form advantages, ˆAi = Ri −mean({R j}Gj=1)std({R j}Gj=1) . (3) The policy is updated by the clipped surrogate LGRPO(θ) = 1G G∑i=1 1|τi| |τi|∑t=1 min ρi... on Pith
- [23] and 88 from SWE-Bench Pro (Deng et al., 2026). We use Qwen3-4B and Qwen3-8B. Each step samples 8 tasks and 8 episodes per task, and makes one policy update on the whole batch. We use a constant learning rate of 10−6 with no warmup, a low-variance KL loss with coefficient 0.001... on Pith
- [40] and every 20 steps (K= 10). Table 4 shows that an interval of 10 scores the highest with Qwen3-4B and ties with an interval of 20 with Qwen3-8B. Update interval (steps) 5 10 20 Qwen3-4B .404.424.404 Qwen3-8B .429.460 .461 Table 4:Overall score of MODHARNESS-RL on CodeSearch un... on Pith
- [128] No program is chosen on an evaluation set, and Dholdtrain is disjoint from every evaluation set we report. A.4 History Budget and Overflow We follow the design of verl-agent (Feng et al., 2025), which drops earlier turns from the history and tells the policy that they existed.... on Pith
show 5 more
- [1976] doi: 10.1111/j.1469-7610.1976.tb00381.x. Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, and Botian Shi. From interactions to principles: Experience-driven self-distillation for evolving LLM agents. ... on Pithdoi:10.1111/j.1469-7610.1976.tb00381.x
- [2023] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, 12:157–173, 2024a. doi: 10.1162/tacl_a_00638. X... on Pithdoi:10.1162/tacl_a_00638
- [2024] URLhttps://arxiv.org/abs/2402.03300. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. HybridFlow: A flexible and efficient RLHF framework. InProceedings of the Twentieth European Conference on Computer Systems, ... on PithPith's reviewarXiv
- [2025] URL https: //arxiv.org/abs/2506.11425. Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa R. Kundurthy, Sean M. Hendryx, Zifan Wang, Chen Bo Calvin Zhang, Noah Jacobson, ... on PithPith's reviewarXiv
- [2026] URL https://trychroma.com/research/ context-1. Mingju Chen, Can Lv, Guibin Zhang, Heng Chang, and Shiji Zhou. HarnessForge: Joint harness and policy evolution for adaptive agent systems, 2026a. URLhttps://arxiv.org/abs/2606.01779. Zhengyu Chen, Teng Xiao, Huaisheng Zhu, Yige Y... on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment