Machine Learning · arXiv:2610.37255 v2 · submitted 27 Oct 2026 · 41 references
Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces
The paper says
“Language model agents are post-trained and deployed through a harness, the prompts, tools, and control logic that present a task to the model and execute its outputs.”
more of the abstract
Harnesses are often revised and judged by how an already trained model performs under them, which assumes that what the model learned under one harness carries over to the next. We ask what a policy learns from its harness and what survives when the harness changes. Under KL-regularized control, we show that the log odds of an optimal policy recover how strongly its harness favors each action over the others, so a trained policy carries the preferences of its harness. Deployed return then equals the ceiling of the harness, the best return it offers, minus the mismatch of the policy, the return lost to its own suboptimal decisions. Training changes only the mismatch, while changing the harness moves both terms. Comparing harnesses with a fixed model therefore mixes what each harness offers with how well that model uses it. In MiniWoB++, post-training makes a model's preferences specific to its training harness. Reversing the order of the observation fields leaves every optimal decision unchanged yet lowers a trained model's success from 85.0% to 19.2%. Conversely, every new harness that a fixed-model comparison rejects but that raises the ceiling outperforms the original agent after further training under it, and ceiling and mismatch measured beforehand predict these gains. For coding-agent harnesses on Terminal-Bench Lite, ceiling minus mismatch estimated with a stronger model as a stand-in for the optimum reproduces the accuracy ranking, while either term alone does not. Code is available at https://github.com/2020-qqtcg/HRP.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 30 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. The controlled MiniWoB++ study is solid and the ceiling-minus-mismatch framing is genuinely useful; the coding-agent arm is a suggestive heuristic, not validation of the identity. This line is wrong
3 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 41 references say.
The paper says, at Section 6.3, Appendix F.7, Eqs. (31)-(32), Table 2
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
eC − fM is not an estimate of C(I) − M(θ,I) from Eq. (8). eC is the teacher's regularized return and fM is a score-error bound to the teacher, while the reference is the evaluated model's own category distribution p; the teacher is not shown to be optimal, and eC and fM are computed under different trajectory distributions (teacher trajectories for eC, target trajectories for fM).
more of objection 1
The authors acknowledge in F.7 that the teacher need not be optimal, but the abstract and Section 6.3 still present Table 2 as reproducing the accuracy ranking via ceiling minus mismatch. Under that acknowledged looseness, the n=5 perfect Spearman correlation is not evidence for Eq. (8); it could be determined by teacher choice, the 13-category taxonomy, the self-reference choice, or the three probe tasks. The paper should either supply formal conditions under which teacher substitution preserves the ordering, with uncertainty and sensitivity analysis, or clearly reposition this result as exploratory and qualify the abstract.
The paper says, at Section 6.3, Appendix F.8
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The exact MiniWoB++ comparison in F.8 does not independently validate the predictive claim, because setting each harness's reference to the evaluated model's own semantic distribution makes the evaluated policy's KL penalty zero and forces J to equal task success, so C − M = J holds by Eq.
more of objection 2
(8) by construction for every harness. Figure 11 is therefore a finite-graph and numerical consistency check rather than a test that ceiling minus mismatch orders harnesses by return. The genuine supporting evidence is the trained-checkpoint work in Sections 3-5, where the reference is a fixed frozen model independent of the evaluated policy; the paper should state this distinction explicitly.
The paper says, at Section 5.3, Table 10, Appendix F.4
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The claims that all seven rejected swaps that raise the ceiling beat the source pair and that T − r2 orders the eight gains with Spearman ρ = 0.95 rest on single training branches. The post-training updates draw tasks stochastically, and the task-level bootstrap intervals reported elsewhere do not cover training-seed variation. Since these empirical claims are phrased universally in the abstract and contributions, the paper should report repeated-seed intervals or add explicit qualifiers such as 'in these controlled runs'.
4 minor objections
The paper says, at Section 6.3, Table 2
Pith says: minor objection 1 of 4
The accuracy values overlap substantially (e.g., 64.1 ± 5.1 vs. 59.8 ± 7.8), and no uncertainty intervals are given for eC, fM, or their difference; at minimum, bootstrap intervals over probe tasks and trajectories should be reported before presenting the ranking as stable.
The paper says, at Appendix F.7, Section B.2
Pith says: minor objection 2 of 4
The 13-category action taxonomy merges actions with different arguments and effects, which is acknowledged in F.7 but is in tension with the semantic-action definition in Section B.2 and Eq. (1); the paper should label the taxonomy as a heuristic and report sensitivity to a finer or coarser category resolution.
show 2 more
The paper says, at Introduction and Section 5.3
Pith says: minor objection 3 of 4
The phrasing 'every new harness that a fixed-model comparison rejects but that raises the ceiling outperforms the original agent' should be tied to the nine swaps with T > 0 in the controlled MiniWoB++ setting; as written it overgeneralizes beyond the evidence.
The paper says, at General
Pith says: minor objection 4 of 4
Figure 6 and Figure 10 appear to be the same figure duplicated across the main text and appendix, and the reference list contains a typo in 'Huan ang Gao'; these should be cleaned up in revision.
What would settle it
On the MiniWoB++ H0-H5 setup, where the true optimal policy and exact C - M are known, emulate the Section 6.3 protocol with a deliberately suboptimal teacher (for example, the untrained base Qwen3.5-2B) in place of the optimum, using the same 13-style coarse action categories and the same estimators and sample sizes. Compare the resulting eC - fM ranking, with bootstrap intervals over tasks and trajectories, against the exact C - M ranking. If the teacher-based ranking fails to match, teacher substitution and category coarsening, not Eq. (8), drive Table 2; if it matches despite a suboptimal teacher, the concern is resolved.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 30 Oct 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Two things to know. First, the core identity J = C - M is the regularized performance gap, correctly stated and then actually tested on MiniWoB++: exact policy gradients, compiled finite graphs, a shared reference, held-out tasks, bootstrap intervals, and predictions recorded before adaptation. That controlled study is the strongest part of the paper, and it earns the main claims about readout drift and adaptation. Second, the coding-agent arm is much weaker than it looks: with a non-optimal teacher standing in for the optimum, eC - fM has no formal relation to C - M, and a five-harness ranking with no intervals on the estimates is not evidence for the framework's deployment story.
What is actually new: the target-shift / training-residual / readout-drift decomposition is a clean way to attribute score errors; the certified-invariant presentation study (reversing field order drops success from 85.0% to 19.2% while every optimal decision is unchanged) is a genuinely informative empirical result; and the adaptation threshold, plus the pre-recorded T - r2 predictions across harness swaps, is a nice use of the identity. The theory is mostly repackaged known material—softmax optimal policies and the O'Donoghue gap identity—but the packaging does real work for harness evaluation, and the regularized-MDP, DPO, and identifiability literature is cited properly.
Soft spots, in proportion. The coding-agent section is the main one. The stress-test note is right: eC is a teacher return and fM is a bound on score error to the teacher, not to the optimum. The authors openly say the teacher need not be optimal, but they still interpret the ranking as support for ceiling-minus-mismatch. It is at best a heuristic probe. Five harnesses, three probe tasks, thirteen coarse action categories, a proprietary teacher, and no uncertainty or sensitivity analysis make the perfect Spearman correlation much less informative than it appears. A second issue: the coding-agent estimates use the evaluated model's own distribution as the reference, so ceiling and mismatch are model-dependent rather than intrinsic harness properties. Again, acknowledged, but it limits what the table can support. Third, the framework assumes a deployed LM agent behaves like a policy optimized under fixed KL regularization; the MiniWoB++ runs support that for a 2B model, but the coding-agent results do not test it.
The math, where checked, checks out. The citation pattern is fine. The repository lacks a commit hash, which matters mainly for reproducing the coding-agent numbers.
Who this is for: anyone building or evaluating LLM agent harnesses, and people working on regularized-RL policy transfer. It deserves a serious referee. I would send it out, with the expectation that the coding-agent claims get trimmed or hardened before publication.
The strongest objection
Eq. (8) is exact for the semantic control problem, and the MiniWoB++ study verifies it under conditions where the optimum and reference are known. The load-bearing gap is the extension to deployed coding agents in Section 6.3 and Appendix F.7. There, the reference is the evaluated model's own category distribution p, the 'ceiling' eC is the teacher q's regularized return, and fM is the Proposition 6.1 score-error bound between p and q. If q is not optimal, eC is merely a lower bound on C, and fM is not the mismatch M(theta,I) of Eq. (8); eC - fM has no formal relation to C - M. The authors acknowledge that the teacher need not be optimal and that the 13 categories merge actions with different arguments and effects, but the unresolved point is that the resulting table is then not a test of the central identity. With five harnesses, three probe tasks, and no uncertainty intervals on eC or fM, the perfect Spearman correlation in Table 2 could be driven by teacher choice, category granularity, reference choice, or probe-task selection rather than by harness ceiling and mismatch.
The formal core of the paper is sound: Proposition 3.1, Proposition 5.1, and Eq. (8) are correct applications of regularized-control identities, and the controlled MiniWoB++ experiments directly support the readout-drift and adaptation predictions. The reader's weakest assumption identifies the same general region: the coding-agent extension assumes a teacher can stand in for the optimum and that coarse categories preserve the score geometry. My concern is narrower and more specific: in Section 6.3 the estimated object is not C - M at all. eC is the teacher's regularized return and fM is an upper bound on distance to the teacher, so neither term is tied to the true optimum; the exact identity J = C - M holds only for the true optimal policy. The paper acknowledges the teacher is not optimal and that categories merge actions, so this is not a hidden limitation, but the consequence is that the perfect ranking in Table 2 is not evidence for the central claim. The proposed test uses the known-optimum MiniWoB++ environment to validate the teacher-based protocol itself, which would settle whether the coding-agent result is a genuine harness property or an artifact of the estimation choices. Since the central theory and the controlled experiments remain intact, the reader's conditional verdict is appropriate; the coding-agent ranking should not be treated as settled until such a check is run.
The referee's report
The central identity is standard regularized performance-gap theory correctly applied, and the contribution lies in making harness evaluation a two-term quantity. If Eq. (8) and the controlled experiments hold, the paper gives a principled explanation for frozen-harness comparison failures and a measurable adaptation criterion. The MiniWoB++ study is unusually credible on its own terms: exact policy gradients over compiled finite graphs, a fixed reference independent of the evaluated checkpoint, held-out tasks, paired bootstrap intervals, and eight adaptation predictions recorded before adaptation. The readout-drift result (Table 1) is a clean demonstration that equivalent optimal decisions do not imply equivalent behavior. The coding-agent arm is considerably weaker and currently overclaimed: eC and fM are not estimates of C and M, so Table 2 does not test Eq. (8). With that section reframed or strengthened, the paper would be a valuable contribution.
Pith's reading of the paper (a machine's summary, not the authors')
Language-model agents are post-trained and deployed through a harness: the prompts, tools, and control logic that present a task and execute its outputs. This paper tries to establish that a trained policy is secretly a record of its harness: under KL-regularized control, the log odds of an optimal policy recover exactly the relative action scores the harness induces, so what a model learns under one harness does not automatically carry to another. The central identity is that deployed return equals the harness's ceiling, the best regularized return any policy can reach, minus the policy's mismatch, the return lost by departing from the optimal policy; training changes only the mismatch, while a harness change moves both terms. Two demonstrations follow: a presentation change that leaves every optimal decision unchanged still changes a trained model's most probable action at about half of the states and drops task success from 85.0% to 19.2%, and harnesses that a frozen-model comparison rejects can become the best choice after adaptation, with the gain predictable from ceiling and mismatch measured in advance. If the paper is right, harness evaluation and automated harness search must separate what a harness offers from how well a particular checkpoint uses it.
What carries the argument. The load-bearing object is the decision geometry of a harness: the centered optimal action-score field $\bar{F}^*_I = P^\perp F^*_I$, whose softmax is the optimal policy, together with the model's readout $\hat{F}_{\theta,I} = \tau P^\perp \log \Pi_{\theta,I}$, the action preferences the weights express through interface $I$. Three identities carry the argument. First, behavioral identification (Proposition 3.1): two interfaces have equivalent optimal decisions if and only if their centered score fields coincide, and an optimal checkpoint's log odds equal the score differences, so behavior pins down relative action preferences but not absolute value. Second, the regularized performance-gap identity (Proposition 5.1): the return lost by a deployed policy equals $\tau/(1-\gamma)$ times the occupancy-weighted KL divergence from the optimal policy, and this KL is a Bregman divergence $D_\Phi(F^*_I, \hat{F}_{\theta,I})$ of the log-partition function $\Phi(F) = \tau \log \sum_a e^{F_a/\tau}$, converting score errors into lost return that grows roughly quadratically in the summed error. Third, the adaptation threshold (Corollary 5.2): a harness revision that initially hurts the frozen checkpoint becomes beneficial after adaptation exactly when the remaining mismatch is below $T = \Delta C + M(\theta_0, I_1)$, the new ceiling minus the source pair's return. These results yield the score-error decomposition of Eq. (5) — deployed error equals target shift plus training residual plus readout drift — and the mismatch bound of Proposition 6.1 that the sampled-action estimates rely on.
Core claim. The paper's central claim is the ceiling-minus-mismatch identity, Eq. (8): for a checkpoint $\theta$ deployed under interface $I$, $J_I(\Pi_{\theta,I}) = C(I) - M(\theta,I)$, where $C(I)$ is the optimal regularized return of the interface's induced control problem and $M(\theta,I)$ is the occupancy-weighted KL divergence from the deployed policy to that interface's optimal policy. A direct corollary is that a weight update under a fixed harness changes only $M$, while a harness change moves both terms, so a frozen comparison of harnesses mixes what each harness offers with how well the current checkpoint uses it. The companion identification result, Proposition 3.1, states that an optimal policy's log odds are proportional to differences of the optimal action scores $F^*_I(s,a) = Q^*_I(s,a) + \tau \log \pi^{\mathrm{ref}}_I(a|s)$, so optimal behavior determines a harness's relative action preferences and nothing more. In the controlled study on MiniWoB++, a benchmark of web-interaction tasks, post-training moves a model's action scores toward its training harness's optimum and makes preferences harness-specific; reversing the order of observation fields, which provably leaves the optimal policy and optimal return unchanged, drops a trained model's task success from 85.0% to 19.2% by moving only the readout. For coding-agent harnesses on Terminal-Bench Lite, a terminal coding benchmark, estimates of ceiling minus mismatch with a stronger teacher model standing in for the optimum reproduce the accuracy ranking of five harnesses exactly, while either term alone misorders them.
The premise it rests on. The entire decomposition rests on modeling a deployed language-model agent as a semantic policy that optimizes a fixed KL-regularized control objective, with the interface readout exactly given by the action-conditional token sums of Eq. (1); if real agents sit far from the regularized optimum, or if a coarse action taxonomy and a teacher model distort the stand-in for the optimal policy, then the identity and the harness rankings do not necessarily reflect achievable harness quality.
If the paper is right (Pith's inferences, not the authors')
Harness evaluations that score candidates with one frozen checkpoint are structurally biased: they penalize high-ceiling harnesses for mismatch that further training under the harness would remove.
Ceiling, mismatch, and the threshold $T = \Delta C + M(\theta_0, I_1)$ predict which rejected harness swaps pay off after adaptation; in the paper's rejected-swap study the predicted gain $T - r_2$ ordered the eight pre-recorded gains with Spearman $\rho = 0.95$, while the frozen return change ordered them with $\rho = 0.17$.
Presentation changes that provably keep the optimal policy fixed, such as field order, menu format, or action-code assignment, can still destroy a trained model's performance through readout drift alone, so interface changes need to be tested on the deployed checkpoint rather than on the control problem.
In domains with no solvable optimum, ceiling minus mismatch estimated from sampled actions with a stronger model in the optimum's place can order harnesses by accuracy where neither term alone can.
Mismatch estimated from finite action samples predicts the small return differences that decide adaptation, recovery and harness-ranking reversal, about ten times more accurately than direct return rollouts at the same query budget, so the identity is measurable in deployment and not only in simulation.
My extension: harness-search procedures that currently accept or reject candidates on frozen-model scores would, under this decomposition, rank candidates by ceiling gain minus predicted residual mismatch, using the corollary's threshold as a selection rule; the paper describes the diagnosis but not this specific procedure.
My extension: the field-reversal result suggests a cheap deployment-time robustness check — re-render the same semantic interface in several presentations and measure readout drift — that would flag presentation-fragile checkpoints before they ship; the paper does not propose this test itself.
My extension: the teacher-stand-in recipe, sample action distributions through the unchanged harness and compare with a stronger model on a small shared action taxonomy, is portable to other open-action benchmarks, but whether 13 coarse categories preserve the score geometry outside terminal coding tasks is untested.
My extension: the theory's dependence on the KL regularizer and on finite action sets is a live question — replacing the regularizer or moving to continuous action spaces would change the Bregman curvature and the quadratic mismatch bound, and the paper leaves that territory open.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
How Pith checks papers · The full audit of this paper · Review history (2 versions)
The paper
Abstract
Language model agents are post-trained and deployed through a harness, the prompts, tools, and control logic that present a task to the model and execute its outputs. Harnesses are often revised and judged by how an already trained model performs under them, which assumes that what the model learned under one harness carries over to the next. We ask what a policy learns from its harness and what survives when the harness changes. Under KL-regularized control, we show that the log odds of an optimal policy recover how strongly its harness favors each action over the others, so a trained policy carries the preferences of its harness. Deployed return then equals the ceiling of the harness, the best return it offers, minus the mismatch of the policy, the return lost to its own suboptimal decisions. Training changes only the mismatch, while changing the harness moves both terms. Comparing harnesses with a fixed model therefore mixes what each harness offers with how well that model uses it. In MiniWoB++, post-training makes a model's preferences specific to its training harness. Reversing the order of the observation fields leaves every optimal decision unchanged yet lowers a trained model's success from 85.0% to 19.2%. Conversely, every new harness that a fixed-model comparison rejects but that raises the ceiling outperforms the original agent after further training under it, and ceiling and mismatch measured beforehand predict these gains. For coding-agent harnesses on Terminal-Bench Lite, ceiling minus mismatch estimated with a stronger model as a stand-in for the optimum reproduces the accuracy ranking, while either term alone does not. Code is available at https://github.com/2020-qqtcg/HRP.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-28
The authors' main claim, as the classifier picked it out 0.39
“Comparing harnesses with a fixed model therefore mixes what each harness offers with how well that model uses it.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.42 · human participants 0.04 · controlled comparison 0.64 · randomized assignment 0.02 · states quantitative result 0.98 · compares with prior results 0.65 · mentions code or data availability 0.98
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Language model agents are post-trained and deployed through a harness, the prompts, tools, and control logic that present a task to the model and execute its outputs. | background 1.00 | 0.07 | 0.03 | asserted 1.00 |
| Harnesses are often revised and judged by how an already trained model performs under them, which assumes that what the model learned under one harness carries over to the next. | background 0.91 | 0.07 | 0.03 | asserted 0.93 |
| We ask what a policy learns from its harness and what survives when the harness changes. | gap 0.48 | 0.18 | 0.06 | unclear 0.89 |
| Under KL-regularized control, we show that the log odds of an optimal policy recover how strongly its harness favors each action over the others, so a trained policy carries the preferences of its harness. | result 0.90 | 0.89 | 0.41 | asserted 1.00 |
| Deployed return then equals the ceiling of the harness, the best return it offers, minus the mismatch of the policy, the return lost to its own suboptimal decisions. | result 0.81 | 0.76 | 0.25 | asserted 1.00 |
| Training changes only the mismatch, while changing the harness moves both terms. | interpretation 0.66 | 0.71 | 0.22 | asserted 1.00 |
| Comparing harnesses with a fixed model therefore mixes what each harness offers with how well that model uses it. | interpretation 0.90 | 0.36 | 0.22 | asserted 1.00 |
| In MiniWoB++, post-training makes a model's preferences specific to its training harness. | result 0.98 | 0.92 | 0.18 | asserted 1.00 |
| Reversing the order of the observation fields leaves every optimal decision unchanged yet lowers a trained model's success from 85.0% to 19.2%. | result 1.00 | 0.90 | 0.08 | asserted 1.00 |
| Conversely, every new harness that a fixed-model comparison rejects but that raises the ceiling outperforms the original agent after further training under it, and ceiling and mismatch measured beforehand predict these gains. | result 0.99 | 0.92 | 0.29 | asserted 1.00 |
| For coding-agent harnesses on Terminal-Bench Lite, ceiling minus mismatch estimated with a stronger model as a stand-in for the optimum reproduces the accuracy ranking, while either term alone does not. | result 1.00 | 0.92 | 0.12 | asserted 1.00 |
| Code is available at https://github.com/2020-qqtcg/HRP. | contribution 0.49 | 0.66 | 0.02 | unclear 0.75 |
What kind of paper the classifier reports this is: theoretical result 0.80 · evidence from observation or experiment 0.80
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-28: 1 request, 8,887 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (11)
Showing 10 of the paper's 11 figures. Figures are the authors', reproduced from the paper.
References (41)
- [1] Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards on Pith
- [2] Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume = on Pith
- [3] Deep learning , author=. 2016 , publisher= on Pith
- [4] Yang, John and Jimenez, Carlos E. and Wettig, Alexander and Lieret, Kilian and Yao, Shunyu and Narasimhan, Karthik and Press, Ofir , booktitle=. 2024 , url= on Pith
- [5] Sclar, Melanie and Choi, Yejin and Tsvetkov, Yulia and Suhr, Alane , booktitle=. Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How. 2024 , url= on Pith
- [6] Zhang, Chaoyun and Huang, He and Ni, Chiming and Mu, Jian and Qin, Si and He, Shilin and Wang, Lu and Yang, Fangkai and Zhao, Pu and Du, Chao and Li, Liqun and Kang, Yu and Jiang, Zhao and Zheng, Suzhen and Wang, Rujia and Qian, Jiaxu and Ma, Minghua and Lou, Jian-Guang and Li... on Pith
- [7] Mu, Jian and Zhang, Chaoyun and Ni, Chiming and Wang, Lu and Qiao, Bo and Mathur, Kartik and Wu, Qianhui and Xie, Yuhang and Ma, Xiaojun and Zhou, Mengyu and Qin, Si and Li, Liqun and Kang, Yu and Ma, Minghua and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei , journal=... on Pith
- [8] Direct Preference Optimization: Your Language Model Is Secretly a Reward Model , author=. Advances in Neural Information Processing Systems , year= on Pith
- [9] Rafailov, Rafael and Hejna, Joey and Park, Ryan and Finn, Chelsea , booktitle=. From. 2024 , url= on Pith
- [10] Geist, Matthieu and Scherrer, Bruno and Pietquin, Olivier , booktitle=. A Theory of Regularized. 2019 , volume= on Pith
- [11] O'Donoghue, Brendan , journal=. On the Connection between. 2022 , url= on Pith
- [12] Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review , author=. arXiv preprint arXiv:1805.00909 , year= on PithPith's reviewarXiv
- [13] Identifiability in Inverse Reinforcement Learning , author=. Advances in Neural Information Processing Systems , volume=. 2021 , url= on Pith
- [14] All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning , author=. arXiv preprint arXiv:2503.01067 , year= on PitharXiv
- [15] Automated Design of Agentic Systems , author=. International Conference on Learning Representations , year= on Pith
- [16] Zhang, Jiayi and Xiang, Jinyu and Yu, Zhaoyang and Teng, Fengwei and Chen, Xionghui and Chen, Jiaqi and Zhuge, Mingchen and Cheng, Xin and Hong, Sirui and Wang, Jinlin and Zheng, Bingnan and Liu, Bang and Luo, Yuyu and Wu, Chenglin , booktitle=. 2025 , url= on Pith
- [17] Hebbar, Prannay and Manawat, Yogendra and Verboomen, Samuel and Ivanova, Alesia and Palanimalai, Selvam and Bhatia, Kunal and Baskaran, Vignesh , journal =. 2026 , url = on Pith
- [18] Chen, Zhengyu and Xiao, Teng and Zhu, Huaisheng and Yuan, Yige and Zhang, Luan and Wang, Jingang , journal =. 2026 , url = on Pith
- [19] Kim, Haechan and Lee, Yoonho and Lee, Gisang and Finn, Chelsea and Lee, Kangwook , journal =. 2026 , url = on Pith
- [20] A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems , author =. 2025 , eprint = on Pith
show 21 more
- [21] A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence , author =. Transactions on Machine Learning Research , year = on Pith
- [22] Wölflein, Georg and Ferber, Dyke and Truhn, Daniel and Arandjelovic, Ognjen and Kather, Jakob Nikolas , editor =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month = jul, year =. doi:10.18653/v1/2025.acl-lon... on Pithdoi:10.18653/v1/2025.acl-long.1266
- [23] Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution , author =. 2023 , eprint = on Pith
- [24] A Self-Improving Coding Agent , author =. 2025 , eprint = on Pith
- [25] Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents , author =. The International Conference on Learning Representations (ICLR) , year = on Pith
- [26] Towards the Transferability of Rewards Recovered via Regularized Inverse Reinforcement Learning , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. 2024 , url = on Pith
- [27] Invariance in Policy Optimisation and Partial Identifiability in Reward Learning , author =. Proceedings of the 40th International Conference on Machine Learning (ICML) , year = on Pith
- [28] Andrew Y. Ng and Daishi Harada and Stuart J. Russell , title =. Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999) , pages =. 1999 , publisher = on Pith
- [29] Andrew Y. Ng and Stuart J. Russell , title =. Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000) , pages =. 2000 , publisher = on Pith
- [30] John Schulman and Xi Chen and Pieter Abbeel , journal =. Equivalence Between Policy Gradients and Soft. 2017 , url = on Pith
- [31] Zixuan Ke and Austin Xu and Yifei Ming and Xuan-Phi Nguyen and Caiming Xiong and Shafiq Joty , booktitle=. 2025 , url= on Pith
- [32] Reinforcement Learning with Deep Energy-Based Policies , author=. 2017 , eprint= on Pith
- [33] Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , author=. 2018 , eprint= on Pith
- [34] Russell, Stuart , booktitle =. 1998 , pages =. doi:10.1145/279943.279964 , url = on Pithdoi:10.1145/279943.279964
- [35] Quantifying Differences in Reward Functions , author=. 2021 , eprint= on Pith
- [36] Reinforcement Learning on Web Interfaces using Workflow-Guided Exploration , author=. International Conference on Learning Representations , year= on Pith
- [37] Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=. 2022 , url= on Pith
- [38] Learning by Distilling Context , author =. arXiv preprint arXiv:2209.15189 , year = on PithPith's reviewarXiv
- [39] Ye, Haoran and Lu, Yuxing and Dong, Haonan and Su, Zhaochen and Song, Guojie , journal =. 2026 , url = on Pith
- [40] Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails , author =. arXiv preprint arXiv:2609.09134 , year = on PithPith's reviewarXiv
- [41] Wei, Hongliang and Tu, Xiaobing and Wang, Yinggui and Liu, Zhengxi and Xue, Rongkun and Ren, Jinkui and Zhang, Xiantao and Zhao, Debin and Fan, Xiaopeng , journal =. 2026 , url = on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment