Artificial Intelligence · arXiv:2611.02839 v1 · submitted 2 Nov 2026 · 32 references
CARE: Grounded Recursive Self-Improvement for Production Customer-Service Agents
The paper says
“Recursive self-improvement (RSI) in production agents involves transforming accumulated experience into reliable model updates while adapting to evolving business policies. We present CARE (Customer-service Agent Retrospective and Evolve), a grounded RSI framework that operationalizes this process in customer service.”
more of the abstract
Its goal is to replace a frontier proprietary incumbent with an in-house mid-sized model that continues learning after deployment. After a one-time cold start on governed production traces, the current model serves production traffic alongside the incumbent, and its own failures drive the next update. A 24/7 inspection process establishes failures through evidence-blinded, independently reviewed judgments. Attribution separates model errors from defects in Skills, tools, or the harness, routing component failures to repair. Model-attributable trajectories are truncated at the first erroneous decision and paired with verified repairs. On-policy self-distillation (OPSD) then updates the model using the same model as teacher, with the diagnosis and repair supplied as privileged information unavailable to the student. To support continued evolution as business requirements change, training mixes failures from different Skill/SOP versions, each judged against the policy in force when it occurred. Candidates run an executable simulation test compiled from the latest SOP and then frozen, followed by offline inspection of the resulting traces. Promotion requires a combined simulation-and-inspection score at least at parity with the deployed version, confirmation through an online comparison with the incumbent, and human approval.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 3 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A serious production RSI architecture with an honest limitation section; the headline Bad Case gain and the training signal share one uncalibrated judge, which is the load-bearing question for referees. This line is wrong
3 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 32 references say.
The paper says, at Sections 5.1, 7.2, 8.2 (Table 6), 9.4, Eq. (23)
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.90):
“Its calibration against multilingual human annotation remains to be reported.”
Pith says: Objection 1 of 3 · major
The headline result, the decline of the inspection Bad Case rate from 6.42% to 1.97% (below the incumbent's 4.60%), and the training labels of Section 6.1 are produced by the same evidence-blinded inspection pipeline. The authors acknowledge the exposure in Section 9.4: attribution accuracy is not reported, and calibration of the confirmed-bad rate against multilingual human annotation remains to be reported.
more of objection 1
The case/denominator separation between training and evaluation prevents label leakage but does not make the judge accurate or invariant. The judge is a frontier model, plausibly from the same family as the GPT-5.4 incumbent whose behavior is imitated in cold start, so correlated blind spots could lower judged Bad Cases without lowering true error rates; and the screener is recalibrated on the loop's own terminal states (Eq. 23), so the judge can drift across iterations even on a fixed case set. The independent human metric reaches only non-significant parity in the final two periods (p=0.85, p=0.47, Section 8.2), so it does not corroborate the 6.42-to-1.97 decline. Because this is the central quantitative claim, I ask for a concrete calibration: re-adjudicate a fixed sample of each version's traces on the 559-case test with multilingual human annotation and show that the judged decline tracks the human-judged decline; until then the gap to the incumbent is a model-assessed rate, not a calibrated error rate.
The paper says, at Sections 7.1 and 8.1, Table 6
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
Section 8.1 states that all versions and reference models are evaluated on the same frozen test of 559 cases, but Section 7.1 states that whenever the Skill/SOP changes, the scenario compiler automatically regenerates the simulation test from the latest version, and Section 7.1 warns that results on tests compiled from different Skill/SOP versions are not a controlled comparison.
more of objection 2
The paper does not state whether any Skill/SOP revision occurred between V0 (July 30) and V5 (October 15). If one did, either the 559-case test was recompiled, contradicting the same-frozen-test claim, or the cross-version comparison in Table 6 is not controlled. Please state explicitly whether the test remained identical across all six versions and how that is consistent with the automatic recompilation rule; if the test changed, restrict the claimed decline to versions that shared the same test and say which test each version used.
This review says the paper's mathematics is wrong. Pith did not check that against the paper's source.
The paper says, at Section 3.2 (Definition 1), Section 8.3, Eq. (3)
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The replacement criterion of Definition 1 is per-context: it requires Q(pi_theta,c) >= Q(pi_star,c) - delta and H(pi_theta,c) <= H(pi_star,c) for every c in C, with a free tolerance delta. Section 8.3 verifies the criterion with aggregate quantities: simulation accuracy on the 559-case test (94.10% vs 96.80%, a 2.7-point deficit), overall online satisfaction parity, and the inspection Bad Case rate mapped to the hard-violation term H even though that rate includes soft quality issues.
more of objection 3
As written, Eq. (3) is not instantiated: delta is never chosen, Q and H are not measured per context, and the aggregate quality proxy shows a deficit, so the claim that V5 satisfies the replacement criteria requires either an explicit delta with per-context measurements or a documented relaxation of Definition 1.
5 minor objections
The paper says, at Section 8.2, Eq. (46)
Pith says: minor objection 1 of 5
The statement that the combined score never decreases (V0 to V5 in Table 6) is a direct consequence of the promotion gate in Eq. (46), which admits a candidate only if S(theta') >= S(theta_k); labeling this as a property of the gate rather than an empirical finding would avoid a misleading reading.
The paper says, at Section 5.1, Eqs. (20) and (22)
Pith says: minor objection 2 of 5
The symbol B_t is overloaded: in Eq. (20) it denotes the admission capacity, while in Eq. (22) it denotes the set of confirmed Bad Cases; please rename one of the two.
show 3 more
The paper says, at Abstract and Section 8.1
Pith says: minor objection 3 of 5
The abstract's headline figure (inspection Bad Case rate from 6.42% after cold start to 1.97%) is measured on the frozen 559-case simulation test, not on production conversations; Section 8.1 clarifies this, but the abstract should state it explicitly so the decline is not read as a production-traffic error rate.
The paper says, at Section 8.2, Table 6
Pith says: minor objection 4 of 5
The Bad Case rates are proportions over 559 cases (6.42% is about 36 cases, 1.97% about 11), and the offline metrics are reported without confidence intervals, as acknowledged in Section 9.4; the V4-to-V5 drop of 2.14 points is roughly 23 to 11 cases and is plausibly within sampling noise, so reporting Wilson intervals or counts, as Table 10 does, would help readers judge which steps of the decline are meaningful.
The paper says, at Figure 7(a)
Pith says: minor objection 5 of 5
The vertical scale of the online-satisfaction panel is omitted, so the reader cannot verify the reported point differences or interval widths; please restore a labeled scale.
What would settle it
Re-adjudicate a fixed stratified sample of simulation traces from V0, V1, V4, and V5 with human expert raters using a frozen rubric, with model-version metadata removed, and compare human Bad Case rates and human attribution labels against the pipeline outputs. A sample of roughly 300 traces (about half the 559-case test) with each version's traces blinded and interleaved would be sufficient to detect a decline on the scale claimed: if the human-rated Bad Case rate still falls from the V0 level to the V5 level, the shared-judge concern is resolved; if the decline shrinks or disappears, the headline improvement is at least partly an artifact of judge bias or drift.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 3 Nov 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The one thing to know: this is the most complete production write-up of a bounded, human-on-the-loop RSI loop I have seen, and the headline offline gain rests on an inspection judge whose calibration is reported nowhere. The authors know this — Section 9.4 says attribution accuracy is not reported and the confirmed-bad rate's calibration against multilingual human annotation remains to be done.
What is actually new: not the learning core, which is OPSD with privileged information from cited prior work, but the full production architecture around it — evidence-blinded 24/7 inspection with a publication gate, causal attribution that routes non-model failures to component repair, first-error truncation with verified minimal counterfactual repairs, the teacher-only privileged boundary, and a promotion gate on a frozen SOP-compiled simulation test. Five real iterations in a seven-region deployment is a serious study, and the ablations are informative: on a few hundred repair tuples, SFT collapses and DPO collapses into immediate handoff, while OPSD holds at 94.1% simulation accuracy.
Where it is soft, in proportion. The stress-test concern is real and unresolved: training failures and the headline Bad Case rate are co-produced by the same external-model inspection procedure. Case/denominator separation prevents leakage but does not make the judge accurate or invariant, and the screener is recalibrated on its own terminal states (Eq. 23). The online satisfaction result reaches only non-significant parity in the final periods (p = 0.85, p = 0.47), so it does not independently confirm the 6.42% to 1.97% decline — though it is a genuine, if weak, check that the model did not degrade service while cutting cost 70.5%. Offline metrics have no confidence intervals; 11 confirmed bad cases out of 559 carries a wide binomial interval, so the gap over the incumbent's 4.60% is plausible but not as sharp as the headline implies. The ablation counts come from separately finalized review batches, which the authors disclose, so those results rank configurations rather than attribute causes. None of this is hidden; the paper's limitation section is unusually honest.
Who it is for: applied RSI researchers and anyone building production agent-improvement loops. It deserves a serious referee. The architecture is worth engaging with, and the honest limitation statement makes it a useful template for what an industrial RSI report should disclose. The main thing I would ask of a revision is a human-calibrated re-adjudication of the judge on a sample, plus intervals on the offline metrics.
The strongest objection
The central claim depends on the inspection pipeline being accurate and stable, because that pipeline supplies both the training labels and the headline metric. Confirmed failures and diagnoses (Sections 5 and 6) feed OPSD, and the offline inspection Bad Case rate in Section 7.2 is computed by the same procedure on simulation traces. The paper acknowledges this exposure: attribution is a model-assisted judgment whose errors propagate into training, and attribution accuracy is not reported (Section 9.4), and calibration of the confirmed-bad rate against multilingual human annotation remains to be reported. The screener is also recalibrated on terminal states (Eq. 23), so the judge can drift across iterations. Case/denominator separation between training and evaluation prevents leakage but does not make the judge accurate or invariant. Because CARE-31B is post-trained from incumbent GPT-5.4 traces and judged by an external frontier model, correlated blind spots between the judge and the GPT-family behavior being imitated could lower judged Bad Cases without lowering true error rates. The independent human satisfaction metric reaches only non-significant parity in the final periods (p=0.85 and p=0.47), so it does not by itself confirm the 6.42% to 1.97% decline. The concern is therefore not that the paper is silent about the risk, but that the central quantitative claim has not been shown to survive an independent, calibrated judgment.
The reader's conditional verdict already rests on this weakest assumption, and my reading of the paper converges on the same point. What would have to be true for the central claim is that the externally driven inspection and attribution pipeline is accurate enough, and stable enough, that the measured Bad Case decline reflects genuine model improvement. The paper works hard to support this: evidence blinding, independent review, fail-closed terminal states, frozen simulation tests, and a separate online satisfaction measure. Those controls reduce obvious leakage and single-judge gaming, but they do not establish accuracy or invariance of the judge itself. The most distinctive numerical result, the drop from 6.42% to 1.97%, is produced by the same kind of model that also generates the training labels, and the independent human metric is consistent with only a modest parity claim. So the load-bearing weakness is not an internal inconsistency or a missing proof; it is an unverified empirical precondition. The proposed human re-adjudication is the direct check: if human raters reproduce the decline on blinded traces from the same frozen test, the concern is settled; if not, the central claim weakens substantially. Since the reader's verdict is already CONDITIONAL and explicitly flags this assumption, I recommend no change to the verdict.
The referee's report
If the improvement claim survives independent calibration, this is one of the more credible demonstrations of bounded, human-on-the-loop RSI in production. The paper's strengths are the specificity of its grounding mechanisms: reviewed and evidence-blinded failure confirmation; executed business state as the arbiter of actions; a promotion gate requiring parity on a fixed evaluation; case/denominator separation between training failures and evaluation; and a limitation section (Section 9.4) that candidly discloses the uncalibrated judge, unmeasured simulator fidelity, harness/model confounding, and non-finalized ablation batches. The formal conditions (Eq. 18 evidence blinding, Eq. 37 oracle blindness, Definition 2, and the OPSD loss family with its ablations) are stated precisely enough to be checked and reused. The loop demonstrably converts the model's own production failures into stable updates on a small data increment, and the cost and latency advantages are measured on concurrent traffic.
Pith's reading of the paper (a machine's summary, not the authors')
The paper claims that a production customer-service agent can keep improving itself after deployment by turning its own confirmed failures into training data, provided every failure is certified by evidence that is independent of the model being improved. CARE runs this loop on a 31B in-house model: a 24/7 inspection pipeline flags failures from the model's own traffic, attribution decides whether the model or a component is at fault, and only model-attributable failures are cut at the first wrong decision, paired with a verified repair, and fed back through on-policy self-distillation. Over five iterations the inspection bad-case rate falls from 6.42% to 1.97%, below the frontier incumbent's 4.60%, and online user satisfaction reaches parity while reply latency and serving cost drop sharply. If this holds, it means recursive self-improvement can be made safe enough for a live business by grounding it in external review rather than in the model's own judgment.
What carries the argument. The load-bearing mechanism is attribution-guided on-policy self-distillation (OPSD) at the first erroneous decision. For each confirmed, model-attributable failure, the trajectory is truncated at the exact request where the model first went wrong; the student samples a fresh decision from that input, while a frozen copy of the same model reads the same prefixes plus privileged information $I = (d, a^-, a^+)$ (diagnosis, erroneous output, verified minimal repair), and the loss is $\ell_t = \max(u^K_t, 0)$ with $u^K_t = \sum_{v \in S^K_t} q_t(v) \log(q_t(v)/p_{\theta,t}(v))$, a forward KL partial sum over the teacher's top-$K=32$ tokens, floored at zero per token and backpropagated directly. The teacher's only advantage is the privileged hindsight, which never reaches the student, so the student is corrected on what it would actually do now at the failing decision rather than on a fixed target text. Around this sit two supporting mechanisms: the evidence-blinded inspection pipeline (screening, deep analysis with a claim ledger, independent review, and a fail-closed publication gate, all driven by an external frontier model on frozen evidence without source metadata) and the executable simulation test compiled from the latest Skill/SOP and frozen, whose traces are inspected offline to produce the headline bad-case metric.
Core claim. On its own terms, the paper establishes that bounded recursive self-improvement works in production customer service when the loop is grounded by three independent checks: executed business state, reviewed evidence, and causal attribution. After a one-time cold start on governed traces of the incumbent GPT-5.4, the in-house CARE-31B model serves real traffic alongside the incumbent, and its own failures drive each update. A frozen, evidence-blinded inspection pipeline confirms failures, attribution routes non-model causes to component repair, and model-attributable failures become verified repair tuples at the first erroneous decision. Training uses on-policy self-distillation with the same model as teacher: the student samples its own decision where it failed, and a frozen copy of itself, given the diagnosis and repair as privileged information, defines the token-level target. After five iterations the confirmed-bad rate drops from 6.42% to 1.97% (less than half the incumbent's 4.60%), simulation accuracy rises from 93.0% to 94.1%, online satisfaction moves from 4.6 points below the incumbent to 1.7 points above, median reply latency falls 27.7%, and replacing the incumbent cuts serving cost per order by 70.5%. The ablations support the paper's central methodological claim: on a small increment of repairs, supervised fine-tuning collapses during training and preference optimization collapses into immediate handoff (15.1% simulated pass rate), while self-distillation trained on the same 280 repair tuples stays at 94.1%.
The premise it rests on. Everything the model learns, and every error-rate figure in the paper, depends on an external frontier model's inspection and attribution judgments, and the paper assumes those judgments are accurate enough and sufficiently independent of the in-house model's own blind spots, an accuracy that the authors explicitly state is not reported.
If the paper is right (Pith's inferences, not the authors')
A mid-sized in-house model can replace a frontier proprietary incumbent on quality, latency, and cost after five bounded self-improvement iterations, satisfying a formal replacement criterion of quality within tolerance, no more hard violations, and lower serving cost.
After the incumbent is retired, improvement does not stop, because the deployed model's own production failures supply both the training experience and the teacher, so fresh data no longer depends on the incumbent's traces.
Component failures (Skills, tools, harness, judge, infrastructure) are routed to component repair and kept out of the weights, so the model is not trained to work around broken parts of the system.
Small increments of repair data are usable only when the training target is the model's own distribution; fixed-text objectives (SFT, DPO) destabilize the model at this data scale, which is why self-distillation is the learning signal the loop depends on.
Business-policy revisions can be absorbed between model updates by mixing failures judged against the SOP in force when they occurred, so the model learns to follow whatever policy it is given rather than memorizing one version.
The authors state that attribution is a model-assisted judgment whose accuracy is not reported, so an unmeasured share of the headline decline could reflect the external judge's behavior; a natural check they do not run is to re-score V0's failure corpus with the final iteration's judge and attribution settings to see how much of the 6.42% to 1.97% drop survives a changed evaluator.
The same machinery transfers to other high-stakes service domains with an executable state model, such as banking or insurance claims, where a simulator can certify refunds, cancellations, and status transitions; the hard requirement is a state-transition function that is independent of the model's descriptions of its own actions.
Using the in-house model itself as the repair proposer, under the same independent verifier, would test whether the external frontier proposer is replaceable, which the paper names as its next step toward a more closed loop.
The paper argues, rather than measures, that mixing several SOP versions teaches policy-following; compiling the frozen simulation test separately for each SOP version, as the paper itself proposes, would turn that argument into a measurement.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Recursive self-improvement (RSI) in production agents involves transforming accumulated experience into reliable model updates while adapting to evolving business policies. We present CARE (Customer-service Agent Retrospective and Evolve), a grounded RSI framework that operationalizes this process in customer service. Its goal is to replace a frontier proprietary incumbent with an in-house mid-sized model that continues learning after deployment. After a one-time cold start on governed production traces, the current model serves production traffic alongside the incumbent, and its own failures drive the next update. A 24/7 inspection process establishes failures through evidence-blinded, independently reviewed judgments. Attribution separates model errors from defects in Skills, tools, or the harness, routing component failures to repair. Model-attributable trajectories are truncated at the first erroneous decision and paired with verified repairs. On-policy self-distillation (OPSD) then updates the model using the same model as teacher, with the diagnosis and repair supplied as privileged information unavailable to the student. To support continued evolution as business requirements change, training mixes failures from different Skill/SOP versions, each judged against the policy in force when it occurred. Candidates run an executable simulation test compiled from the latest SOP and then frozen, followed by offline inspection of the resulting traces. Promotion requires a combined simulation-and-inspection score at least at parity with the deployed version, confirmation through an online comparison with the incumbent, and human approval.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-03
The authors' main claim, as the classifier picked it out 0.60
“We present CARE (Customer-service Agent Retrospective and Evolve), a grounded RSI framework that operationalizes this process in customer service.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.17 · human participants 0.52 · controlled comparison 0.79 · randomized assignment 0.02 · states quantitative result 0.07 · compares with prior results 0.41 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Recursive self-improvement (RSI) in production agents involves transforming accumulated experience into reliable model updates while adapting to evolving business policies. | background 1.00 | 0.06 | 0.06 | asserted 1.00 |
| We present CARE (Customer-service Agent Retrospective and Evolve), a grounded RSI framework that operationalizes this process in customer service. | contribution 1.00 | 0.84 | 0.09 | asserted 1.00 |
| Its goal is to replace a frontier proprietary incumbent with an in-house mid-sized model that continues learning after deployment. | contribution 0.77 | 0.22 | 0.13 | asserted 0.99 |
| After a one-time cold start on governed production traces, the current model serves production traffic alongside the incumbent, and its own failures drive the next update. | method 0.98 | 0.48 | 0.06 | asserted 1.00 |
| A 24/7 inspection process establishes failures through evidence-blinded, independently reviewed judgments. | method 1.00 | 0.73 | 0.03 | asserted 1.00 |
| Attribution separates model errors from defects in Skills, tools, or the harness, routing component failures to repair. | method 1.00 | 0.68 | 0.03 | asserted 1.00 |
| Model-attributable trajectories are truncated at the first erroneous decision and paired with verified repairs. | method 1.00 | 0.76 | 0.04 | asserted 1.00 |
| On-policy self-distillation (OPSD) then updates the model using the same model as teacher, with the diagnosis and repair supplied as privileged information unavailable to the student. | method 1.00 | 0.73 | 0.06 | asserted 1.00 |
| To support continued evolution as business requirements change, training mixes failures from different Skill/SOP versions, each judged against the policy in force when it occurred. | method 0.98 | 0.58 | 0.04 | asserted 0.99 |
| Candidates run an executable simulation test compiled from the latest SOP and then frozen, followed by offline inspection of the resulting traces. | method 1.00 | 0.73 | 0.03 | asserted 1.00 |
| Promotion requires a combined simulation-and-inspection score at least at parity with the deployed version, confirmation through an online comparison with the incumbent, and human approval. | method 0.99 | 0.60 | 0.10 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.52
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-03: 1 request, 8,134 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (7)
Figures are the authors', reproduced from the paper.
References (32)
- [1] Irving John Good. Speculations concerning the first ultraintelligent machine. InAdvances in Computers, volume 6, pages 31–88. Academic Press, 1965 on Pith
- [2] Jürgen Schmidhuber. Gödel machines: Fully self-referential optimal universal self-improvers. In Ben Goertzel and Cassio Pennachin, editors,Artificial General Intelligence, pages 199–226. Springer, 2007 on Pith
- [3] Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, Hongru Wang, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, Qihan Ren, Cheng Qian, Zhenhailong Wan... on PithPith's reviewarXiv
- [4] Zhe Ren, Yimeng Chen, Dandan Guo, Guowei Rong, Tonghui Li, R. B. Xiong, Qingfeng Lan, Wenyi Wang, Li Nanbo, Yibo Yang, Mingchen Zhuge, and Jürgen Schmidhuber. Self-improvements in modern agentic systems: A survey.arXiv preprint arXiv:2607.13104, 2026. URLhttps://arxiv.org/abs/... on PithPith's reviewarXiv
- [5] Mingguang Chen, Licheng Wang, and Bo Qu. Recursive self-improvement in AI: From bounded self-refinement to autonomous research loops.arXiv preprint arXiv:2607.07663, 2026. URL https://arxiv.org/abs/2607. 07663 on PithPith's reviewarXiv
- [6] Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. AI models collapse when trained on recursively generated data.Nature, 631:755–759, 2024 on Pith
- [7] Vansh Wahi. Llm-as-a-judge is not an oracle: Why self-improving agents need deterministic guardrails.arXiv preprint arXiv:2609.02246, 2026. URLhttps://arxiv.org/abs/2609.02246 on PithPith's reviewarXiv
- [8] Qinyuan Ye, Yu Li, Yada Pruksachatkun, Jiaxin Zhang, and Chien-Sheng Wu. On the fragility of self-improving agents: Variance, task order, and underspecification.arXiv preprint arXiv:2608.18066, 2026. URL https: //arxiv.org/abs/2608.18066 on PithPith's reviewarXiv
- [9] RSI Workshop Organizers. ICLR 2026 workshop on AI with recursive self-improvement. https://iclr.cc/ virtual/2026/workshop/10000796, 2026 on Pith
- [10] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. STaR: Bootstrapping reasoning with reasoning. arXiv preprint arXiv:2203.14465, 2022. URLhttps://arxiv.org/abs/2203.14465 on PithPith's reviewarXiv
- [11] Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models.arXiv preprint arXiv:2401.01335, 2024. URL https://arxiv. org/abs/2401.01335 on PithPith's reviewarXiv
- [12] Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language models.arXiv preprint arXiv:2401.10020, 2024. URL https://arxiv.org/ abs/2401.10020 on PithPith's reviewarXiv
- [13] Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.arXiv preprint arXiv:2303.11366, 2023. URL https://arxiv.org/abs/2303.11366 on PithPith's reviewarXiv
- [15] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin godel machine: Open-ended evolution of self-improving agents.arXiv preprint arXiv:2505.22954, 2025. URL https://arxiv.org/abs/ 2505.22954 on PithPith's reviewarXiv
- [16] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. InProceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), pages 627–635, 2011 on Pith
- [17] Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes.arXiv preprint arXiv:2306.13649, 2023. URLhttps://arxiv.org/abs/2306.13649 on PithPith's reviewarXiv
- [18] Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan-ang Gao, Wenkai Yang, Zhiyuan Liu, and Ning Ding. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe.arXiv preprint arXiv:2604.13016, 2026... on PithPith's reviewarXiv
- [19] Vladimir Vapnik and Akshay Vashist. A new learning paradigm: Learning using privileged information.Neural Networks, 22(5–6):544–557, 2009 on Pith
- [20] Emiliano Penaloza, Dheeraj Vattikonda, Nicolas Gontier, Alexandre Lacoste, Laurent Charlin, and Massimo Caccia. Privileged information distillation for language models.arXiv preprint arXiv:2602.04942, 2026. URL https://arxiv.org/abs/2602.04942 on PithPith's reviewarXiv
- [21] Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734, 2026. URL https://arxiv.org/abs/2601.18734 on PithPith's reviewarXiv
show 12 more
- [22] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains.arXiv preprint arXiv:2406.12045, 2024. URL https://arxiv.org/abs/ 2406.12045 on PithPith's reviewarXiv
- [23] Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. τ 2-Bench: Evaluating con- versational agents in a dual-control environment.arXiv preprint arXiv:2506.07982, 2025. URL https: //arxiv.org/abs/2506.07982 on PithPith's reviewarXiv
- [24] Xufang Luo, Yuge Zhang, Zhiyuan He, Zilong Wang, Siyun Zhao, Dongsheng Li, Luna K. Qiu, and Yuqing Yang. Agent Lightning: Train ANY AI agents with reinforcement learning.arXiv preprint arXiv:2508.03680, 2025. URLhttps://arxiv.org/abs/2508.03680 on PithPith's reviewarXiv
- [25] Zaid Khan, Elias Stengel-Eskin, Jaemin Cho, and Mohit Bansal. DataEnvGym: Data generation agents in teacher environments with student feedback.arXiv preprint arXiv:2410.06215, 2024. URL https://arxiv.org/abs/ 2410.06215 on PithPith's reviewarXiv
- [27] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model.arXiv preprint arXiv:2305.18290, 2023. URLhttps://arxiv.org/abs/2305.18290 on PithPith's reviewarXiv
- [29] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. URLhttps://arxiv.org... on PithPith's reviewarXiv
- [30] Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains.arXiv preprint arXiv:2507.17746, 2025. URL https://arxiv.org/abs/2507.17746 on PithPith's reviewarXiv
- [31] Rena Gao, Xuetong Wu, Tatsuki Kuribayashi, Mingrui Ye, Siya Qi, Carsten Roever, Yuanxing Liu, Zheng Yuan, and Jey Han Lau. Can LLMs simulate L2-English dialogue? an information-theoretic analysis of L1-Dependent biases. InProceedings of the 63rd Annual Meeting of the Associati... on Pithdoi:10.18653/v1/2025.acl-long.219
- [2017] URLhttps://arxiv.org/abs/1707.01495 on PithPith's reviewarXiv
- [2023] URLhttps://arxiv.org/abs/2305.16291 on PithPith's reviewarXiv
- [2024] URLhttps://arxiv.org/abs/2402.01364 on PithPith's reviewarXiv
- [2026] URLhttps://arxiv.org/abs/2606.26294. 33 TECHNICALREPORT A Cold-Start Details A.1 Evolution of the data recipe The data engine of Section 4 is the result of several revisions, listed in Table 11. Early recipes concatenated complete human conversations (D2) or inserted human tur... on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment