Pith.

Artificial Intelligence · arXiv:2610.21040 v1 · submitted 15 Oct 2026 · 114 references

Recursive Reward Improvement via Adversarial Meta-Prompt Evolution

Wei Dai, Qianchu Liu, Guanghui Qin, Xing Han, Tristan Naumann, Paul Pu Liang, Sheng Zhang

The paper says

“Recursive Self-Improvement (RSI) has gained wide attention for its ability to work with limited data. In a proposer-solver loop, a proposer model writes training tasks and a solver model learns to solve them through reinforcement learning.”

more of the abstract

Such loops have succeeded in mathematics and coding, where every answer can be checked automatically. Open-ended expert tasks, like clinical report generations, have no answer key, which makes simple question-answer generation not viable. Here the proposer must also write a rubric that lists the content an answer should include and the errors it should avoid. However, the unique challenge behind model-written rubric is the comprehensiveness of it. For naively generated rubric, solvers tend to exploit it, naming the expected topics without the substance behind them, which corrupts the learning process. To solve this problem, we introduce RRI, Recursive Reward Improvement, which uses a reward hacking adversary to supervise the reward. Before the solver trains on a generated task, an adversary writes an answer that games its rubric while being as useless as possible, the same model writes an honest answer without seeing the rubric, and up to two repair steps revise the rubric to make the honest answer win. The exploits found also rewrite how future tasks and rubrics are generated. On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model. In particular, the gains are largest on difficult and writing tasks. We show that the 27B model trained on the HealthBench Professional description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench without seeing a task from either. We further analyze the exploitable points of model-generated question-rubric pairs to facilitate future research on open-ended self-play.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 16 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.

Solid self-play method for open-ended tasks with a genuine contribution in adversarial rubric repair, though out-of-domain gains rest on the same judge used during training. This line is wrong

3 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 114 references say.

The paper says, at Section 4.3, Figure 4, Table 7

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The out-of-domain transfer results (PRBench Hard, ProfBench) are evaluated only with the GPT-5.5-Chat judge—the same judge used during training and in-domain validation. The paper's own analysis (Section I, Table 12) shows that 17% of exploit cases are 'incorrect pair' examples where the rubric-blind comparison does not prefer the better answer, indicating systematic judge unreliability.

more of objection 1

In-domain results were verified with the official GPT-5.4 judge (Table 5), but no such independent verification is provided for out-of-domain results. Since the paper presents out-of-domain transfer as evidence of generalization (abstract lines 11–12, Section 4.3), the claim is weakened unless the authors demonstrate that the observed gains cannot be explained by a judge bias that favors RRI-trained answer styles.

The paper says, at Section I, Table 12

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.74):

“A flagged exploit should be verified before it is repaired, and the loop needs a stopping rule for pairs of equally good answers.”

Pith says: Objection 2 of 3 · major

The paper acknowledges that the model-based judge 'cannot verify statutes, thresholds, or arithmetic' and that 17% of exploit cases are 'incorrect pair' (the honest answer is not actually better). These cases corrupt the repair signal. While the paper discusses limitations and suggests a stopping rule, it does not quantify how much of the reported gain might be artifactual due to these mislabeled pairs.

more of objection 2

The authors should analyze the sensitivity of the final HealthBench scores to filtering out or correcting the 17% incorrect-pair cases, and discuss the implications for the reliability of the reward signal.

The paper says, at Section 4.1, Appendix E

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The paper states that 'Due to compute constraints, we perform no hyperparameter tuning, and all baselines use the same hyperparameters' (Section 4.1). This is a significant limitation because RRI adds several components (adversary, repair, meta-optimizer) and hyperparameters (margin delta δ, memo objective κ, tolerance τ, number of repair rounds) that are not explored.

more of objection 3

Without sensitivity analysis, it is unclear whether the reported gains are robust or could be matched by tuning the Generation Only baseline differently. The authors should provide ablation experiments or discuss how hyperparameter choices could affect the comparison.

6 minor objections

The paper says, at Throughout

Pith says: minor objection 1 of 6

The paper uses 'adjusted score' and 'raw score' without always specifying which metric is being discussed. In several figures (e.g., Figure 2(a) vs 2(b)), the labels 'Score, %' are ambiguous. Please ensure that every figure and table clearly states whether it reports the length-adjusted score or the unadjusted rubric fraction.

The paper says, at Table 2, Appendix E

Pith says: minor objection 2 of 6

Table 2 lists 'Evolve' and 'Adversary' columns. The 'Adversary' column is set to '–' for Generation Only rows, but the meaning of 'yes' vs '–' is not explicitly defined in the caption. Add a footnote or clarify in the caption.

show 4 more

The paper says, at Section L.5, Appendix L

Pith says: minor objection 3 of 6

The repair process statistics (e.g., 'each accepted criterion removed on average 0.23 of the exploit’s normalized margin') are informative, but the standard deviation or distribution is not reported. Adding variance measures would help assess consistency.

The paper says, at Section M, Limitations

Pith says: minor objection 4 of 6

The limitations section correctly notes that 'passing an adversarial repair check does not guarantee that every possible exploit has been removed.' However, it does not discuss the computational cost of the adversarial loop. A brief note on the overhead (e.g., additional calls per task) would be useful for practitioners.

The paper says, at References

Pith says: minor objection 5 of 6

Several references appear to be placeholders with future years (e.g., 'Agrawal et al., 2026', 'Ding et al., 2026'). Please ensure that all cited works are either published or available as preprints, and that the years match the actual publication dates.

The paper says, at Figures 2 and 3

Pith says: minor objection 6 of 6

The figures use small font sizes for axis labels and legend entries. Consider enlarging them for readability, especially in a printed version.

What would settle it

Re-grade all out-of-domain answers from the 27B experiments (Figure 4: Qwen3.6-27B Base, Self Improve, RRI) using the official judges or grading protocols of PRBench Hard and ProfBench if they have specified judges, or alternatively with GPT-5.4 (the official HealthBench judge) as an independent evaluator. If the margins of RRI over Self Improve and over Base are preserved (within 2 points) under the independent judge, the concern is settled. If margins shrink substantially (more than 3 points), the out-of-domain generalization claim is unsupported.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 16 Oct 2026

  1. Reader holds up confidence moderate
  2. Skeptic partly agrees 42 s · 38,405 tokens read, 4,208 written
  3. Editor wrote the verdict above 16 s · 39,286 tokens read, 1,553 written
  4. Referee major revision: 3 major objections, 6 minor 46 s · 38,792 tokens read, 2,433 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Here is my read on this paper. The headline is that this is a well-executed piece of work that makes a genuine, incremental contribution to self-play for open-ended tasks where rewards must be generated alongside tasks. The core idea—test each rubric with an adversarial exploit, repair it by adding criteria that separate the hack from the honest answer, then re-attack—is clean and sensible. The meta-optimizer that updates task-generation prompts from exploit history is a nice addition. The experiments are extensive: multiple model sizes, ablation of components (generation only, co-evolve only, full RRI), analysis of 12,147 exploit cases across four benchmarks, and an all-self setting that removes the stronger teacher. The gains on HealthBench Professional (10–12 points over generation only) are consistent and largest on the hard and writing tasks, which is where the method should help most. The out-of-domain transfer to PRBench and ProfBench is also encouraging. The paper is honest about the limitations: the judge cannot verify statutes, thresholds, or arithmetic (17% incorrect pair cases), and the method does not claim to catch every exploit.

Now the soft spots. The main one is the stress-test concern: the out-of-domain transfer results (PRBench Hard, ProfBench) are evaluated only with the same GPT-5.5-Chat judge that served during training and in-domain validation. The paper does not provide an independent judge verification for these OOD scores, unlike the in-domain results which were re-graded with the official GPT-5.4 judge (Table 5). Given that the appendix shows 17% of exploit cases are incorrect pairs where the judge's preferred answer is not actually better, there is a real risk that the OOD gains partly reflect judge bias toward RRI-trained answer styles. The paper acknowledges this category but does not address it for the OOD eval. This does not sink the paper—the in-domain gains are robust—but it does weaken the generalizability claim. A second issue is that hyperparameters (margin, kappa, tolerance, repair rounds) are not tuned across baselines, which could disadvantage comparison methods. The paper mentions this as a compute constraint, which is fair but worth noting.

Overall, the paper is honest, the experiments are thorough, and the central claim—that adversarial rubric repair improves self-play for open-ended tasks—holds up. The code and data are released. The paper deserves a serious referee; it is publishable after the authors add independent judge verification for the OOD evaluations or discuss the limitation more explicitly.

The strongest objection

The paper reports out-of-domain generalization as a key contribution: RRI-27B improves over the base model by 6.6 points on PRBench Hard and 8.0 on ProfBench. However, these evaluations use the same GPT-5.5-Chat judge that served during training and for in-domain validation. The paper's own analysis (Section I, Figure 6) shows that 17% of exploit cases are 'incorrect pair' cases where the rubric-blind comparison prefers an answer that is not actually better, indicating systematic judge unreliability. The in-domain results were verified with the official GPT-5.4 judge (Table 5), but no such independent verification is performed for out-of-domain results. If the GPT-5.5-Chat judge has biases that favor RRI-trained answer styles (e.g., preferring answers with specific phrasing patterns that correlate with repair criteria), the out-of-domain improvements could be partially artifactual. This is load-bearing because the paper presents out-of-domain transfer as evidence that RRI produces generalizable improvements (abstract lines 11-12, Section 4.3). Without independent judge verification, the claim that RRI transfers to finance, legal, and scientific tasks is weaker than the in-domain claim.

The reader identified judge reliability as the weakest assumption. This concern refines that to a specific, testable gap: the out-of-domain transfer results, which are part of the paper's highlighted contributions, are not independently verified. However, the primary in-domain claim (HealthBench Professional improvements) is robustly supported: it is confirmed by re-grading with the official GPT-5.4 judge (Table 5), by consistent improvements across multiple settings (9B, 27B, all-self, no-retrieval), and by ablation studies (Co-evolve Only vs. full RRI). The out-of-domain results are an important but secondary support. If the concern lands, it would reduce confidence in the transfer claim but would not invalidate the central in-domain result. Therefore, the overall verdict of ACCEPT remains appropriate. The concrete test would settle whether the transfer claims need qualification.

The referee's report

If the results hold under independent verification, RRI makes a strong advance in scalable self-improvement for open-ended domains. The central idea—using an adversarial reward hacker to supervise a model-written reward—is novel and addresses a well-recognized failure mode of self-play loops. The large improvements on writing and difficult tasks, and the transfer to non-medical benchmarks, suggest the method teaches generalizable answer quality rather than rubric-specific patterns. The detailed exploit taxonomy provides actionable guidance for future work in self-play and reward design.

Pith's reading of the paper (a machine's summary, not the authors')

RRI addresses the challenge of model-written rubrics in self-improving loops for open-ended tasks. It introduces an adversary that games each rubric with a useless answer, then repairs the rubric by adding criteria that favor a substantive honest answer. This process repeats, and the exploits found also rewrite how future tasks and rubrics are generated. On HealthBench Professional, RRI improves over the Generation Only baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points when every auxiliary role is served by the frozen 9B model. The gains are largest on difficult and writing tasks, and the approach generalizes to out-of-domain benchmarks like PRBench Hard and ProfBench without ever seeing their tasks.

What carries the argument. The adversarial rubric hack-and-repair loop, where an adversary writes a hacking answer that scores high under the current rubric but is as useless as possible, an honest answer is written without seeing the rubric, and up to two repair steps add criteria that the honest answer meets and the exploit does not. Additionally, a meta-optimizer updates the proposer's generation prompts based on confirmed exploits, evolving the task distribution and rubric quality over training.

Core claim. The central discovery is that a self-improving loop can supervise its own reward by having an adversary probe each generated rubric for exploits, then repairing the rubric based on a rubric-blind comparison between the exploit and an honest answer. The repair step adds criteria that the honest answer meets and the exploit does not, forcing the solver to learn substantive content rather than superficial keyword coverage. The meta-optimizer further uses confirmed exploits to rewrite the instructions for future task and rubric generation, making the improvement cumulative. This yields significant in-domain gains and out-of-domain transfer, outperforming methods like RaR, EvoRubrics, and SibylSense in the same setting.

The premise it rests on. The paper assumes that the model-based judge (GPT-5.5-Chat or frozen 9B) consistently identifies the better answer when comparing the exploit and honest answer; if the judge has systematic biases (e.g., preferring longer answers or certain wording), the repair step may correct for the wrong signal.

If the paper is right (Pith's inferences, not the authors')

Self-improving systems can autonomously improve their reward functions without any human-labeled data or reference answers, making them applicable to domains where expert supervision is scarce.

The approach transfers to tasks outside the training domain (e.g., from medical to finance and legal) without seeing any of those tasks, suggesting broad applicability.

The largest gains are on difficult and writing tasks, where model-written rubrics are most vulnerable to exploitation, indicating that adversarial repair is especially valuable for complex open-ended tasks.

Iterative adversarial testing of rubrics, with re-attack after patching, adds further improvements over a single screening, supporting the principle of continued verification.

The 27B model trained only on a task description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench, demonstrating that the method induces transferable capabilities.

The 17% incorrect pair rate (where the honest answer is not actually better) reveals a fundamental limitation: the rubric-blind judge cannot verify hard facts like statutes, thresholds, or arithmetic, suggesting that future work needs more robust verification beyond the same model.

The exploit categories (missed negatives, ambiguous rubrics) provide a taxonomy of reward failures that could inform general principles for designing robust model-written rewards in self-play loops.

The method's success with a frozen 9B model serving all auxiliary roles suggests that the approach is practical even without a superhuman teacher, as long as the adversary is strong enough to find loopholes.

The meta-optimizer's use of confirmed exploits to rewrite generation prompts hints at a form of procedural knowledge transfer: the loop learns what kinds of criteria are rewindable and adjusts its generation accordingly.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Recursive Self-Improvement (RSI) has gained wide attention for its ability to work with limited data. In a proposer-solver loop, a proposer model writes training tasks and a solver model learns to solve them through reinforcement learning. Such loops have succeeded in mathematics and coding, where every answer can be checked automatically. Open-ended expert tasks, like clinical report generations, have no answer key, which makes simple question-answer generation not viable. Here the proposer must also write a rubric that lists the content an answer should include and the errors it should avoid. However, the unique challenge behind model-written rubric is the comprehensiveness of it. For naively generated rubric, solvers tend to exploit it, naming the expected topics without the substance behind them, which corrupts the learning process. To solve this problem, we introduce RRI, Recursive Reward Improvement, which uses a reward hacking adversary to supervise the reward. Before the solver trains on a generated task, an adversary writes an answer that games its rubric while being as useless as possible, the same model writes an honest answer without seeing the rubric, and up to two repair steps revise the rubric to make the honest answer win. The exploits found also rewrite how future tasks and rubrics are generated. On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model. In particular, the gains are largest on difficult and writing tasks. We show that the 27B model trained on the HealthBench Professional description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench without seeing a task from either. We further analyze the exploitable points of model-generated question-rubric pairs to facilitate future research on open-ended self-play.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-16

The authors' main claim, as the classifier picked it out 0.94

“On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.26 · human participants 0.08 · controlled comparison 0.96 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Recursive Self-Improvement (RSI) has gained wide attention for its ability to work with limited data. background 1.00 0.04 0.02 asserted 1.00
In a proposer-solver loop, a proposer model writes training tasks and a solver model learns to solve them through reinforcement learning. background 0.79 0.09 0.02 asserted 0.99
Such loops have succeeded in mathematics and coding, where every answer can be checked automatically. background 0.99 0.08 0.02 asserted 1.00
Open-ended expert tasks, like clinical report generations, have no answer key, which makes simple question-answer generation not viable. background 0.63 0.07 0.02 asserted 1.00
Here the proposer must also write a rubric that lists the content an answer should include and the errors it should avoid. background 0.83 0.09 0.03 asserted 1.00
However, the unique challenge behind model-written rubric is the comprehensiveness of it. gap 0.51 0.10 0.03 asserted 1.00
For naively generated rubric, solvers tend to exploit it, naming the expected topics without the substance behind them, which corrupts the learning process. background 0.69 0.34 0.03 asserted 0.98
To solve this problem, we introduce RRI, Recursive Reward Improvement, which uses a reward hacking adversary to supervise the reward. contribution 0.90 0.83 0.14 asserted 1.00
Before the solver trains on a generated task, an adversary writes an answer that games its rubric while being as useless as possible, the same model writes an honest answer without seeing the rubric, and up to two repair steps revise the rubric to make the honest answer win. method 1.00 0.88 0.08 asserted 0.98
The exploits found also rewrite how future tasks and rubrics are generated. method 0.76 0.73 0.09 asserted 1.00
On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model. result 1.00 0.96 0.38 asserted 1.00
In particular, the gains are largest on difficult and writing tasks. result 0.77 0.92 0.10 asserted 1.00
We show that the 27B model trained on the HealthBench Professional description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench without seeing a task from either. result 1.00 0.96 0.13 asserted 1.00
We further analyze the exploitable points of model-generated question-rubric pairs to facilitate future research on open-ended self-play. contribution 0.70 0.92 0.05 asserted 0.97

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.94

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-16: 1 request, 10,208 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. Overview of RRI. The proposer generates clinical tasks and weighted rubrics from medical references. An adversary probes each rubric with an exploit and an honest answer. The repair step keeps criteria that distinguish the better answer from the exploit, then tests the revised rubric with another attack. The solver learns through GRPO using rubric rewards. Training feedback and confirmed exploits guide updates to the generation prompts. on a reference outside the loop or on score spread, which shows that a criterion separates answers but not that it prefers the better one.…
Figure 2 of the paper
Figure 2. (a) HealthBench Professional adjusted scores with retrieval. Bars show the untrained, Generation Only, and RRI models at 9B and 27B with a GPT Teacher and at 9B with a Self Teacher. (b) HealthBench Professional raw scores with retrieval. Bars show the same models and settings without the length adjustment. 3.1 PROPOSER The proposer receives only a general description of the target task and no training data. It writes diverse clinician requests that double as retrieval queries, retrieves medical passages for each request from the reference database, and then writes the completed task…
Figure 3 of the paper
Figure 3. (a) Rubric-reward baselines and component ablation in the 9B GPT Teacher setting. Bars show RaR with synthetic tasks, EvoRubrics, and SibylSense, followed by Generation Only, Co-evolve Only, and full RRI with hack-then-patch rounds. (b) Adjusted scores by task category and difficulty in the 9B GPT Teacher setting. Bars compare Generation Only, Co-evolve Only, and RRI-9B. 18.9 19.3 18.4 3 4.5 2 6.5 3 2.5 2 7.1 5 2.0 2 0.1 2 0.4 19.9 3 9.2 3 4.9 3 8.4 2 6.2 5 7.4 2 5.5 2 6.7 2 4.0 4 2.5 3 9.7 3 8.1 2 9.3 6 3.0 PRBench All Finance Legal ProfBench All Chemistry PhD Physics PhD Finance…
Figure 4 of the paper
Figure 4. Zero-shot scores on PRBench Hard and ProfBench. Qwen3.6-27B is trained on the HealthBench Pro description and answers both benchmarks, which it never saw, through the web-search tool loop. PRBench Hard has 550 finance and legal tasks, and ProfBench has 40 report tasks, 10 per domain. Self Improve is our implementation of self-play training on model-written tasks (Huang et al., 2026a; Liu et al., 2025a), in which a proposer with a fixed prompt writes the tasks and rubrics and the solver is trained on them by reinforcement learning. RRI adds the adversary that tests and repairs each…
Figure 5 of the paper
Figure 5. Example of rubric repair on an atrial fibrillation task. The question provides the context for the original rubric, its exploit, and selected repairs. Signed weights reward content or penalize errors. The detailed case appears in Appendix K.1. PRBench Hard therefore comes from improving the reward. Appendix H applies the full recipe to the descriptions of other benchmarks. RRI also improves on unseen HealthBench Hard tasks. On HealthBench Hard, RRI-27B scores 0.246, against 0.188 for Generation Only and 0.143 for the base model, a gain of 5.8 points over Generation Only, as
Figure 6 of the paper
Figure 6. Categories of exploit cases found during training. Each case is a pair of solver answers to one generated task in which the answer the rubric scores highest loses a rubric-blind comparison, assigned to one category by an LLM auditor. Each panel covers the runs trained on the description of one benchmark, HealthBench Pro pools the 9B and 27B runs, and the last panel pools all cases. Appendix I defines the categories and gives the procedure. As shown in
Figure 7 of the paper
Figure 7. (a) HealthBench Professional adjusted scores without retrieval. Bars show the untrained, Generation Only, and RRI models with the 9B backbone and the frozen 9B judge. (b) HealthBench Professional raw scores without retrieval

Figures are the authors', reproduced from the paper.

References (114)
  1. [1] Qoq-med: Building multimodal clinical foundation models with domain-aware grpo training , author=. Advances in Neural Information Processing Systems , volume= on Pith
  2. [2] Synthesizing high-quality visual question answering from medical documents with generator-verifier LMMs , author=. International Conference on Learning Representations , volume= on Pith
  3. [3] MIMIC-IV, a freely accessible electronic health record dataset , author=. Scientific data , volume=. 2023 , publisher= on Pith
  4. [4] Large language models and synthetic health data: progress and prospects , author=. JAMIA open , volume=. 2024 , publisher= on Pith
  5. [5] RareBench: can LLMs serve as rare diseases specialists? , author=. Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining , pages= on Pith
  6. [6] Huatuogpt-o1, towards medical complex reasoning with llms , author=. arXiv preprint arXiv:2412.18925 , year= on PithPith's reviewarXiv
  7. [7] Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models , author=. IEEE transactions on medical imaging , year= on Pith
  8. [8] Medgemma technical report , author=. arXiv preprint arXiv:2507.05201 , year= on PithPith's reviewarXiv
  9. [9] Orca: Progressive learning from complex explanation traces of gpt-4 , author=. arXiv preprint arXiv:2306.02707 , year= on PithPith's reviewarXiv
  10. [10] Ultramedical: Building specialized generalists in biomedicine , author=. Advances in Neural Information Processing Systems , volume= on Pith
  11. [11] The false promise of imitating proprietary llms , author=. arXiv preprint arXiv:2305.15717 , year= on PithPith's reviewarXiv
  12. [12] Absolute zero: Reinforced self-play reasoning with zero data , author=. Advances in Neural Information Processing Systems , volume= on Pith
  13. [13] R-zero: Self-evolving reasoning llm from zero data , author=. International Conference on Learning Representations , volume= on Pith
  14. [14] Self-questioning language models , author=. arXiv preprint arXiv:2508.03682 , year= on PithPith's reviewarXiv
  15. [15] Spice: Self-play in corpus environments improves reasoning , author=. arXiv preprint arXiv:2510.24684 , year= on PitharXiv
  16. [16] The curse of recursion: Training on generated data makes models forget , author=. arXiv preprint arXiv:2305.17493 , year= on PithPith's reviewarXiv
  17. [17] Self-play only evolves when self-synthetic pipeline ensures learnable information gain , author=. arXiv preprint arXiv:2603.02218 , year= on PithPith's reviewarXiv
  18. [18] Retrieval-augmented generation for large language models: A survey , author=. arXiv preprint arXiv:2312.10997 , year= on PithPith's reviewarXiv
  19. [19] Star: Bootstrapping reasoning with reasoning , author=. Advances in Neural Information Processing Systems , volume= on Pith
  20. [20] Reinforced self-training (rest) for language modeling , author=. arXiv preprint arXiv:2308.08998 , year= on PithPith's reviewarXiv
show 80 more
  1. [21] Self-rewarding language models , author=. arXiv preprint arXiv:2401.10020 , year= on PithPith's reviewarXiv
  2. [22] Self-play fine-tuning converts weak language models to strong language models , author=. arXiv preprint arXiv:2401.01335 , year= on PithPith's reviewarXiv
  3. [23] Self-improvement in language models: The sharpening mechanism , author=. International Conference on Learning Representations , volume= on Pith
  4. [24] Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning , author=. International Conference on Learning Representations , volume= on Pith
  5. [25] Self-challenging language model agents , author=. Advances in Neural Information Processing Systems , volume= on Pith
  6. [26] Propose, solve, verify: Self-play through formal verification , author=. arXiv preprint arXiv:2512.18160 , year= on PitharXiv
  7. [27] Coral: Towards autonomous multi-agent evolution for open-ended discovery , author=. arXiv preprint arXiv:2604.01658 , year= on PithPith's reviewarXiv
  8. [28] G-Zero: Self-play for open-ended generation from zero data , author=. arXiv preprint arXiv:2605.09959 , year= on PithPith's reviewarXiv
  9. [29] Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluation , author=. arXiv preprint arXiv:2601.21464 , year= on PithPith's reviewarXiv
  10. [30] Textbooks are all you need , author=. arXiv preprint arXiv:2306.11644 , year= on PithPith's reviewarXiv
  11. [31] Phi-4 technical report , author=. arXiv preprint arXiv:2412.08905 , year= on PithPith's reviewarXiv
  12. [32] Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data , author=. arXiv preprint arXiv:2404.01413 , year= on PithPith's reviewarXiv
  13. [33] Beyond model collapse: Scaling up with synthesized data requires verification , author=. International Conference on Learning Representations , volume= on Pith
  14. [34] Genie: Achieving human parity in content-grounded datasets generation , author=. arXiv preprint arXiv:2401.14367 , year= on PithPith's reviewarXiv
  15. [35] Synthetic continued pretraining , author=. International Conference on Learning Representations , volume= on Pith
  16. [36] Large language models encode clinical knowledge , author=. Nature , volume=. 2023 , publisher= on Pith
  17. [37] Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning , author=. arXiv preprint arXiv:2506.07044 , year= on PithPith's reviewarXiv
  18. [38] Baichuan-m1: Pushing the medical capability of large language models , author=. arXiv preprint arXiv:2502.12671 , year= on PithPith's reviewarXiv
  19. [39] m1: Unleash the potential of test-time scaling for medical reasoning with large language models , author=. arXiv preprint arXiv:2504.00869 , year= on PitharXiv
  20. [40] Med-u1: Incentivizing unified medical reasoning in llms via large-scale reinforcement learning , author=. arXiv preprint arXiv:2506.12307 , year= on PithPith's reviewarXiv
  21. [41] Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs , author=. arXiv preprint arXiv:2504.00993 , year= on PithPith's reviewarXiv
  22. [42] Medagentsim: Self-evolving multi-agent simulations for realistic clinical interactions , author=. International Conference on Medical Image Computing and Computer-Assisted Intervention , pages=. 2025 , organization= on Pith
  23. [43] EvoMDT: a self-evolving multi-agent system for structured clinical decision-making in multi-cancer , author=. npj Digital Medicine , volume=. 2026 , publisher= on Pith
  24. [44] A synthetic data generation framework for scalable and resource-efficient medical AI assistants , author=. 2025 , publisher= on Pith
  25. [45] Healthbench: Evaluating large language models towards improved human health , author=. arXiv preprint arXiv:2505.08775 , year= on PithPith's reviewarXiv
  26. [46] Rubrics as rewards: Reinforcement learning beyond verifiable domains , author=. International Conference on Learning Representations , volume= on Pith
  27. [47] Checklists are better than reward models for aligning language models , author=. Advances in Neural Information Processing Systems , volume= on Pith
  28. [48] Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year= on PithPith's reviewarXiv
  29. [49] Understanding r1-zero-like training: A critical perspective , author=. arXiv preprint arXiv:2503.20783 , year= on PithPith's reviewarXiv
  30. [50] Concrete problems in AI safety , author=. arXiv preprint arXiv:1606.06565 , year= on PithPith's reviewarXiv
  31. [51] Defining and characterizing reward gaming , author=. Advances in neural information processing systems , volume= on Pith
  32. [52] Sycophancy to subterfuge: Investigating reward-tampering in large language models , author=. arXiv preprint arXiv:2406.10162 , year= on PithPith's reviewarXiv
  33. [53] Odin: Disentangled reward mitigates hacking in rlhf , author=. arXiv preprint arXiv:2402.07319 , year= on PithPith's reviewarXiv
  34. [54] Length-controlled alpacaeval: A simple way to debias automatic evaluators , author=. arXiv preprint arXiv:2404.04475 , year= on PithPith's reviewarXiv
  35. [55] Disentangling length from quality in direct preference optimization , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages= on Pith
  36. [56] Scaling laws for reward model overoptimization , author=. International conference on machine learning , pages=. 2023 , organization= on Pith
  37. [57] Dr tulu: Reinforcement learning with evolving rubrics for deep research , author=. arXiv preprint arXiv:2511.19399 , year= on PithPith's reviewarXiv
  38. [58] Large language models as optimizers , author=. International Conference on Learning Representations , volume= on Pith
  39. [59] Dspy: Compiling declarative language model calls into self-improving pipelines , author=. arXiv preprint arXiv:2310.03714 , year= on PithPith's reviewarXiv
  40. [60] Gepa: Reflective prompt evolution can outperform reinforcement learning , author=. International Conference on Learning Representations , volume= on Pith
  41. [61] Textgrad: Automatic" differentiation" via text , author=. arXiv preprint arXiv:2406.07496 , year= on PithPith's reviewarXiv
  42. [62] AI safety via debate , author=. arXiv preprint arXiv:1805.00899 , year= on PithPith's reviewarXiv
  43. [63] Rainbow teaming: Open-ended generation of diverse adversarial prompts , author=. Advances in Neural Information Processing Systems , volume= on Pith
  44. [64] Weak-to-strong generalization: Eliciting strong capabilities with weak supervision , author=. arXiv preprint arXiv:2312.09390 , year= on PithPith's reviewarXiv
  45. [65] Generative verifiers: Reward modeling as next-token prediction , author=. International Conference on Learning Representations , volume= on Pith
  46. [66] Self-instruct: Aligning language models with self-generated instructions , author=. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers) , pages= on Pith
  47. [67] rStar-Math: Small LLMs can master math reasoning with self-evolved deep thinking , author=. arXiv preprint arXiv:2501.04519 , year= on PithPith's reviewarXiv
  48. [68] A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence , author=. arXiv preprint arXiv:2507.21046 , year= on PithPith's reviewarXiv
  49. [69] Wizardlm: Empowering large language models to follow complex instructions , author=. arXiv e-prints , pages= on Pith
  50. [70] Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing , author=. International Conference on Learning Representations , volume= on Pith
  51. [71] Compute as teacher: Turning inference compute into reference-free supervision , author=. arXiv preprint arXiv:2509.14234 , year= on PithPith's reviewarXiv
  52. [72] Openrubrics: Towards scalable synthetic rubric generation for reward modeling and llm alignment , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  53. [73] Rm-r1: Reward modeling as reasoning , author=. International Conference on Learning Representations , volume= on Pith
  54. [74] Automatic prompt optimization with “gradient descent” and beam search , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages= on Pith
  55. [75] Connecting large language models with evolutionary algorithms yields powerful prompt optimizers , author=. International Conference on Learning Representations , volume= on Pith
  56. [76] Promptbreeder: Self-referential self-improvement via prompt evolution , author=. arXiv preprint arXiv:2309.16797 , year= on PithPith's reviewarXiv
  57. [77] Language models learn to mislead humans via rlhf , author=. International Conference on Learning Representations , volume= on Pith
  58. [78] Rlcd: Reinforcement learning from contrastive distillation for language model alignment , author=. arXiv preprint arXiv:2307.12950 , year= on PithPith's reviewarXiv
  59. [79] Rewardbench: Evaluating reward models for language modeling , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages= on Pith
  60. [80] Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume= on Pith
  61. [81] G-eval: NLG evaluation using gpt-4 with better human alignment , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages= on Pith
  62. [82] Debating with more persuasive llms leads to more truthful answers , author=. arXiv preprint arXiv:2402.06782 , year= on PithPith's reviewarXiv
  63. [83] EvoLM: In search of lost language model training dynamics , author=. arXiv preprint arXiv:2506.16029 , year= on PitharXiv
  64. [84] Echo chamber: Rl post-training amplifies behaviors learned in pretraining , author=. arXiv preprint arXiv:2504.07912 , year= on PithPith's reviewarXiv
  65. [85] Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? , author=. Advances in Neural Information Processing Systems , volume= on Pith
  66. [86] Toward expert-level medical question answering with large language models , author=. Nature medicine , volume=. 2025 , publisher= on Pith
  67. [87] Tulu 3: Pushing frontiers in open language model post-training , author=. arXiv preprint arXiv:2411.15124 , year= on PithPith's reviewarXiv
  68. [88] Hybridflow: A flexible and efficient rlhf framework , author=. Proceedings of the Twentieth European Conference on Computer Systems , pages= on Pith
  69. [89] Search-r1: Training llms to reason and leverage search engines with reinforcement learning , author=. arXiv preprint arXiv:2503.09516 , year= on PithPith's reviewarXiv
  70. [90] Webthinker: Empowering large reasoning models with deep research capability , author=. Advances in Neural Information Processing Systems , volume= on Pith
  71. [91] HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats , author=. arXiv preprint arXiv:2604.27470 , year= on PithPith's reviewarXiv
  72. [92] Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year= on PithPith's reviewarXiv
  73. [93] StatPearls , author=. 2024 , publisher= on Pith
  74. [94] DailyMed , author=. National Institutes of Health , year= on Pith
  75. [95] MEDLINEplus: building and maintaining the National Library of Medicine's consumer health Web service , author=. Bulletin of the Medical Library Association , volume= on Pith
  76. [96] ICD-10: international statistical classification of diseases and related health problems: tenth revision , author=. 2004 , publisher= on Pith
  77. [97] Climb: Data foundations for large scale multimodal clinical foundation models , author=. arXiv preprint arXiv:2503.07667 , year= on PithPith's reviewarXiv
  78. [98] Medrag: Enhancing retrieval-augmented generation with knowledge graph-elicited reasoning for healthcare copilot , author=. Proceedings of the ACM on Web Conference 2025 , pages= on Pith
  79. [99] Pubmedqa: A dataset for biomedical research question answering , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages= on Pith
  80. [100] Pmc-vqa: Visual instruction tuning for medical visual question answering , author=. arXiv preprint arXiv:2305.10415 , year= on PithPith's reviewarXiv

The first 100 of 114 references are listed here. All of them are in this paper's citation record.

Cited by 1 Pith paper

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment