Artificial Intelligence · arXiv:2610.21040 v1 · submitted 15 Oct 2026 · 114 references
Recursive Reward Improvement via Adversarial Meta-Prompt Evolution
The paper says
“Recursive Self-Improvement (RSI) has gained wide attention for its ability to work with limited data. In a proposer-solver loop, a proposer model writes training tasks and a solver model learns to solve them through reinforcement learning.”
more of the abstract
Such loops have succeeded in mathematics and coding, where every answer can be checked automatically. Open-ended expert tasks, like clinical report generations, have no answer key, which makes simple question-answer generation not viable. Here the proposer must also write a rubric that lists the content an answer should include and the errors it should avoid. However, the unique challenge behind model-written rubric is the comprehensiveness of it. For naively generated rubric, solvers tend to exploit it, naming the expected topics without the substance behind them, which corrupts the learning process. To solve this problem, we introduce RRI, Recursive Reward Improvement, which uses a reward hacking adversary to supervise the reward. Before the solver trains on a generated task, an adversary writes an answer that games its rubric while being as useless as possible, the same model writes an honest answer without seeing the rubric, and up to two repair steps revise the rubric to make the honest answer win. The exploits found also rewrite how future tasks and rubrics are generated. On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model. In particular, the gains are largest on difficult and writing tasks. We show that the 27B model trained on the HealthBench Professional description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench without seeing a task from either. We further analyze the exploitable points of model-generated question-rubric pairs to facilitate future research on open-ended self-play.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 16 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.
Solid self-play method for open-ended tasks with a genuine contribution in adversarial rubric repair, though out-of-domain gains rest on the same judge used during training. This line is wrong
3 major objections · 6 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 114 references say.
The paper says, at Section 4.3, Figure 4, Table 7
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The out-of-domain transfer results (PRBench Hard, ProfBench) are evaluated only with the GPT-5.5-Chat judge—the same judge used during training and in-domain validation. The paper's own analysis (Section I, Table 12) shows that 17% of exploit cases are 'incorrect pair' examples where the rubric-blind comparison does not prefer the better answer, indicating systematic judge unreliability.
more of objection 1
In-domain results were verified with the official GPT-5.4 judge (Table 5), but no such independent verification is provided for out-of-domain results. Since the paper presents out-of-domain transfer as evidence of generalization (abstract lines 11–12, Section 4.3), the claim is weakened unless the authors demonstrate that the observed gains cannot be explained by a judge bias that favors RRI-trained answer styles.
The paper says, at Section I, Table 12
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.74):
“A flagged exploit should be verified before it is repaired, and the loop needs a stopping rule for pairs of equally good answers.”
Pith says: Objection 2 of 3 · major
The paper acknowledges that the model-based judge 'cannot verify statutes, thresholds, or arithmetic' and that 17% of exploit cases are 'incorrect pair' (the honest answer is not actually better). These cases corrupt the repair signal. While the paper discusses limitations and suggests a stopping rule, it does not quantify how much of the reported gain might be artifactual due to these mislabeled pairs.
more of objection 2
The authors should analyze the sensitivity of the final HealthBench scores to filtering out or correcting the 17% incorrect-pair cases, and discuss the implications for the reliability of the reward signal.
The paper says, at Section 4.1, Appendix E
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The paper states that 'Due to compute constraints, we perform no hyperparameter tuning, and all baselines use the same hyperparameters' (Section 4.1). This is a significant limitation because RRI adds several components (adversary, repair, meta-optimizer) and hyperparameters (margin delta δ, memo objective κ, tolerance τ, number of repair rounds) that are not explored.
more of objection 3
Without sensitivity analysis, it is unclear whether the reported gains are robust or could be matched by tuning the Generation Only baseline differently. The authors should provide ablation experiments or discuss how hyperparameter choices could affect the comparison.
6 minor objections
The paper says, at Throughout
Pith says: minor objection 1 of 6
The paper uses 'adjusted score' and 'raw score' without always specifying which metric is being discussed. In several figures (e.g., Figure 2(a) vs 2(b)), the labels 'Score, %' are ambiguous. Please ensure that every figure and table clearly states whether it reports the length-adjusted score or the unadjusted rubric fraction.
The paper says, at Table 2, Appendix E
Pith says: minor objection 2 of 6
Table 2 lists 'Evolve' and 'Adversary' columns. The 'Adversary' column is set to '–' for Generation Only rows, but the meaning of 'yes' vs '–' is not explicitly defined in the caption. Add a footnote or clarify in the caption.
show 4 more
The paper says, at Section L.5, Appendix L
Pith says: minor objection 3 of 6
The repair process statistics (e.g., 'each accepted criterion removed on average 0.23 of the exploit’s normalized margin') are informative, but the standard deviation or distribution is not reported. Adding variance measures would help assess consistency.
The paper says, at Section M, Limitations
Pith says: minor objection 4 of 6
The limitations section correctly notes that 'passing an adversarial repair check does not guarantee that every possible exploit has been removed.' However, it does not discuss the computational cost of the adversarial loop. A brief note on the overhead (e.g., additional calls per task) would be useful for practitioners.
The paper says, at References
Pith says: minor objection 5 of 6
Several references appear to be placeholders with future years (e.g., 'Agrawal et al., 2026', 'Ding et al., 2026'). Please ensure that all cited works are either published or available as preprints, and that the years match the actual publication dates.
The paper says, at Figures 2 and 3
Pith says: minor objection 6 of 6
The figures use small font sizes for axis labels and legend entries. Consider enlarging them for readability, especially in a printed version.
What would settle it
Re-grade all out-of-domain answers from the 27B experiments (Figure 4: Qwen3.6-27B Base, Self Improve, RRI) using the official judges or grading protocols of PRBench Hard and ProfBench if they have specified judges, or alternatively with GPT-5.4 (the official HealthBench judge) as an independent evaluator. If the margins of RRI over Self Improve and over Base are preserved (within 2 points) under the independent judge, the concern is settled. If margins shrink substantially (more than 3 points), the out-of-domain generalization claim is unsupported.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 16 Oct 2026
- Reader holds up
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 6 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Here is my read on this paper. The headline is that this is a well-executed piece of work that makes a genuine, incremental contribution to self-play for open-ended tasks where rewards must be generated alongside tasks. The core idea—test each rubric with an adversarial exploit, repair it by adding criteria that separate the hack from the honest answer, then re-attack—is clean and sensible. The meta-optimizer that updates task-generation prompts from exploit history is a nice addition. The experiments are extensive: multiple model sizes, ablation of components (generation only, co-evolve only, full RRI), analysis of 12,147 exploit cases across four benchmarks, and an all-self setting that removes the stronger teacher. The gains on HealthBench Professional (10–12 points over generation only) are consistent and largest on the hard and writing tasks, which is where the method should help most. The out-of-domain transfer to PRBench and ProfBench is also encouraging. The paper is honest about the limitations: the judge cannot verify statutes, thresholds, or arithmetic (17% incorrect pair cases), and the method does not claim to catch every exploit.
Now the soft spots. The main one is the stress-test concern: the out-of-domain transfer results (PRBench Hard, ProfBench) are evaluated only with the same GPT-5.5-Chat judge that served during training and in-domain validation. The paper does not provide an independent judge verification for these OOD scores, unlike the in-domain results which were re-graded with the official GPT-5.4 judge (Table 5). Given that the appendix shows 17% of exploit cases are incorrect pairs where the judge's preferred answer is not actually better, there is a real risk that the OOD gains partly reflect judge bias toward RRI-trained answer styles. The paper acknowledges this category but does not address it for the OOD eval. This does not sink the paper—the in-domain gains are robust—but it does weaken the generalizability claim. A second issue is that hyperparameters (margin, kappa, tolerance, repair rounds) are not tuned across baselines, which could disadvantage comparison methods. The paper mentions this as a compute constraint, which is fair but worth noting.
Overall, the paper is honest, the experiments are thorough, and the central claim—that adversarial rubric repair improves self-play for open-ended tasks—holds up. The code and data are released. The paper deserves a serious referee; it is publishable after the authors add independent judge verification for the OOD evaluations or discuss the limitation more explicitly.
The strongest objection
The paper reports out-of-domain generalization as a key contribution: RRI-27B improves over the base model by 6.6 points on PRBench Hard and 8.0 on ProfBench. However, these evaluations use the same GPT-5.5-Chat judge that served during training and for in-domain validation. The paper's own analysis (Section I, Figure 6) shows that 17% of exploit cases are 'incorrect pair' cases where the rubric-blind comparison prefers an answer that is not actually better, indicating systematic judge unreliability. The in-domain results were verified with the official GPT-5.4 judge (Table 5), but no such independent verification is performed for out-of-domain results. If the GPT-5.5-Chat judge has biases that favor RRI-trained answer styles (e.g., preferring answers with specific phrasing patterns that correlate with repair criteria), the out-of-domain improvements could be partially artifactual. This is load-bearing because the paper presents out-of-domain transfer as evidence that RRI produces generalizable improvements (abstract lines 11-12, Section 4.3). Without independent judge verification, the claim that RRI transfers to finance, legal, and scientific tasks is weaker than the in-domain claim.
The reader identified judge reliability as the weakest assumption. This concern refines that to a specific, testable gap: the out-of-domain transfer results, which are part of the paper's highlighted contributions, are not independently verified. However, the primary in-domain claim (HealthBench Professional improvements) is robustly supported: it is confirmed by re-grading with the official GPT-5.4 judge (Table 5), by consistent improvements across multiple settings (9B, 27B, all-self, no-retrieval), and by ablation studies (Co-evolve Only vs. full RRI). The out-of-domain results are an important but secondary support. If the concern lands, it would reduce confidence in the transfer claim but would not invalidate the central in-domain result. Therefore, the overall verdict of ACCEPT remains appropriate. The concrete test would settle whether the transfer claims need qualification.
The referee's report
If the results hold under independent verification, RRI makes a strong advance in scalable self-improvement for open-ended domains. The central idea—using an adversarial reward hacker to supervise a model-written reward—is novel and addresses a well-recognized failure mode of self-play loops. The large improvements on writing and difficult tasks, and the transfer to non-medical benchmarks, suggest the method teaches generalizable answer quality rather than rubric-specific patterns. The detailed exploit taxonomy provides actionable guidance for future work in self-play and reward design.
Pith's reading of the paper (a machine's summary, not the authors')
RRI addresses the challenge of model-written rubrics in self-improving loops for open-ended tasks. It introduces an adversary that games each rubric with a useless answer, then repairs the rubric by adding criteria that favor a substantive honest answer. This process repeats, and the exploits found also rewrite how future tasks and rubrics are generated. On HealthBench Professional, RRI improves over the Generation Only baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points when every auxiliary role is served by the frozen 9B model. The gains are largest on difficult and writing tasks, and the approach generalizes to out-of-domain benchmarks like PRBench Hard and ProfBench without ever seeing their tasks.
What carries the argument. The adversarial rubric hack-and-repair loop, where an adversary writes a hacking answer that scores high under the current rubric but is as useless as possible, an honest answer is written without seeing the rubric, and up to two repair steps add criteria that the honest answer meets and the exploit does not. Additionally, a meta-optimizer updates the proposer's generation prompts based on confirmed exploits, evolving the task distribution and rubric quality over training.
Core claim. The central discovery is that a self-improving loop can supervise its own reward by having an adversary probe each generated rubric for exploits, then repairing the rubric based on a rubric-blind comparison between the exploit and an honest answer. The repair step adds criteria that the honest answer meets and the exploit does not, forcing the solver to learn substantive content rather than superficial keyword coverage. The meta-optimizer further uses confirmed exploits to rewrite the instructions for future task and rubric generation, making the improvement cumulative. This yields significant in-domain gains and out-of-domain transfer, outperforming methods like RaR, EvoRubrics, and SibylSense in the same setting.
The premise it rests on. The paper assumes that the model-based judge (GPT-5.5-Chat or frozen 9B) consistently identifies the better answer when comparing the exploit and honest answer; if the judge has systematic biases (e.g., preferring longer answers or certain wording), the repair step may correct for the wrong signal.
If the paper is right (Pith's inferences, not the authors')
Self-improving systems can autonomously improve their reward functions without any human-labeled data or reference answers, making them applicable to domains where expert supervision is scarce.
The approach transfers to tasks outside the training domain (e.g., from medical to finance and legal) without seeing any of those tasks, suggesting broad applicability.
The largest gains are on difficult and writing tasks, where model-written rubrics are most vulnerable to exploitation, indicating that adversarial repair is especially valuable for complex open-ended tasks.
Iterative adversarial testing of rubrics, with re-attack after patching, adds further improvements over a single screening, supporting the principle of continued verification.
The 27B model trained only on a task description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench, demonstrating that the method induces transferable capabilities.
The 17% incorrect pair rate (where the honest answer is not actually better) reveals a fundamental limitation: the rubric-blind judge cannot verify hard facts like statutes, thresholds, or arithmetic, suggesting that future work needs more robust verification beyond the same model.
The exploit categories (missed negatives, ambiguous rubrics) provide a taxonomy of reward failures that could inform general principles for designing robust model-written rewards in self-play loops.
The method's success with a frozen 9B model serving all auxiliary roles suggests that the approach is practical even without a superhuman teacher, as long as the adversary is strong enough to find loopholes.
The meta-optimizer's use of confirmed exploits to rewrite generation prompts hints at a form of procedural knowledge transfer: the loop learns what kinds of criteria are rewindable and adjusts its generation accordingly.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Recursive Self-Improvement (RSI) has gained wide attention for its ability to work with limited data. In a proposer-solver loop, a proposer model writes training tasks and a solver model learns to solve them through reinforcement learning. Such loops have succeeded in mathematics and coding, where every answer can be checked automatically. Open-ended expert tasks, like clinical report generations, have no answer key, which makes simple question-answer generation not viable. Here the proposer must also write a rubric that lists the content an answer should include and the errors it should avoid. However, the unique challenge behind model-written rubric is the comprehensiveness of it. For naively generated rubric, solvers tend to exploit it, naming the expected topics without the substance behind them, which corrupts the learning process. To solve this problem, we introduce RRI, Recursive Reward Improvement, which uses a reward hacking adversary to supervise the reward. Before the solver trains on a generated task, an adversary writes an answer that games its rubric while being as useless as possible, the same model writes an honest answer without seeing the rubric, and up to two repair steps revise the rubric to make the honest answer win. The exploits found also rewrite how future tasks and rubrics are generated. On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model. In particular, the gains are largest on difficult and writing tasks. We show that the 27B model trained on the HealthBench Professional description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench without seeing a task from either. We further analyze the exploitable points of model-generated question-rubric pairs to facilitate future research on open-ended self-play.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-16
The authors' main claim, as the classifier picked it out 0.94
“On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.26 · human participants 0.08 · controlled comparison 0.96 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Recursive Self-Improvement (RSI) has gained wide attention for its ability to work with limited data. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| In a proposer-solver loop, a proposer model writes training tasks and a solver model learns to solve them through reinforcement learning. | background 0.79 | 0.09 | 0.02 | asserted 0.99 |
| Such loops have succeeded in mathematics and coding, where every answer can be checked automatically. | background 0.99 | 0.08 | 0.02 | asserted 1.00 |
| Open-ended expert tasks, like clinical report generations, have no answer key, which makes simple question-answer generation not viable. | background 0.63 | 0.07 | 0.02 | asserted 1.00 |
| Here the proposer must also write a rubric that lists the content an answer should include and the errors it should avoid. | background 0.83 | 0.09 | 0.03 | asserted 1.00 |
| However, the unique challenge behind model-written rubric is the comprehensiveness of it. | gap 0.51 | 0.10 | 0.03 | asserted 1.00 |
| For naively generated rubric, solvers tend to exploit it, naming the expected topics without the substance behind them, which corrupts the learning process. | background 0.69 | 0.34 | 0.03 | asserted 0.98 |
| To solve this problem, we introduce RRI, Recursive Reward Improvement, which uses a reward hacking adversary to supervise the reward. | contribution 0.90 | 0.83 | 0.14 | asserted 1.00 |
| Before the solver trains on a generated task, an adversary writes an answer that games its rubric while being as useless as possible, the same model writes an honest answer without seeing the rubric, and up to two repair steps revise the rubric to make the honest answer win. | method 1.00 | 0.88 | 0.08 | asserted 0.98 |
| The exploits found also rewrite how future tasks and rubrics are generated. | method 0.76 | 0.73 | 0.09 | asserted 1.00 |
| On HealthBench Professional, RRI improves the adjusted score over the Generation Only self-improvement baseline by 10.9 points at 9B and 12.4 at 27B, and by 10.3 points without any external model. | result 1.00 | 0.96 | 0.38 | asserted 1.00 |
| In particular, the gains are largest on difficult and writing tasks. | result 0.77 | 0.92 | 0.10 | asserted 1.00 |
| We show that the 27B model trained on the HealthBench Professional description improves by 6.6 points on PRBench Hard and 8.0 on ProfBench without seeing a task from either. | result 1.00 | 0.96 | 0.13 | asserted 1.00 |
| We further analyze the exploitable points of model-generated question-rubric pairs to facilitate future research on open-ended self-play. | contribution 0.70 | 0.92 | 0.05 | asserted 0.97 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.94
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-16: 1 request, 10,208 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (7)
Figures are the authors', reproduced from the paper.
References (114)
- [1] Qoq-med: Building multimodal clinical foundation models with domain-aware grpo training , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [2] Synthesizing high-quality visual question answering from medical documents with generator-verifier LMMs , author=. International Conference on Learning Representations , volume= on Pith
- [3] MIMIC-IV, a freely accessible electronic health record dataset , author=. Scientific data , volume=. 2023 , publisher= on Pith
- [4] Large language models and synthetic health data: progress and prospects , author=. JAMIA open , volume=. 2024 , publisher= on Pith
- [5] RareBench: can LLMs serve as rare diseases specialists? , author=. Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining , pages= on Pith
- [6] Huatuogpt-o1, towards medical complex reasoning with llms , author=. arXiv preprint arXiv:2412.18925 , year= on PithPith's reviewarXiv
- [7] Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models , author=. IEEE transactions on medical imaging , year= on Pith
- [8] Medgemma technical report , author=. arXiv preprint arXiv:2507.05201 , year= on PithPith's reviewarXiv
- [9] Orca: Progressive learning from complex explanation traces of gpt-4 , author=. arXiv preprint arXiv:2306.02707 , year= on PithPith's reviewarXiv
- [10] Ultramedical: Building specialized generalists in biomedicine , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [11] The false promise of imitating proprietary llms , author=. arXiv preprint arXiv:2305.15717 , year= on PithPith's reviewarXiv
- [12] Absolute zero: Reinforced self-play reasoning with zero data , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [13] R-zero: Self-evolving reasoning llm from zero data , author=. International Conference on Learning Representations , volume= on Pith
- [14] Self-questioning language models , author=. arXiv preprint arXiv:2508.03682 , year= on PithPith's reviewarXiv
- [15] Spice: Self-play in corpus environments improves reasoning , author=. arXiv preprint arXiv:2510.24684 , year= on PitharXiv
- [16] The curse of recursion: Training on generated data makes models forget , author=. arXiv preprint arXiv:2305.17493 , year= on PithPith's reviewarXiv
- [17] Self-play only evolves when self-synthetic pipeline ensures learnable information gain , author=. arXiv preprint arXiv:2603.02218 , year= on PithPith's reviewarXiv
- [18] Retrieval-augmented generation for large language models: A survey , author=. arXiv preprint arXiv:2312.10997 , year= on PithPith's reviewarXiv
- [19] Star: Bootstrapping reasoning with reasoning , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [20] Reinforced self-training (rest) for language modeling , author=. arXiv preprint arXiv:2308.08998 , year= on PithPith's reviewarXiv
show 80 more
- [21] Self-rewarding language models , author=. arXiv preprint arXiv:2401.10020 , year= on PithPith's reviewarXiv
- [22] Self-play fine-tuning converts weak language models to strong language models , author=. arXiv preprint arXiv:2401.01335 , year= on PithPith's reviewarXiv
- [23] Self-improvement in language models: The sharpening mechanism , author=. International Conference on Learning Representations , volume= on Pith
- [24] Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning , author=. International Conference on Learning Representations , volume= on Pith
- [25] Self-challenging language model agents , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [26] Propose, solve, verify: Self-play through formal verification , author=. arXiv preprint arXiv:2512.18160 , year= on PitharXiv
- [27] Coral: Towards autonomous multi-agent evolution for open-ended discovery , author=. arXiv preprint arXiv:2604.01658 , year= on PithPith's reviewarXiv
- [28] G-Zero: Self-play for open-ended generation from zero data , author=. arXiv preprint arXiv:2605.09959 , year= on PithPith's reviewarXiv
- [29] Conversation for Non-verifiable Learning: Self-Evolving LLMs through Meta-Evaluation , author=. arXiv preprint arXiv:2601.21464 , year= on PithPith's reviewarXiv
- [30] Textbooks are all you need , author=. arXiv preprint arXiv:2306.11644 , year= on PithPith's reviewarXiv
- [31] Phi-4 technical report , author=. arXiv preprint arXiv:2412.08905 , year= on PithPith's reviewarXiv
- [32] Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data , author=. arXiv preprint arXiv:2404.01413 , year= on PithPith's reviewarXiv
- [33] Beyond model collapse: Scaling up with synthesized data requires verification , author=. International Conference on Learning Representations , volume= on Pith
- [34] Genie: Achieving human parity in content-grounded datasets generation , author=. arXiv preprint arXiv:2401.14367 , year= on PithPith's reviewarXiv
- [35] Synthetic continued pretraining , author=. International Conference on Learning Representations , volume= on Pith
- [36] Large language models encode clinical knowledge , author=. Nature , volume=. 2023 , publisher= on Pith
- [37] Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning , author=. arXiv preprint arXiv:2506.07044 , year= on PithPith's reviewarXiv
- [38] Baichuan-m1: Pushing the medical capability of large language models , author=. arXiv preprint arXiv:2502.12671 , year= on PithPith's reviewarXiv
- [39] m1: Unleash the potential of test-time scaling for medical reasoning with large language models , author=. arXiv preprint arXiv:2504.00869 , year= on PitharXiv
- [40] Med-u1: Incentivizing unified medical reasoning in llms via large-scale reinforcement learning , author=. arXiv preprint arXiv:2506.12307 , year= on PithPith's reviewarXiv
- [41] Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs , author=. arXiv preprint arXiv:2504.00993 , year= on PithPith's reviewarXiv
- [42] Medagentsim: Self-evolving multi-agent simulations for realistic clinical interactions , author=. International Conference on Medical Image Computing and Computer-Assisted Intervention , pages=. 2025 , organization= on Pith
- [43] EvoMDT: a self-evolving multi-agent system for structured clinical decision-making in multi-cancer , author=. npj Digital Medicine , volume=. 2026 , publisher= on Pith
- [44] A synthetic data generation framework for scalable and resource-efficient medical AI assistants , author=. 2025 , publisher= on Pith
- [45] Healthbench: Evaluating large language models towards improved human health , author=. arXiv preprint arXiv:2505.08775 , year= on PithPith's reviewarXiv
- [46] Rubrics as rewards: Reinforcement learning beyond verifiable domains , author=. International Conference on Learning Representations , volume= on Pith
- [47] Checklists are better than reward models for aligning language models , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [48] Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year= on PithPith's reviewarXiv
- [49] Understanding r1-zero-like training: A critical perspective , author=. arXiv preprint arXiv:2503.20783 , year= on PithPith's reviewarXiv
- [50] Concrete problems in AI safety , author=. arXiv preprint arXiv:1606.06565 , year= on PithPith's reviewarXiv
- [51] Defining and characterizing reward gaming , author=. Advances in neural information processing systems , volume= on Pith
- [52] Sycophancy to subterfuge: Investigating reward-tampering in large language models , author=. arXiv preprint arXiv:2406.10162 , year= on PithPith's reviewarXiv
- [53] Odin: Disentangled reward mitigates hacking in rlhf , author=. arXiv preprint arXiv:2402.07319 , year= on PithPith's reviewarXiv
- [54] Length-controlled alpacaeval: A simple way to debias automatic evaluators , author=. arXiv preprint arXiv:2404.04475 , year= on PithPith's reviewarXiv
- [55] Disentangling length from quality in direct preference optimization , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages= on Pith
- [56] Scaling laws for reward model overoptimization , author=. International conference on machine learning , pages=. 2023 , organization= on Pith
- [57] Dr tulu: Reinforcement learning with evolving rubrics for deep research , author=. arXiv preprint arXiv:2511.19399 , year= on PithPith's reviewarXiv
- [58] Large language models as optimizers , author=. International Conference on Learning Representations , volume= on Pith
- [59] Dspy: Compiling declarative language model calls into self-improving pipelines , author=. arXiv preprint arXiv:2310.03714 , year= on PithPith's reviewarXiv
- [60] Gepa: Reflective prompt evolution can outperform reinforcement learning , author=. International Conference on Learning Representations , volume= on Pith
- [61] Textgrad: Automatic" differentiation" via text , author=. arXiv preprint arXiv:2406.07496 , year= on PithPith's reviewarXiv
- [62] AI safety via debate , author=. arXiv preprint arXiv:1805.00899 , year= on PithPith's reviewarXiv
- [63] Rainbow teaming: Open-ended generation of diverse adversarial prompts , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [64] Weak-to-strong generalization: Eliciting strong capabilities with weak supervision , author=. arXiv preprint arXiv:2312.09390 , year= on PithPith's reviewarXiv
- [65] Generative verifiers: Reward modeling as next-token prediction , author=. International Conference on Learning Representations , volume= on Pith
- [66] Self-instruct: Aligning language models with self-generated instructions , author=. Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers) , pages= on Pith
- [67] rStar-Math: Small LLMs can master math reasoning with self-evolved deep thinking , author=. arXiv preprint arXiv:2501.04519 , year= on PithPith's reviewarXiv
- [68] A survey of self-evolving agents: What, when, how, and where to evolve on the path to artificial super intelligence , author=. arXiv preprint arXiv:2507.21046 , year= on PithPith's reviewarXiv
- [69] Wizardlm: Empowering large language models to follow complex instructions , author=. arXiv e-prints , pages= on Pith
- [70] Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing , author=. International Conference on Learning Representations , volume= on Pith
- [71] Compute as teacher: Turning inference compute into reference-free supervision , author=. arXiv preprint arXiv:2509.14234 , year= on PithPith's reviewarXiv
- [72] Openrubrics: Towards scalable synthetic rubric generation for reward modeling and llm alignment , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
- [73] Rm-r1: Reward modeling as reasoning , author=. International Conference on Learning Representations , volume= on Pith
- [74] Automatic prompt optimization with “gradient descent” and beam search , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages= on Pith
- [75] Connecting large language models with evolutionary algorithms yields powerful prompt optimizers , author=. International Conference on Learning Representations , volume= on Pith
- [76] Promptbreeder: Self-referential self-improvement via prompt evolution , author=. arXiv preprint arXiv:2309.16797 , year= on PithPith's reviewarXiv
- [77] Language models learn to mislead humans via rlhf , author=. International Conference on Learning Representations , volume= on Pith
- [78] Rlcd: Reinforcement learning from contrastive distillation for language model alignment , author=. arXiv preprint arXiv:2307.12950 , year= on PithPith's reviewarXiv
- [79] Rewardbench: Evaluating reward models for language modeling , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages= on Pith
- [80] Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume= on Pith
- [81] G-eval: NLG evaluation using gpt-4 with better human alignment , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages= on Pith
- [82] Debating with more persuasive llms leads to more truthful answers , author=. arXiv preprint arXiv:2402.06782 , year= on PithPith's reviewarXiv
- [83] EvoLM: In search of lost language model training dynamics , author=. arXiv preprint arXiv:2506.16029 , year= on PitharXiv
- [84] Echo chamber: Rl post-training amplifies behaviors learned in pretraining , author=. arXiv preprint arXiv:2504.07912 , year= on PithPith's reviewarXiv
- [85] Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [86] Toward expert-level medical question answering with large language models , author=. Nature medicine , volume=. 2025 , publisher= on Pith
- [87] Tulu 3: Pushing frontiers in open language model post-training , author=. arXiv preprint arXiv:2411.15124 , year= on PithPith's reviewarXiv
- [88] Hybridflow: A flexible and efficient rlhf framework , author=. Proceedings of the Twentieth European Conference on Computer Systems , pages= on Pith
- [89] Search-r1: Training llms to reason and leverage search engines with reinforcement learning , author=. arXiv preprint arXiv:2503.09516 , year= on PithPith's reviewarXiv
- [90] Webthinker: Empowering large reasoning models with deep research capability , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [91] HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats , author=. arXiv preprint arXiv:2604.27470 , year= on PithPith's reviewarXiv
- [92] Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year= on PithPith's reviewarXiv
- [93] StatPearls , author=. 2024 , publisher= on Pith
- [94] DailyMed , author=. National Institutes of Health , year= on Pith
- [95] MEDLINEplus: building and maintaining the National Library of Medicine's consumer health Web service , author=. Bulletin of the Medical Library Association , volume= on Pith
- [96] ICD-10: international statistical classification of diseases and related health problems: tenth revision , author=. 2004 , publisher= on Pith
- [97] Climb: Data foundations for large scale multimodal clinical foundation models , author=. arXiv preprint arXiv:2503.07667 , year= on PithPith's reviewarXiv
- [98] Medrag: Enhancing retrieval-augmented generation with knowledge graph-elicited reasoning for healthcare copilot , author=. Proceedings of the ACM on Web Conference 2025 , pages= on Pith
- [99] Pubmedqa: A dataset for biomedical research question answering , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages= on Pith
- [100] Pmc-vqa: Visual instruction tuning for medical visual question answering , author=. arXiv preprint arXiv:2305.10415 , year= on PithPith's reviewarXiv
The first 100 of 114 references are listed here. All of them are in this paper's citation record.
Cited by 1 Pith paper
Reviewed papers on Pith that cite this one. The full citation record
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment