Pith.

Computation and Language · arXiv:2610.23839 v1 · submitted 16 Oct 2026 · 43 references

DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses

Jianyuan Zhong, Chenwei Wu, Zhaochen Yu, Xixian Jiang, Xuewei Li, Hao Xiao, Xilin Dai, Junyang Wang

show 16 more authors

Jiaru Zou, Zhiqing Cui, Boyuan Zheng, Jing Xiong, Zhen Zhang, Qiang Gao, Michael Morley, Luis Nakayama, Qiang Xu, Liyue Shen, Tao Gui, Leo Anthony Celi, James Zou, Zhenfei Yin, Yingcheng Wu, Ling Yang

The paper says

“Scientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging.”

more of the abstract

We introduce an operational definition of valuable and testable research hypotheses: evidence-grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. Our DiscoveryRSI framework develops hypothesis trees through multi-agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. We also introduce DFM-Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. Across four life-science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement, with the large gains in falsifiability and data resolvability. We further validate hypothesis quality through expert review, comparing our DiscoveryRSI-27B with other research agents. Preliminary human assessments show that DiscoveryRSI-27B scores higher than GPT-6 Luna on hypothesis importance and whether claims are plausible and evidence-bounded. DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 19 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A detailed, self-aware pipeline for recursive hypothesis improvement, but the central quantitative evidence for self-improvement across rounds relies on an automated judge from the same model family, which the authors acknowledge. The paper deserves peer review but the core claim needs independent validation. This line is wrong

1 major objection · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 43 references say.

The paper says, at Section 7.1 and Appendix D.1

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.90):

“The resulting labels are correlated automated assessments rather than independent expert reviews.”

Pith says: Objection 1 of 1 · major

The central evidence for recursive self-improvement across rounds—the increase in mean hypothesis quality from 4.86 to 5.39—rests entirely on scores produced by GPT-5.6 Sol. The authors acknowledge in Appendix D.1 that this automated judge 'measures correlated AI perspectives, not three independent expert opinions,' and the human evaluation in Section 7.3 does not compare hypotheses from different rounds.

more of objection 1

Because the round-to-round improvement is a core contribution listed in Section 1 ('Recursive self-improvement from research experience'), this limitation is load-bearing. The authors should either provide an independent evaluation (e.g., human assessment of hypotheses from different rounds) or clearly qualify the strength of the evidence by reframing the claim to reflect that it has been measured only by a correlated judge and that the independent evidence is limited to cross-sectional comparisons.

5 minor objections

The paper says, at Abstract

Pith says: minor objection 1 of 5

'largely outperform GPT-6' should be 'largely outperforms GPT-6' for grammatical agreement.

The paper says, at Figure 1 caption

Pith says: minor objection 2 of 5

'with the large gains in falsifiability and data resolvability' is vague; specify that these gains refer to the automated quality profile and cite the relevant figure or section.

show 3 more

The paper says, at Table 1

Pith says: minor objection 3 of 5

The check marks for DFM-Bench should be cross-referenced with the definitions in Appendix D.6 to ensure consistency between the table and the supporting text, particularly for 'Evidence use' and 'Contribution maturity.'

The paper says, at Appendix B.1

Pith says: minor objection 4 of 5

The derivation of posterior update equations for TrueSkill (Eqs. 11-13) assumes Gaussian-incoming beliefs and a single pairwise comparison. It would be helpful to state explicitly that this is a didactic illustration and that the actual implementation uses the full message-passing schedule.

The paper says, at Section 6, Battery setting

Pith says: minor objection 5 of 5

The description of the battery setting reads as a fifth domain rather than a separate transfer test. Clarify early in the paragraph that this is intended to test generalization beyond life science.

What would settle it

Have an independent evaluator (e.g., Llama 3 70B or a panel of domain experts) score the same hypotheses from rounds 0, 1, and 2 using the DFM-Bench protocol (rubric, eight dimensions, same weights). If the independent evaluator reproduces the monotonic increase (4.86 < 5.18 < 5.39) within overlapping 95% confidence intervals, the concern is resolved. If the trend disappears or reverses, the automated judge likely introduced systematic bias.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 19 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 24 s · 30,693 tokens read, 2,991 written
  3. Editor wrote the verdict above 10 s · 31,685 tokens read, 1,087 written
  4. Referee major revision: 1 major objection, 5 minor 30 s · 31,153 tokens read, 3,193 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This is a thoughtful and ambitious paper. The core idea—turning hypothesis development into a recursive learning process through credit attribution, harness replay, and rediscovery—is well motivated and the implementation is serious. The operational definition of valuable/testable hypotheses, the credit allocation scheme, and the DFM-Bench evaluation framework are all genuine contributions. The authors are unusually transparent about their limitations, which I respect.

The main result is the round-to-round improvement in automated quality scores (4.86→5.18→5.39). But the judge is GPT-5.6 Sol, from the same family as the trained system. The authors say in Appendix D.1 that the panel “measures correlated AI perspectives, not three independent expert opinions.” That is the right caveat, but it means the evidence for what distinguishes this paper—recursive self-improvement over rounds—is not independently validated. The human evaluation (Figure 8) compares final model variants, not rounds. The literature-inspiration metric compares final outputs. So while the human and literature evidence are valuable cross-sectional validations, they do not support the longitudinal claim.

That said, the paper is still worthwhile. The framework is a concrete step toward agents that learn from their own research process. The human-rated revision improvement (+0.07 per step vs. +0.03 and −0.01) is a meaningful signal, even if it does not test rounds. The behavior changes (more evolve/repair, fewer restarts) are plausible correlates of improvement. The battery-domain results add breadth.

A serious referee would ask for round-to-round human evaluation or an independent automated judge. The paper as it stands is a strong technical report with a load-bearing validity gap. I would accept it for peer review, but the editor should expect conditionals. The authors have the right instincts; they just need a cleaner evaluation protocol to make the central claim land.

Who this is for: researchers in agentic scientific discovery, automated hypothesis generation, and recursive self-improvement. The methodology sections are the main value. The evaluation sections need caveats but still offer useful design ideas.

The strongest objection

The paper's central claim of recursive self-improvement rests on automated quality scores that increase across rounds (Figure 6). The judge, GPT-5.6 Sol, is from the same model family as DiscoveryRSI-27B, and the authors acknowledge in Appendix D.1 that the automated panel 'measures correlated AI perspectives, not three independent expert opinions.' While the authors state that the training signal comes from a separate Critique agent (not GPT-5.6 Sol), the evaluation itself is not independent. The human expert assessments (Figure 8) compare DiscoveryRSI-27B to other agents but do not evaluate hypotheses from different rounds; they provide cross-sectional validation, not longitudinal. The literature-inspiration metric (RQ4) also compares final models, not rounds. Consequently, the contribution that distinguishes this work—self-improvement across rounds—is supported almost entirely by an automated judge whose independence is questionable. The authors are transparent about this limitation, but it remains the weakest point in the argument. Without an independent evaluator confirming the round-to-round quality increase, the strength of the evidence for recursive improvement is substantially reduced.

The reader's weakest-assumption correctly identifies the automated judge circularity. My analysis confirms that this is the most load-bearing concern because the paper's signature claim (effective recursive self-improvement across rounds) depends on that automated evaluation. The human assessment and literature-inspiration metrics provide complementary but not round-wise validation. The authors acknowledge the limitation, and the framework and methodology are otherwise well-motivated. Therefore, the verdict should remain CONDITIONAL: the paper merits acceptance contingent on independent round-to-round validation. The unchanged verdict reflects that the concern is real but does not, in itself, overturn the paper's contributions; it does, however, demand a concrete test.

The referee's report

If the claimed improvements hold, the work represents a significant advance toward agents that can improve their own hypothesis-generation capabilities through accumulated research experience. The formal definition of valuable and testable hypotheses, the detailed credit-allocation procedure, the DFM-Bench evaluation framework covering five levels of assessment, and the empirical results across multiple domains collectively provide a substantial contribution to the emerging field of AI-driven scientific discovery. The paper's thorough documentation of methods and limitations also serves as a valuable reference for future work on recursive self-improvement in research agents.

Pith's reading of the paper (a machine's summary, not the authors')

The paper tries to show that an AI system can continuously improve its ability to generate valuable, testable scientific hypotheses by treating each round of hypothesis development and investigation as a source of training signal for the next round. To do this, the authors define a structured hypothesis format (claim, investigation plan, decision criteria, alternative explanations) and build the DiscoveryRSI framework, which uses multi-agent search to build hypothesis trees, compiles selected investigations into executable Harbor tasks, and then applies two learning mechanisms: hypothesis credit optimization (which links scientific assessments back to the specific agent contributions that produced them) and harness replay (which diagnoses recurring failures and refines the workflow). Across four life-science domains, automated evaluation shows mean hypothesis quality rising from 4.86 to 5.39 after two self-improvement rounds, with gains concentrated in falsifiability and data resolvability. Human experts rate DiscoveryRSI revisions as improving more consistently than those of GPT-6 Luna or GPT-5.6 Sol, and the generated hypotheses are more likely to connect to concrete research directions in subsequent published literature (6.3% vs 2.5% for GPT).

What carries the argument. Hypothesis credit optimization: a procedure that first uses TrueSkill paired-comparison inference to derive a conservative preference score for each hypothesis, then links that score through documented contribution records back to specific agent trajectories, and finally applies bounded weighted supervised fine-tuning (Eq. 4) to only the supported assistant tokens in those trajectories. Harness replay complements this by keeping model weights fixed and instead editing the workflow (agent coordination, evidence access, review timing) based on diagnosed recurring failures.

Core claim. Hypothesis credit optimization and harness replay can turn multi-agent hypothesis development into a recursive self-improvement loop. Credit optimization attributes a hypothesis's scientific assessment to the documented contributions (Lead, Generation, Evolution trajectories) that produced it, then uses bounded weighted training to reinforce supported actions. Harness replay holds the model fixed, diagnoses repeated exploration, tool failures, and premature closure from recorded execution records, and tests targeted workflow edits. Rediscovery then applies accepted model and harness updates to fresh research questions. The paper claims that this loop measurably lifts hypothesis quality, shifts agent behavior toward more sustained refinement, and yields human-recognized revision gains and higher correspondence with post-cutoff research.

The premise it rests on. The main quantitative evidence for self-improvement (hypothesis quality rising from 4.86 to 5.39) relies on an automated judge (GPT-5.6 Sol) from the same model family as the system being evaluated, which the paper acknowledges (Appendix D.1) measures correlated AI perspectives rather than independent expert opinions.

If the paper is right (Pith's inferences, not the authors')

Research agents can improve their hypothesis quality without fresh human-labeled data, using only the recorded outcomes of their own investigations.

The same credit-attribution method may transfer to other multi-agent scientific workflows where final assessments need to be decomposed into contributing actions.

DFM-Bench provides a five-level evaluation framework (hypothesis, revision, development path, research run, contribution) that other scientific-discovery benchmarks can adopt.

The 6.3% literature-correspondence rate suggests that generated hypotheses can anticipate or align with real research directions, though prospective tracking is needed.

If the recursive self-improvement holds under independent evaluation, it would imply that the bottleneck for AI-driven discovery is not the model's innate creativity but the structure of the learning signal extracted from research histories.

The decline in importance and novelty scores (from 6.48 to 6.04 and 4.87 to 4.58) alongside sharp gains in falsifiability and data resolvability suggests a trade-off: the training pressure may favour cautious, well-specified investigations over bold but risky claims.

Because the automated judge (GPT-5.6 Sol) belongs to the same model family as the system being evaluated, the quantitative improvement scores should be re-estimated with a judge from a different architecture or with expert human adjudication on a held-out set.

The framework's reliance on detailed record-keeping (every contribution, every execution artifact) may limit its portability to domains where such logging is expensive or impossible.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Scientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging. We introduce an operational definition of valuable and testable research hypotheses: evidence-grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. Our DiscoveryRSI framework develops hypothesis trees through multi-agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. We also introduce DFM-Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. Across four life-science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement, with the large gains in falsifiability and data resolvability. We further validate hypothesis quality through expert review, comparing our DiscoveryRSI-27B with other research agents. Preliminary human assessments show that DiscoveryRSI-27B scores higher than GPT-6 Luna on hypothesis importance and whether claims are plausible and evidence-bounded. DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-20

The authors' main claim, as the classifier picked it out 0.72

“DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.39 · human participants 0.41 · controlled comparison 0.84 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.06

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Scientific discovery requires developing worthwhile hypotheses into practical investigations, yet learning from this evolving research process remains challenging. background 0.79 0.05 0.03 asserted 1.00
We introduce an operational definition of valuable and testable research hypotheses: evidence-grounded claims about consequential uncertainties, coupled with feasible investigations, explicit decision criteria, and alternative explanations. contribution 0.96 0.83 0.08 asserted 0.99
Our DiscoveryRSI framework develops hypothesis trees through multi-agent evolutionary search, compiles selected investigations into executable Harbor tasks, and recursively learns through three mechanisms: hypothesis credit optimization attributes scientific assessments to documented research contributions; harness replay identifies productive strategies and recurring failures to refine research workflows; and rediscovery applies accepted improvements to new investigations, generating further experience for subsequent learning. method 0.95 0.93 0.13 asserted 1.00
We also introduce DFM-Bench, which provides comprehensive assessment of hypothesis quality and cumulative research progress at the levels of hypotheses, revisions, development paths and research runs. contribution 0.89 0.95 0.06 asserted 1.00
Across four life-science domains, automated evaluation shows that hypothesis quality rises from 4.86 for the base model to 5.39 after two rounds of self-improvement, with the large gains in falsifiability and data resolvability. result 1.00 0.96 0.23 asserted 1.00
We further validate hypothesis quality through expert review, comparing our DiscoveryRSI-27B with other research agents. method 0.53 0.93 0.07 asserted 0.96
Preliminary human assessments show that DiscoveryRSI-27B scores higher than GPT-6 Luna on hypothesis importance and whether claims are plausible and evidence-bounded. result 1.00 0.94 0.11 asserted 0.85
DiscoveryRSI-27B achieved a 6.3% rate of hypothesis correspondence with research published after the model cutoff, largely outperform GPT-6 with 2.5% and Claude Opus 5 with 2.3%, while delivering greater human-rated improvement along research paths. result 1.00 0.95 0.28 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from simulation or computation 0.76

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-20: 1 request, 6,493 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (10)
Figure 1 of the paper
Figure 1. (a) More actionable links to published research. The rate is the fraction of generated hypotheses linked to post-cutoff expert publications supplying a concrete design change, comparator, measurement, or testable evidence gap. DiscoveryRSI leads at 6.3% (Appendix F). (b) RSI improves quality and shifts evolutionary behavior. Score measures research quality and action share is the fraction of episodes using each operation. Research quality rises from 4.86 to 5.39 as evolve/repair use rises from 77.5% to 91.7%. (c) Larger human-rated gains per revision. Mean revision change averages…
Figure 2 of the paper
Figure 2. DiscoveryRSI couples multi-agent hypothesis development with recursive self-improvement. (a) Within an episode, a Lead coordinates Generation and Evolution; candidate hypothesis versions are critiqued and progress is recorded. Selected hypotheses are compiled into Harbor tasks, with linked evidence, data, drafts, artifacts, assessments, contributions, and execution records. (b) Research trees carry hypotheses across RSI rounds: the recorded state 𝐻𝑘 enters RSI, which produces updated parameters (𝜃𝑘+1, 𝜓𝑘+1) and fresh questions for rediscovery. (c) Two update branches learn from…
Figure 3 of the paper
Figure 3. instantiates Definition 1 with a MIMIC-IV proposal. Its value lies in asking whether repeated creatinine–urine output mismatch adds information about adverse outcomes after ICU discharge. Its testability comes from a defined population and outcome window, patient-level held-out comparisons against simpler baselines, explicit interpretation criteria, and alternative explanations that bound the claim. A supported lack of added information would also inform whether more complex models merit further investigation. Recorded hypothesis versions retain links to their predecessors and…
Figure 4 of the paper
Figure 4. Expert-supervised assessment of scientific hypotheses. (a) An anonymous MIMIC-IV proposal presents its scientific question, planned comparison, and claim limits. (b) Reviewers record an overall judgment, assess six anchored criteria, and identify design flaws; flaw details and comments are optional. The criterion list summarizes the rubric, while radio-option excerpts reproduce one saved assessment. Identifying metadata is omitted. The example illustrates the review procedure rather than an aggregate evaluation result. 6. Experimental Setup Research domains. We evaluate…
Figure 5 of the paper
Figure 5. Shared research process across life-science and battery domains. The process strip summarizes how evidence becomes a question, investigation, and evidence-bounded interpretation with iterative critique. The two rows show the four life-science domains and four battery domains used in the experiments. Improvements concentrate on the specification of testable investigations. Falsifiability rises from 4.00 to 6.20, consideration of alternative explanations from 4.44 to 5.98, data resolvability from 3.73 to 4.99, and feasibility from 4.79 to 5.87. These changes indicate more explicit…
Figure 6 of the paper
Figure 6. Automated hypothesis quality across self-improvement rounds. (a) Top: example research tree from each round; color shows weighted quality across eight evaluation axes. Middle: mean hypothesis quality per round. Error bars show 95% confidence intervals and shading shows the interquartile range. Bottom: mean quality per domain with 95% confidence intervals. (b) Mean and axis-specific scores. All scores are produced by the automated grader using the weights in Appendix D. 0 50 100 Share of episodes (%) Base Round 1 Round 2 58 20 22 69 21 6 4 64 28 5 3 −0.5 0 0.5 1.0 1.5 Δ path score,…
Figure 7 of the paper
Figure 7. Changes in research behavior across rounds. (a) Share of each action across all episodes in a round. (b) Share of campaigns taking each action at each episode position, sampled from six campaigns per round; blank cells indicate 0%. (c) Change in development-path scores from the base model to round 2. 15
Figure 8 of the paper
Figure 8. Human assessment of hypothesis quality and revision progress. (a) Mean ratings across six criteria on a 1–4 scale. (b) Mean score change per revision. (c) Fractions of improved, unchanged, and worse revisions, in the same model order as panel (b). Absolute quality ratings show a mixed profile. DiscoveryRSI-27B scores higher than GPT-6 Luna on importance (3.10 versus 2.94) and plausible, evidence-bounded claims (3.37 versus 3.16). Both receive 2.31 for advance over known work, while GPT-6 Luna scores higher on data fit, alternative explanations, and informative results. Out of all…
Figure 9 of the paper
Figure 9. Design changes and human ratings along a revision path. (a) A MIMIC-IV path and assessed siblings; node values are six-criterion means on a 1–4 scale. Numbered changes are explained below the tree. Dashed ochre nodes mark fatal flaws; the two final fatal siblings lack specified AKI thresholds. (b) Criterion ratings along the highlighted path. 7.4. RQ4: Do the hypotheses inspire subsequent research? We define “inspiration” as a concrete connection between a generated hypothesis and subsequent literature. A publication after the recorded cutoff must provide an experimen￾tal design,…
Figure 10 of the paper
Figure 10. Hypothesis inspiration measured through subsequent literature. The upper panel orders model groups by the proportion of generated hypothesis versions meeting the inspiration criterion. The lower panel reports the same rate across four domains. DiscoveryRSI-27B pools two production runs; Qwen original is omitted. The figure’s reference-actionability labels denote this retrospective inspiration measure. positive automated judgment under the above definition: 𝑅insp = Í ℎ 𝑎ℎ |H | , (7) where 𝑎ℎ indicates a positive judgment and H contains all generated hypothesis versions in the group

Figures are the authors', reproduced from the paper.

References (43)
  1. [1] Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflecti... on PithPith's reviewarXiv
  2. [2] Anthropic. How Claude Code Works, 2026. URL https://code.claude.com/docs/en/ how-claude-code-works. Official documentation, accessed 2026-10-14 on Pith
  3. [3] Ziru Chen, Shijie Chen, Yuting Ning, et al. ScienceAgentBench: Toward rigorous assessment of language agents for data-driven scientific discovery. arXiv:2410.05080v1, 2024. URL https: //arxiv.org/abs/2410.05080v1 on PithPith's reviewarXiv
  4. [4] Taoyong Cui, Pinxian Zeng, Wanghan Xu, Hao Xiao, Shuhan Xue, Qiang Gao, Wanli Ouyang, Pheng Ann Heng, Zhenfei Yin, Yingcheng Wu, and Ling Yang. RSI-Anything: Unlocking Recursive Self-Improvement with World Models. Technical Report PHAI-TR- 2026-05, PhAI Labs, 2026. URL https:/... on Pith
  5. [5] Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. RAFT: Reward ranked finetuning for generative foundation model alignment.Transactions on Machine Learning Research, 2023. URL https: //arxiv.org/a... on PithPith's reviewarXiv
  6. [6] Ron Edgar, Michael Domrachev, and Alex E. Lash. Gene Expression Omnibus: NCBI gene expression and hybridization array data repository.Nucleic Acids Research, 30(1):207–210, 2002. doi: 10.1093/ nar/30.1.207. URLhttps://doi.org/10.1093/nar/30.1.207 on Pithdoi:10.1093/nar/30.1.207
  7. [7] Hejia Geng, Zesen Huang, Haoyang Li, et al. Scienceide: Turning world’s scientific codebase into agent learnable environments. arXiv:2609.19134v1, 2026. URLhttps://arxiv.org/abs/2609. 19134v1 on PithPith's reviewarXiv
  8. [8] Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, et al. Accelerating scientific discovery with co-scientist. arXiv:2502.18864v2, 2026. URLhttps://arxiv.org/abs/2502.18864v2 on PithPith's reviewarXiv
  9. [9] KenGu, RuoxiShang, RuienJiang, etal. BLADE:Benchmarkinglanguagemodelagentsfordata-driven science. arXiv:2408.09667v3, 2025. URLhttps://arxiv.org/abs/2408.09667v3 on PitharXiv
  10. [10] Harbor contributors. Harbor: Task format. Online documentation. URL https://docs. harborframework.com/tasks/overview. Accessed 2026-10-09 on Pith
  11. [11] RalfHerbrich, TomMinka, andThoreGraepel. TrueSkill: Abayesianskillratingsystem. InAdvancesin Neural Information Processing Systems, volume 19, 2006. URLhttps://proceedings.neurips. cc/paper/2006/hash/f44ee263952e65b3610b8ba51229d1f9-Abstract.html on Pith
  12. [12] Kexin Huang, Serena Zhang, Hanchen Wang, et al. Biomni: A general-purpose biomedical AI agent. bioRxiv, 2025. URLhttps://doi.org/10.1101/2025.05.30.656746 on Pithdoi:10.1101/2025.05.30.656746
  13. [13] Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, et al. MIMIC-IV, a freely accessible electronic health record dataset.Scientific Data, 10:1, 2023. doi: 10.1038/s41597-022-01899-x. URL https://doi.org/10.1038/s41597-022-01899-x on Pithdoi:10.1038/s41597-022-01899-x
  14. [14] Penelope K. Jones, Ulrich Stimming, and Alpha A. Lee. Impedance-based forecasting of lithium-ion battery performance amid uneven usage.Nature Communications, 13:4806, 2022. doi: 10.1038/ s41467-022-32422-w. URLhttps://doi.org/10.1038/s41467-022-32422-w on Pithdoi:10.1038/s41467-022-32422-w
  15. [15] Niall Kirkaldy, Mohammad A. Samieian, Gregory J. Offer, Monica Marinescu, and Yatish Patel. Lithium-ion battery degradation: Comprehensive cycle ageing data and analysis for commercial 21700 cells.Journal of Power Sources, 603:234185, 2024. doi: 10.1016/j.jpowsour.2024.234185.... on Pithdoi:10.1016/j.jpowsour.2024.234185
  16. [16] Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Somayeh Sojoudi, Matei Zaharia, and Yujin Tang. Recursive Self-Improvement through Multi-Agent Self-Supervision. arXiv:2610.12176, 2026. URLhttps://arxiv.org/abs/2610.12176 on PithPith's reviewarXiv
  17. [17] Ruihe Li, Niall D. Kirkaldy, Fabian F. Oehler, Monica Marinescu, Gregory J. Offer, and Simon E. J. O’Kane. The importance of degradation mode analysis in parameterising lifetime predic- 20 DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses tion models of lithiu... on Pithdoi:10.1038/s41467-025-57968-3
  18. [18] Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Zhiheng Xi, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui, and Yu-Gang Jiang. Agentic Harness Engineering: Observability- Driven Automatic Evolution of Coding-Agent Harnesses. arXiv:2604.25850, 2026. URLhttps: //a... on PithPith's reviewarXiv
  19. [19] Jiarui Liu, Renjie Tao, Yiwei Liao, Chuanyang Jin, Kai Sun, Xiao Yang, et al. IdeaScientist: Orchestrating Agents for Grounded Scientific Ideation. arXiv:2610.04074, 2026. URL https: //arxiv.org/abs/2610.04074 on PithPith's reviewarXiv
  20. [20] Matthias Luh and Thomas Blank. Comprehensive battery aging dataset: capacity and impedance fade measurements of a lithium-ion NMC/C-SiO cell.Scientific Data, 11:1004, 2024. doi: 10.1038/ s41597-024-03831-x. URLhttps://doi.org/10.1038/s41597-024-03831-x on Pithdoi:10.1038/s41597-024-03831-x
show 23 more
  1. [21] Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal, and Peter Clark. DiscoveryBench: Towards data-driven discovery with large language models. arXiv:2407.01725v1, 2024.... on PithPith's reviewarXiv
  2. [22] Yunxiang Mo, Tianshi Zheng, Yisen Gao, et al. Agentideabench: Benchmarking scientific ideation in the agent era. arXiv:2609.07611v1, 2026. URLhttps://arxiv.org/abs/2609.07611v1 on PithPith's reviewarXiv
  3. [23] OpenAI. Codexasa Platform: Buildon theOpen AgentHarness, 2026. URLhttps://developers. openai.com/blog/codex-as-a-platform. Official documentation, accessed 2026-10-14 on Pith
  4. [24] Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning. arXiv:1910.00177v3, 2019. URLhttps: //arxiv.org/abs/1910.00177v3 on PithPith's reviewarXiv
  5. [25] Tom J. Pollard, Alistair E. W. Johnson, Jesse D. Raffa, Leo A. Celi, Roger G. Mark, and Omar Badawi. The eICU Collaborative Research Database, a freely available multi-center database for critical care research.Scientific Data, 5(1):180178, 2018. doi: 10.1038/sdata.2018.178. U... on Pithdoi:10.1038/sdata.2018.178
  6. [26] Trishna Raj, Andrew A. Wang, Charles W. Monroe, and David A. Howey. Investigation of path- dependent degradation in lithium-ion batteries.Batteries & Supercaps, 3:1377–1385, 2020. doi: 10.1002/batt.202000160. URLhttps://doi.org/10.1002/batt.202000160 on Pithdoi:10.1002/batt.202000160
  7. [27] Ugis Sarkans, Anja Füllgrabe, Ahmed Ali, Awais Athar, Ehsan Behrangi, Nestor Diaz, et al. From ArrayExpress to BioStudies.Nucleic Acids Research, 49(D1):D1502–D1506, 2021. doi: 10.1093/nar/ gkaa1062. URLhttps://doi.org/10.1093/nar/gkaa1062 on Pithdoi:10.1093/nar/
  8. [28] Chengyang Shi, Xianglin Ji, Jintao Huang, Jicheng Wang, Yifeng He, and Jiachen Liu. Open- endedness bench: Measuring epistemic process from agent records. arXiv:2610.02588v1, 2026. URL https://arxiv.org/abs/2610.02588v1 on PithPith's reviewarXiv
  9. [29] Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366, 2023. URLhttps://arxiv.org/abs/2303.11366 on PithPith's reviewarXiv
  10. [30] Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron Parisi, Abhishek Kumar, Alex Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Elsayed, Hanie Sedghi, Igor ... on PithPith's reviewarXiv
  11. [31] Cathie Sudlow, John Gallacher, Naomi Allen, et al. UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age.PLOS Medicine, 12(3):e1001779, 2015. doi: 10.1371/journal.pmed.1001779. URLhttps://doi.org/10.1371/ journ... on Pithdoi:10.1371/journal.pmed.1001779
  12. [32] Shuhan Xue, Jianyuan Zhong, Ziyuan Nan, Wenbin Li, Zhaochen Yu, Jinchao Ding, Qiang Gao, Pengyu Zhan, Yuntong Zhang, Tian Cheng, Zhenfei Yin, Yingcheng Wu, and Ling Yang. Sciencebuddy: Recursive-in-recursive self-improvement for interactive scientific agents. Tech- nical repor... on Pith
  13. [33] Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The AI scientist-v2: Workshop-level automated scientific discovery via agentic tree search. arXiv:2504.08066, 2025. URLhttps://arxiv.org/abs/2504.08066 on PithPith's reviewarXiv
  14. [34] Junlin Yang, Che Jiang, Yu Fu, et al. Frontis-MA1: Training an AI4AI model towards recursive self-improvement in machine learning engineering. arXiv:2607.28568v1, 2026. URLhttps:// arxiv.org/html/2607.28568v1 on PithPith's reviewarXiv
  15. [35] Ling Yang, Zhenfei Yin, and Yingcheng Wu. Discovery foundation models: Toward open-ended discov- ery intelligence. arXiv:2609.15973v1, 2026. URLhttps://arxiv.org/abs/2609.15973v1 on PithPith's reviewarXiv
  16. [36] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing Reasoning and Acting in Language Models. InInternational Conference on Learning Representations, 2023. URLhttps://arxiv.org/abs/2210.03629 on PithPith's reviewarXiv
  17. [37] Haotian Ye, Haowei Lin, Jingyi Tang, Yizhen Luo, Rahul Thapa, Caiyin Yang, Chang Su, Rui Yang, Ruihua Liu, Rundao Li, Zeyu Li, Pengwei Sun, Chong Gao, Dachao Ding, Guangrong He, Miaolei Zhang, Lina Sun, Wenyang Wang, Yuchen Zhong, Zhuohao Shen, Puheng Li, Pan Lu, Bianxiao Cui,... on PithPith's reviewarXiv
  18. [38] Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, and Shuyue Hu. Self-Harness: Harnesses That Improve Themselves. arXiv:2606.09498, 2026. URL https://arxiv.org/abs/2606.09498 on PithPith's reviewarXiv
  19. [39] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Godel Machine: Open- Ended Evolution of Self-Improving Agents. arXiv:2505.22954, 2025. URLhttps://arxiv.org/ abs/2505.22954 on PithPith's reviewarXiv
  20. [40] Jenny Zhang, Bingchen Zhao, Wannan Yang, Jakob Foerster, Jeff Clune, Minqi Jiang, Sam Devlin, and Tatiana Shavrina. Hyperagents. arXiv:2603.19461, 2026. URLhttps://arxiv.org/abs/ 2603.19461 on PithPith's reviewarXiv
  21. [41] Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kama- nuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. arXiv:2... on PithPith's reviewarXiv
  22. [42] YunweiZhang,QiaochuTang,YaoZhang,JiabinWang,UlrichStimming,andAlphaA.Lee. Identifying degradation patterns of lithium ion batteries from impedance spectroscopy using machine learning. Nature Communications, 11:1706, 2020. doi: 10.1038/s41467-020-15235-7. URLhttps://doi. org/10... on Pithdoi:10.1038/s41467-020-15235-7
  23. [43] Tong Zheng, Xidong Wu, Zheng Zhang, et al. Dream-RSI: Recursive self-improvement through evolving worlds. arXiv:2609.14858v1, 2026. URLhttps://arxiv.org/abs/2609.14858v1. 22 DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses DiscoveryRSI Supplementary Methods a... on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment