Pith.

Computation and Language · arXiv:2609.15973 v1 · submitted 14 Sep 2026 · 60 references

Discovery Foundation Models: Toward Open-Ended Discovery Intelligence

Ling Yang, Zhenfei Yin, Yingcheng Wu

The paper says

“Foundation models have progressed from learning and reasoning over existing knowledge, to increasingly learning through action, tool use, and outcome feedback.”

more of the abstract

We argue that the next frontier is a further transition: from solving and acting within problems specified by humans to participating in the process by which new problems, representations, explanations, and knowledge are created. We refer to this capability as Discovery Intelligence. We formulate Discovery Foundation Models (DFMs) as general-purpose model systems for open-ended discovery. A DFM operates over a revisable research state and supports seven coupled capabilities spanning problem discovery, formulation, representation construction, hypothesis formation, intervention, evidence-grounded revision, and continual discovery improvement. We instantiate this framework with Zetema, which couples explicit research-state dynamics, verification and experimental gating, external grounding, and cross-task Discovery Skill evolution. We further ground the framework with GALILEO, a real therapeutic-discovery system in which Dry-Lab reasoning, robotic and hands-on Wet-Lab experimentation, external biological evidence, and iterative hypothesis and design revision form a closed physical discovery loop. We then formulate a unified approach to capability formation and process-centered evaluation, enabling discovery behavior to be trained, improved, and measured beyond final-answer performance. Together, these components establish discovery as a learnable, executable, and evaluable capability of foundation-model systems. We view this shift as a broader progression in intelligence scaling: from learning over existing knowledge, to learning from action outcomes, and ultimately to participating in the construction, testing, and revision of the structures through which new knowledge is discovered. Code: https://github.com/Gen-Verse/DFM-Plans

from the abstract

Pith says

Machine review · deepseek-v4-flash · 15 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.

A thoughtful conceptual framework for discovery AI, but the load-bearing training assumptions and absence of empirical validation mean this is a roadmap, not a working system. This line is wrong

4 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 60 references say.

The paper says, at Section 6.2, Eq. (21)

The review did not quote this passage.

Pith says: Objection 1 of 4 · major

The per-step scientific reward includes components δK_t (externally supported knowledge progress) and IG_t (information gain). As the skeptic correctly notes, operationalizing these requires access to ground-truth latent structure—precisely what the system is supposed to discover. Section 8.2 acknowledges that simulation-grounded environments can provide this, but the central DFM claim concerns open-ended discovery across tasks and domains where such ground truth is unavailable.

more of objection 1

The paper does not explain how δK_t and IG_t would be estimated from observations alone, without already knowing what is worth discovering. This is load-bearing: if the reward cannot be computed for genuinely novel problems, Algorithm 1 cannot be applied, and the claim that DFM capabilities are 'learnable' (Abstract, Section 6) remains unsupported. The authors should either propose a concrete estimation strategy (e.g., using the Research World Model as a proxy, or relying solely on process-level verifiers) or explicitly delimit the training framework to settings where ground-truth rewards are available.

The paper says, at Section 5.7 and Section 1

The review did not quote this passage.

Pith says: Objection 2 of 4 · major

The paper presents DFMs as a general-purpose category ('Discovery Foundation Models are a general-purpose model system category that can identify valuable unknowns...' (Definition 1) and 'Discovery Foundation Models as a model-system category for this setting' (Section 1)). The only empirical support for this generality is the GALILEO case study, which is explicitly domain-specific (therapeutic peptide discovery in dynamic membrane systems).

more of objection 2

The paper itself states that GALILEO 'should not be interpreted as evidence that the full general-purpose DFM problem has already been solved' (Section 5.7). There is a disconnect: the title, abstract, and introduction claim a general-purpose capability, but the evidence is limited to a single, heavily structured domain. The authors should either provide evidence of generality (e.g., demonstration of cross-domain transfer, even in a simulated setting) or re-scope the claims to indicate that the framework is a proposal whose generality has not yet been tested.

The paper says, at Section 6, Eqs. (16)–(25) and Algorithm 1

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

The training objectives require trajectory data with explicit state transitions (L_act, L_state), preference pairs (L_pref), interaction data for world-model and verifier learning (L_wm, L_ver), and interactive reinforcement learning from scientific feedback (L_RL). The paper acknowledges that historical artifacts are partial and simulated trajectories inherit assumptions (Section 6.1), but it does not discuss how to obtain the required supervision at scale for real scientific domains.

more of objection 3

For example, annotating research-state transitions and attributing them to specific operations is labor-intensive and may require expert judgment. The paper should discuss the feasibility of data acquisition, or propose a curriculum that starts with simpler synthetic environments and gradually increases complexity.

The paper says, at Section 3.2 and Figure 2

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

The seven capabilities are listed but their inter-dependencies and separability are not examined. For instance, C_repr (representation construction) and C_form (problem formulation) may interact: changing the representation can change what counts as a well-formed problem. The paper does not discuss how to avoid redundant or conflicting capability definitions, nor does it provide a principled criterion for when a capability is 'learned.'

more of objection 4

The evaluation protocol (Section 7) would benefit from operational definitions that map each capability to measurable behaviors in a controlled environment.

7 minor objections

The paper says, at Section 5.7, last paragraph

Pith says: minor objection 1 of 7

The mapping of GALILEO to DFM capabilities is asserted but not illustrated with concrete research-state transitions. For example, the claim that 'target nomination and translational-gap analysis instantiate aspects of C_find and C_form' would be strengthened by showing the initial research state S_0 and the state after the formulation step. A small table or state snippet would help.

The paper says, at Figure 3

Pith says: minor objection 2 of 7

The arrows in the figure are not labeled with the corresponding capabilities. For clarity, each arrow could indicate which of the seven capabilities (e.g., C_form, C_repr) is involved in that transition.

show 5 more

The paper says, at Section 6.2, Eq. (21)

Pith says: minor objection 3 of 7

The quantity IG_t is defined as 'realized or counterfactually estimated information gain.' Counterfactual estimation over research decisions is a complex problem; the paper should cite relevant work on counterfactual inference in sequential decision-making or explain the intended approach briefly.

The paper says, at Section 4.2, Eq. (7)

Pith says: minor objection 4 of 7

The definition of candidate explanation H_i includes D_i (validity range). In many scientific contexts, specifying the validity range precisely is itself a discovery problem. The paper could acknowledge that validity ranges are often not known a priori and may be part of what the system learns.

The paper says, at Section 7.5

Pith says: minor objection 5 of 7

The discussion of benchmark construction mentions that 'partial progress is retained,' but does not specify how partial credit is assigned. For example, recovering the correct representation without solving the task should receive some score. A concrete example of partial-credit assignment would help operationalize the evaluation.

The paper says, at Throughout

Pith says: minor objection 6 of 7

The paper uses many specialized terms (Zetema, Research State, Discovery Skill, GalilEO, etc.) without a glossary. A table of key terms and their definitions would improve readability.

The paper says, at Section 4.4

Pith says: minor objection 7 of 7

The concept of 'candidate lessons' is introduced but not connected to the Discovery Skill memory of Section 5.3. Clarifying the relationship between candidate lessons and formalized skills m_i would help.

What would settle it

Construct a simulated discovery environment with a hidden causal mechanism unknown to the training procedure. Implement the reward from Eq. 21 using only observable outcomes (e.g., prediction error, intervention effect sizes) without access to the hidden mechanism or ground-truth knowledge progress. Train a policy using Algorithm 1 with this observable-only reward. Measure whether the policy recovers the hidden mechanism more efficiently than a baseline using only final-outcome reward. If the observable-only reward does not improve discovery performance, then the training formulation is insufficient for open-ended discovery without oracle access.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 15 Sep 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 26 s · 28,258 tokens read, 2,578 written
  3. Editor wrote the verdict above 13 s · 29,105 tokens read, 1,074 written
  4. Referee major revision: 4 major objections, 7 minor 29 s · 28,619 tokens read, 2,953 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The main thing to know: this paper provides the cleanest formalization I've seen of what an AI system capable of open-ended scientific discovery would need to do. The seven-capability breakdown (problem discovery, formulation, representation, hypothesis, intervention, revision, continual improvement) is well-organized and the research-state dynamics are internally sound. The GALILEO case study, while domain-specific, at least shows physical evidence actually changing subsequent decisions rather than just validating a final candidate. The paper earns its reader's ACCEPT as a conceptual proposal.

The soft spot is the stress-test concern, and it's real. Equation 21 defines per-step scientific rewards that include knowledge progress δK_t and information gain IG_t. For open-ended discovery, you can't compute these without knowing the ground truth about what's worth discovering. The paper acknowledges simulators provide this, but that's precisely the setting where the DFM claim is most claimed to apply. The training procedure (Algorithm 1) and the learnability claim in the abstract depend on these reward signals being computable. The paper does not resolve this circularity for genuinely novel discovery problems.

Additionally, the seven-capability joint learning and transfer across domains is purely speculative. No evidence is given that the training objectives (Eq. 25) produce anything like the claimed behavior. The paper is upfront that this is a framework, but the central claims about learnability and evaluation remain architectural hypotheses.

The self-citations are noticeable (about a third of references involve the first author) but mostly to related methodological work on reasoning templates and agent harnesses. They don't invalidate the framework, but they aren't independent validation either.

This paper is for researchers designing next-generation scientific AI systems who want a structured target to aim at. It doesn't deliver a working system, but it clarifies what would count as one. The paper deserves serious peer review because it addresses an important problem and is internally coherent. It should come with an explicit note that the training assumptions remain untested. I'd recommend acceptance for a venue that values conceptual contributions, with a request to acknowledge the reward circularity more prominently.

The strongest objection

The training formulation in Section 6.2 defines per-step scientific rewards (Eq. 21) that include δK_t (externally supported knowledge progress) and IG_t (realized or counterfactually estimated information gain). These components are intuitive, but operationalizing them requires access to ground truth about the latent structure of the world—precisely what a discovery system is supposed to uncover. In simulation-grounded environments (Section 8.2) the hidden mechanism is known to the evaluator, so these rewards can be computed. However, the paper's central claim concerns open-ended discovery across tasks and domains, where ground truth is not available to the trainer. The paper does not address how δK_t and IG_t would be reliably estimated from observations alone, without already knowing what is worth discovering. This is not an implementation detail: if the reward signal cannot be computed for genuinely novel discovery problems, then the training procedure cannot be applied, and the claim that DFM capabilities are 'learnable' (Abstract, Section 6) remains unsupported. The paper acknowledges the usefulness of simulators but does not resolve the circularity for real-world settings. This is the most load-bearing assumption because the entire training framework (Algorithm 1) depends on these reward signals for policy improvement.

The reader's weakest assumption already identified that the training procedures (Section 6) must correspond to a tractable learning problem and that the reward signals are sufficient for policy improvement. My stress-test refines this to a specific circularity: the reward terms require ground-truth knowledge that is unavailable in genuinely open-ended discovery. This is a load-bearing concern because the paper claims DFMs are 'learnable,' and the training derivation is a core contribution. However, the paper is a conceptual framework, not an empirical claim; it does not assert that the training problem is solved, only that it is formulated. The reader's ACCEPT verdict acknowledges the paper's value as a structured target. My concern does not invalidate the framework—it identifies a needed line of future work. Therefore, the verdict should remain unchanged. The agreement_with_reader is 'agree' because the reader flagged the tractability assumption, and my concern is a specific instance of it.

The referee's report

If the framework holds, it provides a principled decomposition of the discovery process that goes beyond task-specific scientific AI and anchors evaluation in intermediate research-state transitions rather than final outputs alone. The seven-capability factorization, the distinction between within-task and cross-task learning, and the explicit treatment of the research state as a revisable object are conceptually important. However, the paper is almost entirely architectural: no experiment tests whether the proposed training objectives produce the claimed skills, and the only empirical grounding (GALILEO) is a domain-specific system whose mapping to the general-purpose DFM definition is asserted rather than demonstrated. The significance therefore lies in the framework's potential as a foundation for future work, not in any validated result.

Pith's reading of the paper (a machine's summary, not the authors')

The paper tries to establish that AI systems can be extended from solving fixed scientific tasks to participating in the process that creates new problems, representations, explanations, and knowledge. It defines Discovery Foundation Models (DFMs) as a general-purpose model-system category that operates over a revisable research state and supports seven coupled capabilities: problem discovery, formulation, representation construction, hypothesis formation, intervention design, evidence-grounded revision, and continual improvement. If correct, this would shift the frontier of AI from optimization within a supplied structure to the ability to change that structure when evidence warrants. The paper instantiates the framework with Zetema and grounds it with GALILEO, a real therapeutic-discovery loop in which physical experiments revise subsequent scientific decisions.

What carries the argument. The explicit revisable research state (P, R, H, E, X, B, M) that tracks problem formulations, representations, hypotheses, evidence with provenance, intervention plans, resource budgets, and memory. The research world model and verification gates that filter actions before costly execution. The Discovery Skill memory (trigger, operation, expected effect, validation evidence) that stores reusable research operations from validated cross-task experience.

Core claim. The central claim is that open-ended discovery intelligence can be decomposed into seven coupled capabilities (C_find, C_form, C_repr, C_hyp, C_int, C_rev, C_cont) and that a model system equipped with an explicit revisable research state, a research world model and action gating, external grounding, and validated cross-task skill memory can realize these capabilities. The paper formalizes the discovery process as state transitions over problems, representations, hypotheses, interventions, and evidence, and proposes training objectives (state-conditioned imitation, preference learning, process verification, world-model learning, reinforcement learning from scientific feedback, and contrastive skill learning) that together target those transitions. Evaluation is defined as both externally validated knowledge progress in the current episode and measurable improvement in future discovery behavior under matched resources and retrieval controls.

The premise it rests on. The paper assumes that the seven coupled discovery capabilities can be jointly learned and transferred across domains through the described training procedures (state-conditioned imitation, preference learning, process verification, world-model learning, RL from scientific feedback, contrastive skill learning), without empirical evidence that such joint learning scales or that the reward signals and process supervision are sufficient for policy improvement.

If the paper is right (Pith's inferences, not the authors')

AI systems could autonomously identify which scientific unknowns deserve investigation and formulate testable research problems with explicit scope and success criteria.

Training protocols would reward informative interventions and evidence-grounded revisions, not just whether a final answer matches a held-out label.

Benchmarks for scientific AI would need to measure process quality (e.g., representation adequacy, failure attribution, efficiency of intervention design) alongside final-answer accuracy.

Cross-domain discovery skills, such as diagnosing a hidden variable or constructing a boundary test, could transfer from simulation to physical laboratory settings.

Physical experiments can become part of an iterative learning loop where outcome feedback directly updates hypothesis, representation, and intervention policies.

The framework implicitly assumes that the discovery process can be formalized into state transitions that are amenable to gradient-based learning; real-world discovery often involves tacit knowledge and social dynamics that may resist such formalization.

The GALILEO case provides concrete evidence that physical experimental feedback can change research decisions in a closed loop, which is a necessary but not sufficient condition for the full DFM vision; it suggests near-term feasibility in structured drug-discovery settings.

The training data requirements (including failed trajectories, abandoned formulations, and attribution records) may be difficult to scale, especially in physical sciences where such data are rarely recorded; synthetic data and interactive environments may become critical bottlenecks.

The recursive-update risk analysis offers useful governance principles, but operational audits and staged deployment protocols are needed to turn those principles into practice before any recursive system can be evaluated safely.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper · Review history (3 versions)

The paper

Abstract

Foundation models have progressed from learning and reasoning over existing knowledge, to increasingly learning through action, tool use, and outcome feedback. We argue that the next frontier is a further transition: from solving and acting within problems specified by humans to participating in the process by which new problems, representations, explanations, and knowledge are created. We refer to this capability as Discovery Intelligence. We formulate Discovery Foundation Models (DFMs) as general-purpose model systems for open-ended discovery. A DFM operates over a revisable research state and supports seven coupled capabilities spanning problem discovery, formulation, representation construction, hypothesis formation, intervention, evidence-grounded revision, and continual discovery improvement. We instantiate this framework with Zetema, which couples explicit research-state dynamics, verification and experimental gating, external grounding, and cross-task Discovery Skill evolution. We further ground the framework with GALILEO, a real therapeutic-discovery system in which Dry-Lab reasoning, robotic and hands-on Wet-Lab experimentation, external biological evidence, and iterative hypothesis and design revision form a closed physical discovery loop. We then formulate a unified approach to capability formation and process-centered evaluation, enabling discovery behavior to be trained, improved, and measured beyond final-answer performance. Together, these components establish discovery as a learnable, executable, and evaluable capability of foundation-model systems. We view this shift as a broader progression in intelligence scaling: from learning over existing knowledge, to learning from action outcomes, and ultimately to participating in the construction, testing, and revision of the structures through which new knowledge is discovered. Code: https://github.com/Gen-Verse/DFM-Plans

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 0.97

“Together, these components establish discovery as a learnable, executable, and evaluable capability of foundation-model systems.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.07 · human participants 0.10 · controlled comparison 0.02 · randomized assignment 0.01 · states quantitative result 0.08 · compares with prior results 0.10 · mentions code or data availability 0.94

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Foundation models have progressed from learning and reasoning over existing knowledge, to increasingly learning through action, tool use, and outcome feedback. background 1.00 0.07 0.04 asserted 1.00
We argue that the next frontier is a further transition: from solving and acting within problems specified by humans to participating in the process by which new problems, representations, explanations, and knowledge are created. contribution 0.37 0.13 0.31 asserted 0.77
We refer to this capability as Discovery Intelligence. contribution 0.83 0.57 0.07 asserted 0.95
We formulate Discovery Foundation Models (DFMs) as general-purpose model systems for open-ended discovery. contribution 0.83 0.76 0.09 asserted 0.99
A DFM operates over a revisable research state and supports seven coupled capabilities spanning problem discovery, formulation, representation construction, hypothesis formation, intervention, evidence-grounded revision, and continual discovery improvement. contribution 0.51 0.62 0.09 asserted 1.00
We instantiate this framework with Zetema, which couples explicit research-state dynamics, verification and experimental gating, external grounding, and cross-task Discovery Skill evolution. contribution 0.53 0.88 0.05 asserted 1.00
We further ground the framework with GALILEO, a real therapeutic-discovery system in which Dry-Lab reasoning, robotic and hands-on Wet-Lab experimentation, external biological evidence, and iterative hypothesis and design revision form a closed physical discovery loop. contribution 0.64 0.67 0.07 asserted 1.00
We then formulate a unified approach to capability formation and process-centered evaluation, enabling discovery behavior to be trained, improved, and measured beyond final-answer performance. contribution 0.75 0.85 0.13 asserted 0.99
Together, these components establish discovery as a learnable, executable, and evaluable capability of foundation-model systems. interpretation 0.54 0.67 0.62 asserted 1.00
We view this shift as a broader progression in intelligence scaling: from learning over existing knowledge, to learning from action outcomes, and ultimately to participating in the construction, testing, and revision of the structures through which new knowledge is discovered. interpretation 1.00 0.13 0.34 asserted 0.83
Code: https://github.com/Gen-Verse/DFM-Plans future work 0.45 0.37 0.05 unclear 0.76

What kind of paper the classifier reports this is: position or commentary 0.56 · evidence from argument 0.92

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 8,234 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

References (60)
  1. [1] Milad Abolhasani and Eugenia Kumacheva. The rise of self-driving labs in chemical and materials sciences.Nature Synthesis, 2:483–492, 2023. doi: 10.1038/s44160-022-00231-0. 39 Discovery Foundation Models on Pithdoi:10.1038/s44160-022-00231-0
  2. [2] Josh Abramson, Jonas Adler, Jack Dunger, et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3.Nature, 630:493–500, 2024. doi: 10.1038/s41586-024-07487-w on Pithdoi:10.1038/s41586-024-07487-w
  3. [3] Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Researchagent: Iterative research idea generation over scientific literature with large language models. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Com... on Pithdoi:10.18653/v1/2025.naacl-long.342
  4. [4] Daniil A. Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models.Nature, 624:570–578, 2023. doi: 10.1038/s41586-023-06792-0 on Pithdoi:10.1038/s41586-023-06792-0
  5. [5] Rishi Bommasani et al. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258, 2021. URLhttps://arxiv.org/abs/2108.07258 on PithPith's reviewarXiv
  6. [6] Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. Augmenting large language models with chemistry tools.Nature Machine Intelligence, 6:525–535, on Pith
  7. [7] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, J... on Pith
  8. [8] Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems.Proceedings of the National Academy of Sciences, 113(15):3932–3937, 2016. doi: 10.1073/pnas.1517384113 on Pithdoi:10.1073/pnas.1517384113
  9. [9] Kathryn Chaloner and Isabella Verdinelli. Bayesian experimental design: A review.Statistical Science, 10(3):273–304, 1995. doi: 10.1214/ss/1177009939 on PitharXiv
  10. [10] Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. MLE-bench: Evaluating machine learning agents on machine learning engineering. InInternational Confe... on Pith
  11. [11] Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, Vishal Dey, Mingyi Xue, Frazier N. Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, and Huan Sun. Scienceagentbench: Toward ... on Pith
  12. [12] KanishkGandhi, MichaelY.Li, LyleGoodyear, etal. BoxingGym: Benchmarkingprogressinautomated experimental design and model discovery. InNeurIPS Workshop on Scaling Environments for Agents (SEA), 2025. URLhttps://openreview.net/forum?id=TgobzsU03X on Pith
  13. [13] Aniketh Garikaparthi, Manasi Patwardhan, and Arman Cohan. Researchgym: Evaluating language model agents on real-world AI research.arXiv preprint arXiv:2602.15112, 2026. URL https: //arxiv.org/abs/2602.15112 on PitharXiv
  14. [14] Ali E. Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J. Szostkiewicz, Dmytro Shved, Gavin J. Gyimesi, Jon M. Laurent, Samantha M. Wright, Muhammed T. Razzak, Andrew D. White, Silvia C. Finnemann, Michaela M. Hinks, and Samuel G. Rodriques. A multi-agent syst... on Pithdoi:10.1038/s41586-026-106
  15. [15] Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, et al. Accelerating scientific discovery with Co-Scientist.Nature, 655:487–496, 2026. doi: 10.1038/s41586-026-10644-y on Pithdoi:10.1038/s41586-026-10644-y
  16. [16] Daya Guo, Dejian Yang, Haowei Zhang, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature, 645:633–638, 2025. doi: 10.1038/s41586-025-09422-z on Pithdoi:10.1038/s41586-025-09422-z
  17. [17] Jiacheng Guo, Ling Yang, Peter Chen, Qixin Xiao, Yinjie Wang, Xinzhe Juan, Jiahao Qiu, Ke Shen, and Mengdi Wang. GenEnv: Difficulty-aligned co-evolution between LLM agents and environment simulators.arXiv preprint arXiv:2512.19682, 2025 on PitharXiv
  18. [18] Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models.Nature, 640:647–653, 2025. doi: 10.1038/s41586-025-08744-2 on Pithdoi:10.1038/s41586-025-08744-2
  19. [19] Qianyue Hao, Fengli Xu, Yong Li, and James Evans. Artificial intelligence tools expand scientists’ impact but contract science’s focus.Nature, 649(8099):1237–1243, 2026. doi: 10.1038/s41586-025 -09922-y on Pithdoi:10.1038/s41586-025
  20. [20] Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. MLAgentBench: Evaluating language agents on machine learning experimentation. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Learning Research, pages 20271–20309... on Pith
show 40 more
  1. [21] John Jumper, Richard Evans, Alexander Pritzel, et al. Highly accurate protein structure prediction with AlphaFold.Nature, 596:583–589, 2021. doi: 10.1038/s41586-021-03819-2 on Pithdoi:10.1038/s41586-021-03819-2
  2. [22] Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, et al. Learning skillful medium-range global weather forecasting.Science, 382(6677):1416–1421, 2023. doi: 10.1126/science.adi2336 on Pithdoi:10.1126/science.adi2336
  3. [23] Shi Xuan Leong, Caleb E. Griesbach, Rui Zhang, et al. Steering towards safe self-driving laboratories. Nature Reviews Chemistry, 9:707–722, 2025. doi: 10.1038/s41570-025-00747-x on Pithdoi:10.1038/s41570-025-00747-x
  4. [24] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, JohnSchulman, IlyaSutskever, andKarlCobbe. Let’sverifystepby step. InInternationalConferenceon Learning Representations, 2024. URLhttps://proceedings.iclr.cc/paper_files/pape r/2... on Pith
  5. [25] Chris Lu, Cong Lu, Robert Tjarko Lange, Yutaro Yamada, Shengran Hu, Jakob Foerster, David Ha, and Jeff Clune. Towards end-to-end automation of AI research.Nature, 651(8107):914–919, 2026. doi: 10.1038/s41586-026-10265-5 on Pithdoi:10.1038/s41586-026-10265-5
  6. [26] Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal, and Peter Clark. Discoverybench: Towards data-driven discovery with large language models. InInternational Conferenc... on Pith
  7. [27] Amil Merchant, Simon Batzner, Samuel S. Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery.Nature, 624:80–85, 2023. doi: 10.1038/s41586-023-06735-9 on Pithdoi:10.1038/s41586-023-06735-9
  8. [28] Lisa Messeri and M. J. Crockett. Artificial intelligence and illusions of understanding in scientific research.Nature, 627:49–58, 2024. doi: 10.1038/s41586-024-07146-0 on Pithdoi:10.1038/s41586-024-07146-0
  9. [29] Brian A. Nosek et al. Promoting an open research culture.Science, 348(6242):1422–1425, 2015. doi: 10.1126/science.aab2374 on Pithdoi:10.1126/science.aab2374
  10. [30] Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, et al. Probabilistic weather forecasting with machine learning.Nature, 637:84–90, 2025. doi: 10.1038/s41586-024-08252-9 on Pithdoi:10.1038/s41586-024-08252-9
  11. [31] Martiño Ríos-García, Nawaf Alampara, Chandan Gupta, Indrajeet Mandal, Sajid Mannan, Ali Asghar Aghajani, N. M. Anoop Krishnan, and Kevin Maik Jablonka. AI scientists produce results without reasoning scientifically.arXiv preprint arXiv:2604.18805, 2026. doi: 10.48550/arXiv.2604.18805 on PithPith's reviewarXivdoi:10.48550/arxiv.2604.18805
  12. [32] Christoph Scheurer and Karsten Reuter. Role of the human-in-the-loop in emerging self-driving laboratories for heterogeneous catalysis.Nature Catalysis, 8:13–19, 2025. doi: 10.1038/s41929-024 -01275-5. 41 Discovery Foundation Models on Pithdoi:10.1038/s41929-024
  13. [33] Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. InAdvances in Neural Information Processing Systems, volume 36, pages... on Pithdoi:10.52202/075280-2997
  14. [34] Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning.Proceedings of the IEEE, 109(5): 612–634, 2021. doi: 10.1109/JPROC.2021.3058954 on PitharXiv
  15. [35] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. InAdvances in Neural Information Processing Systems, volume 36, 2023. URLhttps://proceedings.neurips.cc/paper_files/paper /2023/has... on Pith
  16. [36] Michael D. Skarlinski, Sam Cox, Jon M. Laurent, James D. Braza, Michaela Hinks, Michael J. Ham- merling, Manvitha Ponnapati, Samuel G. Rodriques, and Andrew D. White. Language agents achieve superhuman synthesis of scientific knowledge.arXiv preprint arXiv:2409.13740, 2024. UR... on PithPith's reviewarXiv
  17. [37] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. InInternational Conference on Learning Representations, 2025. URLhttps://arxiv.org/abs/2408.03314 on PithPith's reviewarXiv
  18. [38] Zhangde Song, Jieyu Lu, Yuanqi Du, et al. Evaluating large language models in scientific discovery. arXiv preprint arXiv:2512.15567, 2025. URLhttps://arxiv.org/abs/2512.15567 on PithPith's reviewarXiv
  19. [39] Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, Johannes Heidecke, Amelia Glaese, and Tejal Patwardhan. PaperBench: Evaluating AI’s ability to replicate AI research. InProceedings ... on Pith
  20. [40] Kyle Swanson, Wesley Wu, Nash L. Bulaong, John E. Pak, and James Zou. The virtual lab of AI agents designs new SARS-CoV-2 nanobodies.Nature, 646:716–723, 2025. doi: 10.1038/s41586-025-09442 -9 on Pithdoi:10.1038/s41586-025-09442
  21. [41] NathanJ.Szymanski,BernardusRendy,YuxingFei,etal. Anautonomouslaboratoryfortheaccelerated synthesis of inorganic materials.Nature, 624:86–91, 2023. doi: 10.1038/s41586-023-06734-w on Pithdoi:10.1038/s41586-023-06734-w
  22. [42] Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, Wangchunshu Zhou, Meng Qu, Yilun Zhao, Jian Tang, Zhuosheng Zhang, Arman Cohan, Dov Greenbaum, Zhiyong Lu, and Mark Gerstein. Risks of AI scientists: Prioritizing safeguarding over autonomy.Nature Communications, 1... on Pithdoi:10.1038/s41467-025-63913-1
  23. [43] Florian Trost, Bide Zhang, Ines Aring, et al. An agentic framework for autonomous scientific discovery in cancer pathology.Nature Medicine, 32:2254–2266, 2026. doi: 10.1038/s41591-026-04357-y on Pithdoi:10.1038/s41591-026-04357-y
  24. [44] Silviu-Marian Udrescu and Max Tegmark. AI Feynman: A physics-inspired method for symbolic regression.Science Advances, 6(16):eaay2631, 2020. doi: 10.1126/sciadv.aay2631 on Pithdoi:10.1126/sciadv.aay2631
  25. [45] GuanzhiWang,YuqiXie,YunfanJiang,AjayMandlekar,ChaoweiXiao,YukeZhu,LinxiFan,andAnima Anandkumar. Voyager: An open-ended embodied agent with large language models.Transactions on Machine Learning Research, 2024 on Pith
  26. [46] Hanchen Wang, Tianfan Fu, Yuanqi Du, et al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60, 2023. doi: 10.1038/s41586-023-06221-2 on Pithdoi:10.1038/s41586-023-06221-2
  27. [47] Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. SciMON: Scientific inspiration machines optimized for novelty. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 279–299. Association for Computational Ling... on Pithdoi:10.18653/v1/2024.acl-long.18
  28. [48] Yinjie Wang, Ling Yang, Ye Tian, Ke Shen, and Mengdi Wang. Co-evolving LLM coder and unit tester via reinforcement learning. InAdvances in Neural Information Processing Systems, 2025. 42 Discovery Foundation Models on Pith
  29. [49] Yinjie Wang, Tianbao Xie, Ke Shen, Mengdi Wang, and Ling Yang. RLAnything: Forge environment, policy, and reward model in completely dynamic RL system. InProceedings of the Forty-Third International Conference on Machine Learning, 2026 on Pith
  30. [50] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, 2022. URLhttps://proceedin... on Pith
  31. [51] Mark D. Wilkinson et al. The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3:160018, 2016. doi: 10.1038/sdata.2016.18 on Pithdoi:10.1038/sdata.2016.18
  32. [52] Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E. Gonzalez, and Bin Cui. Buffer of Thoughts: Thought-augmented reasoning with large language models. In Advances in Neural Information Processing Systems, 2024 on Pith
  33. [53] Ling Yang, Zhaochen Yu, Bin Cui, and Mengdi Wang. ReasonFlux: Hierarchical LLM reasoning via scaling thought templates.arXiv preprint arXiv:2502.06772, 2025 on PithPith's reviewarXiv
  34. [54] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations, 2023. URLhttps://iclr.cc/virtual/2023/poster/11003 on Pith
  35. [55] Xunjian Yin, Xinyi Wang, Liangming Pan, Li Lin, Xiaojun Wan, and William Yang Wang. Gödel agent: A self-referential agent framework for recursively self-improvement. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper... on Pithdoi:10.18653/v1/2025.acl-long.1354
  36. [56] Zhaochen Yu, Yingcheng Wu, Zhenfei Yin, Kaiyuan Chen, Zhe Zhao, Mengdi Wang, Shuicheng Yan, and Ling Yang. Recursive experiential-working memory evolution for long-horizon agent harnesses. arXiv preprint arXiv:2608.24876, 2026 on PithPith's reviewarXiv
  37. [57] Claudio Zeni, Robert Pinsler, Daniel Zügner, et al. A generative model for inorganic materials design. Nature, 639:624–632, 2025. doi: 10.1038/s41586-025-08628-5 on Pithdoi:10.1038/s41586-025-08628-5
  38. [58] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin gödel machine: Open-ended evolution of self-improving agents.arXiv preprint arXiv:2505.22954, 2025. URL https://arxiv.org/abs/2505.22954 on PithPith's reviewarXiv
  39. [59] Yanbo Zhang, Sumeer A. Khan, Adnan Mahmud, Huck Yang, Alexander Lavin, Michael Levin, Jeremy Frey, Jared Dunnmon, James Evans, Alan Bundy, Saso Dzeroski, Jesper Tegner, and Hector Zenil. Exploring the role of large language models in the scientific method: From hypothesis to d... on Pithdoi:10.1038/s44387-025-00019-5
  40. [2024] doi: 10.1038/s42256-024-00832-8 on Pithdoi:10.1038/s42256-024-00832-8
Cited by 4 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment