Artificial Intelligence · arXiv:2610.40589 v1 · submitted 29 Oct 2026 · 23 references
Collaborative AI agents for autonomous experimental research with solid-state quantum spins
The paper says
“An autonomous research laboratory could conduct an entire study, from formulating questions to preparing manuscripts. Artificial intelligence (AI) offers one route towards this goal.”
more of the abstract
Recent AI systems take on several research roles, but may draw conclusions that the evidence does not support. Here, we present an autonomous research laboratory in which agents based on large language models assess one another's work and can stop or redirect investigations. They can generate and reject research questions, require separate theoretical checks and conduct experiments designed to challenge their claims. We demonstrate these capabilities by investigating ensembles of quantum spins in diamond. The agents derived the minimum number of calibration settings needed to predict responses to a defined family of microwave pulse sequences. They checked the result computationally and designed experiments to distinguish competing predictions. An initial experiment agreed with the theoretical predictions, but repeated reference measurements showed that the signal was drifting during data collection. The agents therefore treated the result as inconclusive and revised the measurement protocol without relaxing the acceptance criteria. The revised experiment met those criteria and supported the theoretical predictions. This study demonstrates that collaborating AI agents can select research questions, derive theoretical results and test their predictions experimentally.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 30 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A disciplined AI-run experiment with a genuinely novel rejection-and-revision loop, whose load-bearing dimension theorem is asserted rather than proved in the text; deserves a serious referee, conditional on the proof and data. This line is wrong
2 major objections · 3 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 23 references say.
The paper says, at §3 and §7.4
The review did not quote this passage.
Pith says: Objection 1 of 2 · major
The central theorem — d_L = L^2+1 calibration measurements are necessary and sufficient for sequences of at most L pulses — is stated and attributed to a proof by the PI agent (b78-b80, b152), but the proof does not appear in the manuscript.
more of objection 1
Section 7.4 defines F_L, asserts its dimension, and describes a construction (five phases 2pi k/5 to resolve the phase-Fourier components, then selection of d_L linearly independent sequence responses as a basis), but it does not carry out the dimension count, the linear-independence argument, or the lower-bound construction of distinct admissible ensembles that agree on an arbitrary set of fewer than L^2+1 calibrations yet differ on an unmeasured sequence. The computational verification in Section 7.5 confirms d_5 = 26 and d_6 = 37 by symbolic reconstruction, and the experiment tests one sufficiency instance at L=3; neither establishes the general necessary-and-sufficient formula, and the necessity direction is not experimentally tested at all. Because both the physics claim and the demonstration that the agents 'derived' a result rest entirely on this theorem, the proof must be included (an appendix is acceptable) or the claim must be explicitly downgraded to a conjecture supported by computational and experimental evidence.
The paper says, at §5, §7.6, §7.9
The review did not quote this passage.
Pith says: Objection 2 of 2 · major
The protocol precommitted a nine-parameter model of coherent ensemble evolution to be fitted to all ten calibration means (Section 7.6) and required its prediction to be locked alongside the others, but the manuscript nowhere reports this model's locked prediction, its value on the target contrast S, or whether it satisfied the acceptance criteria.
more of objection 2
This matters because the design's stated purpose was to distinguish the calibration law from competing explanations: the two reported baselines (Fourier interpolants) predict S = 0 by construction, so they are weak competitors, whereas a nine-parameter physical model fitted to all ten calibrations is exactly the kind of alternative whose prediction would calibrate the reader's confidence in the law. Please report the nine-parameter model's locked prediction and its outcome classification, or state explicitly why it was omitted.
3 minor objections
The paper says, at §7.9
Pith says: minor objection 1 of 3
The excess-variance factor F* is specified only through its allowed range (floor of 1.1, variance-validity failure above 1.25, sentence b302). Please state how F* is estimated — from which data, with which estimator, and whether the estimate was made before target means were computed — and report its value for the final acquisition; as written, the reader cannot tell how much of the reported standard errors is data-dependent scaling.
The paper says, at §7.1 and Open-Source Release
Pith says: minor objection 2 of 3
The Auctopus release provides workflow templates but not the prompts used in this study (b327). Since the agent behaviour itself is the paper's central object of study, the inability to reproduce the exact prompts or to view representative transcripts is a substantive reproducibility limitation; please consider releasing the prompts or redacted transcripts alongside the code.
show 1 more
The paper says, at §7.4
Pith says: minor objection 3 of 3
The photon-noise bound for worst-case mean-squared prediction error, stated as at most M d_L/(N - d_L + 1), is given without derivation or citation; a short argument or a reference would allow the reader to check the N - d_L + 1 denominator.
What would settle it
Run an exact symbolic linear-algebra check for L=4 using the model in Section 7.4: enumerate all sequences of at most 4 pulses with phases on the five-phase grid 2*pi*k/5 (first phase fixed), expand p_w(a) as real polynomials in Re a and Im a over K=[3/5,4/5]x[-1/5,1/5], and compute the rank of their real span; the claimed theorem requires dim F_4=17 and reproduces dim F_3=10, F_5=26, F_6=37. Independently, construct two admissible ensembles that agree on an insufficient calibration set but differ on a target, to verify the lower bound. If either check fails, the central claim is contradicted; if both pass, the remaining task is to supply the missing general proof or a machine-checked certificate.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 30 Oct 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 2 major objections, 3 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Two things to know before you read it. The experimental half is unusually disciplined: predictions and acceptance criteria were fixed before target analysis, the first design was rejected because Fourier interpolation would reproduce its "predictions," and a run that passed all ten accuracy criteria was still declared inconclusive when control checks failed — correctly traced to the signal drifting when a lab member switched on a room light. The agents then revised the protocol without loosening the criteria. That rejection-and-revision loop is the real demonstration, and it is well documented. The second thing is that the central theoretical claim — response-space dimension L2+1 — is asserted as proved but the proof is not in the text. Section 7.4 defines FL, states its dimension, and sketches a basis construction; it does not carry out the dimension count or the lower-bound construction. Since dL is an original result rather than a cited theorem, the reader cannot verify it from the preprint.
Credit where it is due. The L2+1 necessary-and-sufficient count is new relative to the cited prior work, and the cross-length design genuinely separates the law from the two specified interpolation baselines. Targets were withheld, predictions locked and hashed before analysis, and the final contrast (40.0±0.4 vs 39.7±1.3 predicted; zero under interpolation) matches. The first run's failure is honestly reported, not buried. The computational checks for L=5 and L=6 are exact certificates, and the stress test of the persistence assumption is a genuine robustness analysis. The persistence assumption itself is explicitly stated — each spin keeps the same error parameter throughout a sequence — and the authors flag and test it; I would not count that as a hidden flaw.
The soft spots, in proportion. The missing proof is the main one, and it is load-bearing: if dL is wrong, the ten calibrations need not span the L=3 response space and the measured contrast becomes a coincidence. The L=3 experimental span is stated as an exact rank calculation, which mitigates but does not replace a general proof for all L. The experiment is a single L=3 contrast on one sample, and neither raw data nor prompt logs are deposited; the open-source pipeline ships without the actual prompts used. None of this falsifies the claim, but each is a legitimate condition on acceptance.
Who benefits: people designing AI research agents and autonomous laboratories will get the most from it, mostly as a protocol model for precommitted, self-critical experimentation. The quantum-spin platform is the vehicle, not the point.
Send it to review. The experimental governance is worth referee time on its own, and the dimension theorem is checkable and should be checked. Acceptance should be conditional on the proof (or a formal certificate), deposited data and prompts, and ideally a second independent run.
The strongest objection
The most load-bearing step is the exact response-space dimension theorem, not the persistence assumption. The text states that the PI 'proved' d_L=L^2+1 (b78, b152) and that fewer calibrations cannot guarantee reconstruction (b80), but Section 7.4 only defines F_L, asserts its dimension, and describes a basis construction; it does not carry out the dimension count or the lower-bound construction. Because d_L is an original result rather than a cited theorem, the reader cannot verify the central premise. The computational check in Section 7.5 confirms dimensions 26 and 37 for L=5 and 6, and the experiment tests sufficiency on one L=3 contrast, but these do not establish the general necessary-and-sufficient formula. If d_L is wrong, the ten calibrations may not actually span the L=3 response space, and the measured contrast of 40.0 plus or minus 0.4 would be a coincidence rather than support for the law. The persistence assumption (b76, b212) is explicitly stated and stress-tested computationally, so it is secondary: even under perfect persistence, the paper's conclusion depends on the unproved dimension theorem.
I read the paper in good faith and find the experimental workflow genuinely strong: the rejected initial design, the retained inconclusive verdict after control failures, and the precommitted acceptance criteria are meaningful safeguards. The computational verification at L=5 and 6 and the four withheld targets at L=3 are real supporting evidence, but they do not replace a proof of the general theorem. The reader identified both the persistence assumption and the missing proof; I put more weight on the missing proof because it is upstream of every experimental interpretation. If d_L were not actually L^2+1, even an experiment with perfectly persistent per-spin errors could happen to agree on one contrast while the claimed calibration law is false. The persistence assumption is explicitly acknowledged and stress-tested, so criticizing it alone would not change the verdict. The appropriate outcome remains the reader's CONDITIONAL: the central claim is plausible and well supported but not fully verifiable from the preprint as written.
The referee's report
If the central claims hold, this is one of the more complete end-to-end demonstrations of an autonomous research loop: question selection with substantive rejection behaviour, an original theoretical derivation, exact computational verification (including 252,620 machine-checked certificates for robustness against hidden phase rotations), and an experiment with genuinely out-of-sample, precommitted predictions. The experimental side is unusually careful: the transfer matrix, decision rule, accuracy margins and controls were fixed before acquisition; the initial grid design was rejected because a Fourier interpolant would mimic the law; the first run's failure was honestly classified as inconclusive; numerical predictions were hashed before target means were computed; and standard errors were propagated with the calibration-target covariance. The measured target contrast agrees with the locked law prediction and rules out the two interpolation baselines. The mathematical claim d_L = L^2+1 is interesting and corroborated by exact symbolic checks at L=5 and L=6 and by the L=3 experiment. The open-source release of the pipeline is a concrete asset.
Pith's reading of the paper (a machine's summary, not the authors')
This paper tries to establish a physical information law and, in the same stroke, a claim about what collaborating AI agents can do in a laboratory. The law: for an inhomogeneous ensemble of quantum spins driven by phase-controlled microwave pulses of fixed duration, amplitude, and shape, exactly $L^2+1$ calibration measurements are necessary and sufficient to predict the mean response to every sequence of at most $L$ pulses — no matter how per-spin pulse errors are distributed, provided each spin's error stays constant through a sequence. The demonstration claim: a multi-agent laboratory built from large language models selected this question after rejecting 75 candidate proposals, derived the law, verified it with an independent computational pass, and designed an experiment that could separate it from plain Fourier interpolation. The agents then rejected an apparently successful run because its controls failed, revised the protocol without relaxing the acceptance criteria, and the final measurements matched the law's locked prediction while the interpolation baselines predicted zero. If both claims hold, an exponentially large family of quantum-control responses becomes predictable from a quadratically small set of calibrations, and AI agents have shown they can enforce scientific standards against their own work.
What carries the argument. The load-bearing object is the ensemble-response space $F_L$, the real span of the single-spin return probabilities $p_w(a)$ for all phase-controlled sequences of at most $L$ pulses; the theory's content is the exact dimension $\dim F_L = L^2+1$. A pulse of phase $\phi$ is the $2\times2$ unitary $U_\phi(a)$ with a per-spin error parameter $a$ that is assumed to persist across the whole sequence, and the prediction rule needs no knowledge of the distribution $\nu$: once $d_L$ linearly independent calibration sequences are chosen — in the general construction, phases from the five-point grid $2\pi k/5$ resolve the phase-Fourier components — any target expands as $p_w = \sum_j h_j(w)\,p_{b_j}$ with $\sum_j h_j = 1$, and the measured calibration means give $\hat\mu(w) = \sum_j h_j(w)\,\hat\mu(b_j)$. The experimental variant of the same idea exploits cross-length information: five three-pulse calibrations plus five shorter sequences span the $L=3$ space while leaving four withheld three-pulse targets that Fourier interpolation cannot determine, and the test contrast $S = (-T_1+T_2+T_3-T_4)/4$ is compared with the locked prediction $P = \tfrac{3}{4}(C_0-C_1+C_6) - \tfrac{1}{4}(C_2+C_3+C_4)$, both combinations cancelling the common fluorescence background.
Core claim. Restated on the authors' own terms: an inhomogeneous ensemble of nitrogen-vacancy spins in diamond is described by an unknown distribution $\nu$ of single-spin pulse errors, and the mean fluorescence response to a phase-controlled sequence $w$ is $\mu(w) = B + \kappa\int_K p_w(a)\,d\nu(a)$ with unknown background $B$ and contrast $\kappa$. The central result is that the real span of the response functions $p_w$ over all sequences of at most $L$ pulses has dimension $d_L = L^2+1$ under persistent per-spin pulse errors, so $L^2+1$ linearly independent calibration settings are both necessary and sufficient to predict every unmeasured sequence response: the calibrated means supply expansion coefficients that sum to one and therefore cancel the unknown background and contrast, while fewer settings leave a response direction unresolved and distinct admissible ensembles can agree on every calibration yet differ on an unmeasured sequence. The authors report that an independent computational pass reproduced the response-space dimensions for five- and six-pulse sequences and certified with exact rational arithmetic that hidden phase rotations up to about $1.2^\circ$ keep prediction error at the photon-noise level. Finally, they report an experiment in which the agents locked the law's predictions and acceptance criteria before analysing four withheld targets; after a first run was judged inconclusive because global-phase and reference controls failed, the revised acquisition measured a target contrast of $(40.0 \pm 0.4)\times10^{-3}$ weighted counts per shot against a locked prediction of $(39.7 \pm 1.3)\times10^{-3}$, where two interpolation rivals predicted zero.
The premise it rests on. The load-bearing premise is that every spin keeps exactly the same pulse-error parameter throughout every sequence — errors never drift with time, never depend on pulse order, and are identical in calibration and target sequences — and this persistence is never measured directly, only inferred from aggregate agreement on four withheld targets; the preprint also states the $L^2+1$ dimension result without displaying the full proof (Section 3 and Methods 7.4), so the mathematical claim cannot be checked from the text alone.
If the paper is right (Pith's inferences, not the authors')
Calibration effort grows quadratically, not exponentially, in sequence length: the exact count $L^2+1$ replaces the proposal's original upper bound $8L^2+6L+1$, cutting the requirement from 91 to 10 settings for $L=3$ while the family of possible sequences grows as $5^L$.
An entire family of unmeasured responses becomes predictable without reconstructing the unknown distribution of spin errors; the calibrated mean responses alone carry the information the law needs.
Fewer than $L^2+1$ linearly independent settings is a hard information floor: distinct physically admissible ensembles can produce identical calibration measurements yet differ on unmeasured sequences, so no more flexible predictor can guarantee recovery.
Shorter calibration sequences can carry information that three-pulse interpolation cannot: for the four withheld targets the law predicted a contrast of $(39.7\pm1.3)\times10^{-3}$ where two Fourier-based rivals predicted $0$, and the measured $(40.0\pm0.4)\times10^{-3}$ fell inside the precommitted $\pm20\%$ window.
Collaborating AI agents can police their own standards: a run that met all ten accuracy criteria was classified inconclusive because the global-phase and reference-pulse controls failed, and the revised protocol passed without loosening the acceptance criteria.
The law's scope is wider than the paper tests: the same span calculation that fixes $\dim F_L$ should yield polynomial calibration counts for other pulse families (variable amplitude, duration, or shaped pulses), a purely algebraic check that could be carried out before any further experiment.
Persistence is validated only indirectly, through aggregate predictions on four withheld targets; a direct test would introduce deliberately order-dependent or time-dependent pulse errors and check that the $L^2+1$ predictions degrade exactly when persistence fails.
The reusable lesson may be the architecture rather than the specific models: separating question generation from evaluation, with precommitted criteria and control checks, is a transferable template for other instrumented domains where rival hypotheses make quantitatively different predictions.
Reproducing the agent behaviour is harder than copying the workflow: the release omits the exact prompts used, so readers can port the pipeline structure but not the specific research behaviour it produced.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
An autonomous research laboratory could conduct an entire study, from formulating questions to preparing manuscripts. Artificial intelligence (AI) offers one route towards this goal. Recent AI systems take on several research roles, but may draw conclusions that the evidence does not support. Here, we present an autonomous research laboratory in which agents based on large language models assess one another's work and can stop or redirect investigations. They can generate and reject research questions, require separate theoretical checks and conduct experiments designed to challenge their claims. We demonstrate these capabilities by investigating ensembles of quantum spins in diamond. The agents derived the minimum number of calibration settings needed to predict responses to a defined family of microwave pulse sequences. They checked the result computationally and designed experiments to distinguish competing predictions. An initial experiment agreed with the theoretical predictions, but repeated reference measurements showed that the signal was drifting during data collection. The agents therefore treated the result as inconclusive and revised the measurement protocol without relaxing the acceptance criteria. The revised experiment met those criteria and supported the theoretical predictions. This study demonstrates that collaborating AI agents can select research questions, derive theoretical results and test their predictions experimentally.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-30
The authors' main claim, as the classifier picked it out 1.00
“This study demonstrates that collaborating AI agents can select research questions, derive theoretical results and test their predictions experimentally.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.32 · human participants 0.03 · controlled comparison 0.07 · randomized assignment 0.02 · states quantitative result 0.11 · compares with prior results 0.10 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| An autonomous research laboratory could conduct an entire study, from formulating questions to preparing manuscripts. | background 1.00 | 0.05 | 0.05 | hedged possibility 1.00 |
| Artificial intelligence (AI) offers one route towards this goal. | background 1.00 | 0.04 | 0.02 | asserted 0.62 |
| Recent AI systems take on several research roles, but may draw conclusions that the evidence does not support. | background 0.64 | 0.04 | 0.03 | hedged possibility 0.99 |
| Here, we present an autonomous research laboratory in which agents based on large language models assess one another's work and can stop or redirect investigations. | contribution 0.98 | 0.88 | 0.13 | asserted 1.00 |
| They can generate and reject research questions, require separate theoretical checks and conduct experiments designed to challenge their claims. | method 0.57 | 0.69 | 0.09 | asserted 0.87 |
| We demonstrate these capabilities by investigating ensembles of quantum spins in diamond. | contribution 0.65 | 0.82 | 0.04 | asserted 1.00 |
| The agents derived the minimum number of calibration settings needed to predict responses to a defined family of microwave pulse sequences. | result 0.90 | 0.72 | 0.05 | asserted 1.00 |
| They checked the result computationally and designed experiments to distinguish competing predictions. | method 0.85 | 0.72 | 0.04 | asserted 1.00 |
| An initial experiment agreed with the theoretical predictions, but repeated reference measurements showed that the signal was drifting during data collection. | result 0.97 | 0.87 | 0.02 | asserted 1.00 |
| The agents therefore treated the result as inconclusive and revised the measurement protocol without relaxing the acceptance criteria. | method 0.63 | 0.82 | 0.05 | asserted 1.00 |
| The revised experiment met those criteria and supported the theoretical predictions. | result 1.00 | 0.84 | 0.11 | asserted 1.00 |
| This study demonstrates that collaborating AI agents can select research questions, derive theoretical results and test their predictions experimentally. | interpretation 0.66 | 0.91 | 0.92 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 0.97 · evidence from observation or experiment 1.00
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-30: 1 request, 8,538 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (5)
Figures are the authors', reproduced from the paper.
References (23)
- [1] Ross D. King, Kenneth E. Whelan, Ffion M. Jones, Philip G. K. Reiser, Christopher H. Bryant, Stephen H. Muggleton, Douglas B. Kell, and Stephen G. Oliver. Functional genomic hypothesis generation and experimentation by a robot scientist.Nature, 427(6971):247–252, on Pith
- [2] Ross D. King, Jem Rowland, Stephen G. Oliver, Michael Young, Wayne Aubrey, Emma Byrne, Maria Liakata, Magdalena Markham, Pinar Pir, Larisa N. Soldatova, et al. The automation of science.Science, 324(5923):85–89, 2009. doi: 10.1126/science.1165620 on Pithdoi:10.1126/science.1165620
- [3] Nathan J. Szymanski, Bernardus Rendy, Yuxing Fei, Rishi E. Kumar, Tanjin He, David Milsted, Matthew J. McDermott, Max Gallant, Ekin Dogus Cubuk, Amil Merchant, et al. An autonomous laboratory for the accelerated synthesis of inorganic materials.Nature, 624 (7990):86–91, 2023. ... on Pithdoi:10.1038/s41586-023-06734-w
- [4] Benjamin Burger, Phillip M. Maffettone, Vladimir V. Gusev, Catherine M. Aitchison, Yang Bai, Xiaoyan Wang, Xiaobo Li, Ben M. Alston, Buyi Li, Rob Clowes, et al. A mobile robotic chemist.Nature, 583(7815):237–241, 2020. doi: 10.1038/s41586-020-2442-2. 15 on Pithdoi:10.1038/s41586-020-2442-2
- [5] Chris Lu, Cong Lu, Robert Tjarko Lange, Yutaro Yamada, Shengran Hu, Jakob Foerster, David Ha, and Jeff Clune. Towards end-to-end automation of AI research.Nature, 651(8107): 914–919, 2026. doi: 10.1038/s41586-026-10265-5 on Pithdoi:10.1038/s41586-026-10265-5
- [6] Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Petar Sirkovic, Artiom Myaskovsky, Grzegorz Glowaty, Felix Weissenberger, Alessio Orlandi, Dan Popovici, et al. Accelerating scientific discovery with Co-Scientist.Nature, 655(8122):487–496, 2026. doi: 10.1038/s41586-026-10644-y on Pithdoi:10.1038/s41586-026-10644-y
- [7] Ali E. Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J. Szostkiewicz, Dmytro Shved, Gavin J. Gyimesi, Jon M. Laurent, Samantha M. Wright, Muhammed T. Razzak, et al. A multi-agent system for automating scientific discovery.Nature, 655(8122): 497–505, 2026. do... on Pithdoi:10.1038/s41586-026-10652-y
- [8] Takuya Isogawa, Ryotaro Okabe, Nutdech Phadetsuwannukun, Mingda Li, and Paola Cap- pellaro. Agentic AI for scientific reasoning in autonomous quantum sensing experiments. arXiv:2607.25145v1, 2026. URLhttps://arxiv.org/abs/2607.25145 on PithPith's reviewarXiv
- [9] Shuxing Yang, Fujia Chen, Rui Zhao, Junyao Wu, Yize Wang, Haiyao Luo, Ning Han, Qiaolu Chen, Yuze Hu, Wenhao Li, et al. End-to-end autonomous scientific discovery on a real optical platform. arXiv:2604.27092v1, 2026. URLhttps://arxiv.org/abs/2604.27092 on PithPith's reviewarXiv
- [10] Xianrui Zeng, Pengfei Liu, Yirui Zang, Yang Shen, Fei Yu, Chunlei Yu, Minghao Liu, and Yang Du. Socratic agents for autonomous scientific discovery in high-dimensional physical systems. arXiv:2606.26722v1, 2026. URLhttps://arxiv.org/abs/2606.26722 on PithPith's reviewarXiv
- [11] Joeran Beel, Min-Yen Kan, and Moritz Baumgart. Evaluating Sakana’s AI Scientist: Bold claims, mixed results, and a promising future?ACM SIGIR Forum, 59(1):1–20, 2025. doi: 10.1145/3769733.3769747 on Pithdoi:10.1145/3769733.3769747
- [12] Robert K. Merton. The normative structure of science. In Norman W. Storer, editor,The Sociology of Science: Theoretical and Empirical Investigations, pages 267–278. University of Chicago Press, Chicago, 1973. Originally published as “A Note on Science and Democracy”, Journal o... on Pith
- [13] Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor. The preregistration revolution.Proceedings of the National Academy of Sciences, 115(11):2600– 2606, 2018. doi: 10.1073/pnas.1708274114 on Pithdoi:10.1073/pnas.1708274114
- [14] JosephP.Simmons, LeifD.Nelson, andUriSimonsohn. False-positivepsychology: Undisclosed flexibilityindatacollectionandanalysisallowspresentinganythingassignificant.Psychological Science, 22(11):1359–1366, 2011. doi: 10.1177/0956797611417632 on Pithdoi:10.1177/0956797611417632
- [15] Andrew Gelman and Eric Loken. The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Unpublished manuscript, November 2013. URL https://sites.stat.... on Pith
- [16] Marcus W. Doherty, Neil B. Manson, Paul Delaney, Fedor Jelezko, Jörg Wrachtrup, and Lloyd C. L. Hollenberg. The nitrogen-vacancy colour centre in diamond.Physics Reports, 528(1):1–45, 2013. doi: 10.1016/j.physrep.2013.02.001 on Pithdoi:10.1016/j.physrep.2013.02.001
- [17] C. L. Degen, F. Reinhard, and P. Cappellaro. Quantum sensing.Reviews of Modern Physics, 89(3):035002, 2017. doi: 10.1103/RevModPhys.89.035002. 16 on PithPith's reviewdoi:10.1103/revmodphys.89.035002
- [18] John F. Barry, Jennifer M. Schloss, Erik Bauch, Matthew J. Turner, Connor A. Hart, Linh M. Pham, and Ronald L. Walsworth. Sensitivity optimization for NV-diamond magnetometry. Reviews of Modern Physics, 92(1):015004, 2020. doi: 10.1103/RevModPhys.92.015004 on Pithdoi:10.1103/revmodphys.92.015004
- [19] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In International Conference on Learning Representations (ICLR 2024), 2024. URL https: //openreview.net/forum?id=Ik... on PithPith's reviewarXiv
- [20] Zhuo Li, Yupeng Zhang, Pengyu Cheng, Jiajun Song, Mengyu Zhou, Hao Li, Shujie Hu, Yu Qin, Erchao Zhao, Xiaoxi Jiang, and Guanjun Jiang. MARCH: Multi-agent reinforced check for hallucination. InProceedings of the 64th Annual Meeting of the Association for Computational Linguist... on Pithdoi:10.18653/v1/2026.acl-long.1828
show 3 more
- [21] Why AI cannot do good science without humans.Nature, 653(8115):650, 2026. doi: 10.1038/ d41586-026-01551-3 on Pith
- [22] Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can LLMs generate novel research ideas? A large-scale human study with 100+ NLP researchers. InInternational Conference on Learning Representations (ICLR 2025), 2025. URLhttps://openreview.net/forum?id=M23dTGWCZy. arXiv:2409.04109. 17 on PithPith's reviewarXiv
- [2004] doi: 10.1038/nature02236 on Pithdoi:10.1038/nature02236
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment