Pith. sign in

REVIEW 2 major objections 3 minor 42 references

Intervention-centered Auto Research—searching each possible change separately, freezing its code, then validating it on data the search loop never saw—survives on nine of ten materials-prediction benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 19:00 UTC pith:QQDM3Q27

load-bearing objection A genuinely useful audit protocol for AI-scientist search; the empirical claim is probably right but needs a contamination check and a more honest framing of the compatibility test. the 2 major comments →

arxiv 2607.17100 v2 pith:QQDM3Q27 submitted 2026-07-19 cs.MA cs.AIcs.SE

Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer

Jingjie Ning , Xiaochuan Li , Shanshan Zhong , Ji Zeng , Guolin Ke This is my paper
classification cs.MA cs.AIcs.SE
keywords intervention-centered Auto Researchheld-out transferselection regretaxis-isolated searchouter-holdout matrixmaterials property predictionlanguage-model agentsfeedback overfitting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces intervention-centered Auto Research, a workflow where a language-model agent proposes changes confined to one declared axis—features, model, representation, or data—and each axis is searched independently under inner five-fold cross-validation. The best code on each axis is frozen, and then all alternatives are validated together on an outer holdout that the search loop never saw. Across ten materials-prediction benchmarks and 701 agent-executed attempts, the inner feedback selected the best tested intervention on nine outer holdouts, with 0.228 percentage-point mean selection regret and 89.3% agreement between inner and outer orderings. The audit also rejected one structure representation change that inner feedback alone would have endorsed. This design makes the research decision, not the terminal pipeline score, the unit of evidence, and it exposes which technical changes remain reusable after adaptive search.

Core claim

On the paper's own terms: a method that isolates interventions into four axes, freezes each selected code change, and evaluates the full set of frozen alternatives on a once-only outer holdout can separate search-driven overfitting from genuine transfer. The evidence shows that inner five-fold feedback picks the best tested intervention on nine of ten endpoints, with mean regret of 0.228 percentage points and pairwise ordering agreement of 89.3%. The one disagreement—two-dimensional exfoliation, the smallest structure holdout—produced 2.279 percentage-point regret. The audit also found an information-dependent hierarchy: on composition-only tasks, several axes worked; on structure-informed t

What carries the argument

The central mechanism is the frozen-code outer-holdout matrix. Each campaign searches one declared axis (Feature, Model, Representation, or Data) using only inner five-fold feedback; the selected code, configuration, and a SHA-256 manifest are frozen before any outer-holdout label is evaluated. Then a once-only outer-holdout matrix compares every frozen alternative against the baseline, producing selection regret, pairwise ordering agreement, endpoint breadth, and compatibility. This design makes adaptive search auditable because the validation evidence is structurally separated from the feedback that shaped the candidate.

Load-bearing premise

The entire audit rests on the claim, stated in the outer-holdout protocol, that the holdout labels never reach the agent process; if the pretrained language model's own training data already contained those labels and leaked them into its proposed code, the 'unseen' validation is contaminated and the 9-of-10 transfer count is inflated.

What would settle it

Re-run the same seven campaigns against a freshly collected materials-property holdout that was published after the agent's training cutoff, or use an agent whose pretraining provably excludes the benchmark labels; if the 9-of-10 transfer count collapses to chance on the replacement holdout, the reported transfer is leakage—if it persists, the audit is real.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Terminal pipeline scores can hide which technical decision carried a gain; the outer-holdout matrix attributes improvement to a specific axis.
  • Nine of ten inner-selected interventions remained best on untouched holdouts, implying that five-fold inner feedback transfers despite repeated reuse.
  • On structure tasks, separately found Feature and Model changes are complementary: their union improved all six structure endpoints and raised mean improvement from 19.0% to 26.3%.
  • The rejected Representation intervention—positive inside the loop, negative outside—is a concrete example of how single-loop validation can endorse an overfit change.
  • Information availability, not input dimension, predicts which interventions help: additional composition features did not replace sparse structural descriptors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If replicated in other domains, this decision-level audit could become a standard evaluation pattern for agentic research, replacing single terminal scores with selection regret and ordering checks.
  • A direct extension would compare these language-model agents against fixed random or heuristic search on the same axes to measure how much of the transfer comes from agent proposals versus the validation protocol itself.
  • The sign-reversal example suggests practical Auto Research should always reserve a small untouched audit set before trusting any agent-endorsed change, especially when task samples are small.
  • Because the official fold-0 replay reuses examples from the search pool, one could rerun the audit on multiple genuinely fresh holdouts to estimate variance in the 9-of-10 transfer count.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper introduces intervention-centered Auto Research, a workflow in which an LLM agent searches independently along Feature, Model, Representation, and Data axes using inner five-fold feedback, freezes each selected intervention, and then evaluates all frozen interventions on a Matbench outer holdout. The central empirical claim is that inner feedback selects the best tested intervention on nine of ten endpoints, with 0.228 percentage-point mean selection regret and 89.3% pairwise ordering agreement. The paper also reports a structure Representation intervention that inner feedback endorsed but the outer holdout rejected, and a post-hoc Feature+Model compatibility assembly that improves mean outer-holdout MAE from 19.0% to 26.3%.

Significance. If the 9-of-10 transfer result holds up, this is a meaningful methodological contribution: it provides a clean separation between adaptive search feedback and validation, makes individual research decisions the unit of evidence, and demonstrates an audit that rejects an overfit intervention. The protocol's code-freeze/SHA-256 manifest, once-only outer-holdout evaluation, and exact blocked permutation test are careful and creditable design choices. The negative Representation result is a useful illustration that outer validation can overturn inner feedback. The main uncertainty is whether the outer holdout is genuinely unseen by the pretrained LLM, which the current manuscript does not establish.

major comments (2)
  1. [§4, 'Outer-holdout protocol' and 'Agent and compute'] The core transfer claim depends on the outer holdout being genuinely unseen. The protocol only prevents the agent process from loading outer-holdout labels; it does not address memorization by DeepSeek-V4-Pro, a pretrained LLM. Matbench fold-0 targets are public and could appear in training data, so code suggestions could be implicitly conditioned on those targets. No contamination check (model cutoff, memorization probe, or decontamination) is reported. Thus the 9-of-10 count, 0.228 pp regret, and 89.3% agreement are not yet established as out-of-sample evidence. This is fixable by a private holdout or a documented cutoff/audit.
  2. [§5, 'Feature and Model Changes Remain Compatible' and Table 3] The F+M compatibility result is selected after inspecting the outer-holdout matrix: the complementary profiles in that matrix motivate the deterministic union, and the same matrix is then used to score it. This is selection on the test set, so the 26.3% mean improvement is an optimistic post-selection estimate, not a fresh held-out result. The paper excludes it from the primary 9-of-10 claim, but the Abstract and Table 3 still present it as a headline. It should be reframed as exploratory, or validated on a new split if it is to support a compatibility claim.
minor comments (3)
  1. [§4 'Materials Instantiation' vs §5 'Input type, not input dimension'] The structure baseline is described as 140 inputs (132 Magpie descriptors plus an eight-descriptor block), while the controlled comparison uses 141 inputs (132 plus nine light structure descriptors). Please reconcile this discrepancy.
  2. [Abstract and §3] The phrase 'evidence the loop never sees' is overstrong given the pretraining-leakage concern. Suggest 'evidence never loaded into the agent process' or an explicit caveat, in addition to addressing the underlying issue.
  3. [§5 'Search dynamics'] The text reports 701 attempts and 699 scored attempts; this is clear in the body, but consider noting it in the relevant figure/table caption to avoid an apparent inconsistency.

Circularity Check

1 steps flagged

The 9-of-10 held-out selection result is not circular, but the F+M compatibility test is selected on the outer-holdout matrix and then scored on the same matrix.

specific steps
  1. fitted input called prediction [Section 5, 'Feature and Model Changes Remain Compatible' (near Table 3)]
    "The complementary profiles in the completed single-axis matrix motivate one deterministic compatibility test without further search or tuning. Their union improves all six structure tasks and reduces mean outer-holdout MAE by 26.3%, compared with 19.0% for the best single intervention and 14.6% for Feature alone."

    The 'completed single-axis matrix' is the outer-holdout matrix. Reading complementary profiles from it means deciding to evaluate the Feature+Model assembly because Feature won on phonons/perovskites and Model won on elastic/refractive on the outer holdout. The 'union improves all six structure tasks' is then measured on the same outer-holdout labels that motivated the assembly. The F+M result is therefore not an independent test of compatibility; it is an in-sample evaluation of a test-selected hypothesis. This is selection on the test set (the reusable-holdout problem the paper itself cites via Dwork et al.), reducing the compatibility claim to the matrix it was chosen from.

full rationale

The primary claim—inner five-fold feedback selects the best tested intervention on nine of ten outer holdouts—is not circular. Equations (1)-(4) define selection entirely from inner folds; the outer-holdout h_t enters only after freezing, and the permutation null is computed from relabelings of those frozen choices. Under the stated protocol, outer labels are not used for the 9-of-10 result. The LLM-pretraining-leakage concern is a contamination risk about the agent's priors, not a derivation that reduces the claim to its inputs. The one genuinely circular step is the F+M compatibility test: the outer-holdout matrix motivates the assembly and the same matrix supplies its score. Because the paper explicitly excludes the assembly from the primary 9-of-10 result, this is a partial, localized circularity rather than a collapse of the whole method.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The method introduces no new physical or ontological entities. Its free parameters are experimental design choices: fold count, attempt budget, normalization rule, and baseline descriptors. The largest burden is the assumption that the once-only outer holdout is truly unseen by the agent, which is not established because the agent is a public-benchmark-trained LLM. The controlled and compatibility analyses add post-hoc selection burden.

free parameters (4)
  • K=5 inner folds = 5
    Fixed number of cross-validation folds used for agent feedback (Eq. 2); affects selection noise and is chosen by hand, not inferred.
  • 100-attempt stopping rule per campaign = 100
    Stopping rule for each axis campaign; the frozen intervention depends on this budget and the order of attempts.
  • Direction-corrected normalized improvement = relative MAE/ROC-AUC change (Eq. 1)
    Normalization choice makes MAE and ROC-AUC commensurable across endpoints; different normalization would change which candidate is selected.
  • Baseline descriptor sets = 132 Magpie inputs (composition), 140 with density-symmetry block (structure)
    The baseline contract is chosen by the authors and defines the reference b_{t,k} for all reported improvements.
axioms (4)
  • domain assumption Outer-holdout labels have not influenced frozen code, including via LLM pretraining.
    The central held-out claim requires this; the paper only prevents direct loading of labels into the agent process (Section 4).
  • domain assumption The five-fold inner feedback is a stable selection signal that generalizes to the once-only outer holdout.
    Selection regret and ordering claims presuppose this stability; no repeated outer-evaluation is provided to assess split variance.
  • ad hoc to paper Direction-corrected normalized improvement makes MAE and ROC-AUC comparable across endpoints.
    Eq. (1) defines the objective the agent sees; it is a modeling choice specific to this paper and not forced by the benchmark.
  • domain assumption Files outside an assigned axis remain byte-identical, verified by hashes.
    Axis isolation is required to attribute gains to a single intervention; the paper states hashes verify this but does not ship the manifest.

pith-pipeline@v1.3.0-alltime-deepseek · 10839 in / 14041 out tokens · 129451 ms · 2026-08-01T19:00:45.871424+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer." pith.science (2026). https://pith.science/paper/QQDM3Q27

@misc{pith2026260717100,
  author       = {Pith},
  title        = {Pith review of: Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QQDM3Q27}},
  note         = {Machine review of arXiv:2607.17100}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Auto Research uses language-model agents to propose, implement, and evaluate machine-learning changes in a closed loop, but is usually judged by its terminal pipeline. A terminal score cannot reveal which technical decision produced a gain or distinguish a reusable discovery from a change adapted to development feedback. We introduce intervention-centered Auto Research, which validates research decisions rather than only final artifacts and makes their reliability measurable. Feature, Model, Representation, and Data axes are searched independently with inner five-fold feedback. Each axis winner is frozen before an outer-holdout matrix compares all alternatives on evidence the loop never sees. Across 701 agent-executed attempts spanning ten Matbench endpoints, outer evidence confirms the selected intervention on nine of ten endpoints and preserves 89.3\% of non-tied intervention orderings. It also rejects an aggregate Representation gain that inner feedback endorsed. The resulting matrix reveals an information-dependent hierarchy. Composition-only tasks support several routes to improvement, whereas structure-informed tasks favor local geometry features and complementary tree ensembles. A subsequent compatibility test combines already frozen Feature and Model code without further search or tuning and raises mean outer-holdout improvement from 19.0\% to 26.3\%. By validating decisions rather than only artifacts, this design turns adaptive search into reusable evidence wherever agents propose executable alternatives against a fixed evaluator.

Figures

Figures reproduced from arXiv: 2607.17100 by Guolin Ke, Jingjie Ning, Ji Zeng, Shanshan Zhong, Xiaochuan Li.

Figure 1
Figure 1. Figure 1: Overview of intervention-centered Auto Research. Independent Feature, Model, Representation, and Data searches [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Intervention selection on ten outer holdouts. Each row reports the effect of the frozen intervention chosen by inner [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Intervention hierarchies depend on input. Composition supports several useful changes, while structure separates [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references

  1. [1]

    npj Computational Materials , volume =

    Dunn, Alexander and Wang, Qi and Ganose, Alex and Dopp, Daniel and Jain, Anubhav , title =. npj Computational Materials , volume =. 2020 , doi =

    2020

  2. [2]

    Riebesell, Janosh and Goodall, Rhys E. A. and Benner, Philipp and Chiang, Yuan and Deng, Bowen and Ceder, Gerbrand and Asta, Mark and Lee, Alpha A. and Jain, Anubhav and Persson, Kristin A. , title =. Nature Machine Intelligence , volume =. 2025 , doi =

    2025

  3. [3]

    npj Computational Materials , volume =

    Ward, Logan and Agrawal, Ankit and Choudhary, Alok and Wolverton, Christopher , title =. npj Computational Materials , volume =. 2016 , doi =

    2016

  4. [4]

    Ward, Logan and Dunn, Alexander and Faghaninia, Alireza and Zimmermann, Nils E. R. and Bajaj, Saurabh and Wang, Qi and Montoya, Joseph and Chen, Jiming and Bystrom, Kyle and Dylla, Maxwell and Chard, Kyle and Asta, Mark and Persson, Kristin and Snyder, G. Jeffrey and Foster, Ian and Jain, Anubhav , title =. Computational Materials Science , volume =. 2018 , doi =

    2018

  5. [5]

    Advances in Neural Information Processing Systems , volume =

    Prokhorenkova, Liudmila and Gusev, Gleb and Vorobev, Aleksandr and Dorogush, Anna Veronika and Gulin, Andrey , title =. Advances in Neural Information Processing Systems , volume =

  6. [6]

    Advances in Neural Information Processing Systems , volume =

    Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan , title =. Advances in Neural Information Processing Systems , volume =

  7. [7]

    npj Computational Materials , volume =

    De Breuck, Pierre-Paul and Hautier, Geoffroy and Rignanese, Gian-Marco , title =. npj Computational Materials , volume =. 2021 , doi =

    2021

  8. [8]

    and Ceder, Gerbrand and Jain, Anubhav , title =

    Tshitoyan, Vahe and Dagdelen, John and Weston, Leigh and Dunn, Alexander and Rong, Ziqin and Kononova, Olga and Persson, Kristin A. and Ceder, Gerbrand and Jain, Anubhav , title =. Nature , volume =. 2019 , doi =

    2019

  9. [9]

    Chemistry of Materials , volume =

    Chen, Chi and Ye, Weike and Zuo, Yunxing and Zheng, Chen and Ong, Shyue Ping , title =. Chemistry of Materials , volume =. 2019 , doi =

    2019

  10. [10]

    Connectivity Optimized Nested Line Graph Networks for Crystal Structures , journal =

    Ruff, Robin and Reiser, Patrick and St. Connectivity Optimized Nested Line Graph Networks for Crystal Structures , journal =. 2024 , doi =

    2024

  11. [11]

    Goodall, Rhys E. A. and Lee, Alpha A. , title =. Nature Communications , volume =. 2020 , doi =

    2020

  12. [12]

    and Murdock, Ryan J

    Wang, Anthony Yu-Tung and Kauwe, Steven K. and Murdock, Ryan J. and Sparks, Taylor D. , title =. npj Computational Materials , volume =. 2021 , doi =

    2021

  13. [13]

    , title =

    Xie, Tian and Grossman, Jeffrey C. , title =. Physical Review Letters , volume =. 2018 , doi =

    2018

  14. [14]

    npj Computational Materials , volume =

    Choudhary, Kamal and DeCost, Brian , title =. npj Computational Materials , volume =. 2021 , doi =

    2021

  15. [15]

    Science , volume =

    Dwork, Cynthia and Feldman, Vitaly and Hardt, Moritz and Pitassi, Toniann and Reingold, Omer and Roth, Aaron , title =. Science , volume =. 2015 , doi =

    2015

  16. [16]

    and Talbot, Nicola L

    Cawley, Gavin C. and Talbot, Nicola L. C. , title =. Journal of Machine Learning Research , volume =

  17. [17]

    Advances in Neural Information Processing Systems , volume =

    Feurer, Matthias and Klein, Aaron and Eggensperger, Katharina and Springenberg, Jost Tobias and Blum, Manuel and Hutter, Frank , title =. Advances in Neural Information Processing Systems , volume =

  18. [18]

    and Le, Quoc V

    Real, Esteban and Liang, Chen and So, David R. and Le, Quoc V. , title =. Proceedings of the 37th International Conference on Machine Learning , series =. 2020 , publisher =

    2020

  19. [19]

    Pawan and Dupont, Emilien and Ruiz, Francisco J

    Romera-Paredes, Bernardino and Barekatain, Mohammadamin and Novikov, Alexander and Balog, Matej and Kumar, M. Pawan and Dupont, Emilien and Ruiz, Francisco J. R. and Ellenberg, Jordan S. and Wang, Pengming and Fawzi, Omar and Kohli, Pushmeet and Fawzi, Alhussein , title =. Nature , volume =. 2024 , doi =

    2024

  20. [20]

    Proceedings of the 41st International Conference on Machine Learning , series =

    Huang, Qian and Vora, Jian and Liang, Percy and Leskovec, Jure , title =. Proceedings of the 41st International Conference on Machine Learning , series =

  21. [21]

    The Thirteenth International Conference on Learning Representations , year =

    Chan, Jun Shern and Chowdhury, Neil and Jaffe, Oliver and Aung, James and Sherburn, Dane and Mays, Evan and Starace, Giulio and Liu, Kevin and Maksin, Leon and Patwardhan, Tejal and Weng, Lilian and M. The Thirteenth International Conference on Learning Representations , year =

  22. [22]

    Nature , volume =

    Lu, Chris and Lu, Cong and Lange, Robert Tjarko and Yamada, Yutaro and Hu, Shengran and Foerster, Jakob and Ha, David and Clune, Jeff , title =. Nature , volume =. 2026 , doi =

    2026

  23. [23]

    2025 , howpublished =

    Jiang, Zhengyao and Schmidt, Dominik and Srikanth, Dhruv and Xu, Dixing and Kaplan, Ian and Jacenko, Deniss and Wu, Yuxiang , title =. 2025 , howpublished =

    2025

  24. [24]

    Novikov, Alexander and Vu, Ngan and Eisenberger, Marvin and Dupont, Emilien and Huang, Po-Sen and Wagner, Adam Zsolt and Shirobokov, Sergey and Kozlovskii, Borislav and Ruiz, Francisco J. R. and Mehrabian, Abbas and Kumar, M. Pawan and See, Abigail and Chaudhuri, Swarat and Holland, George and Davies, Alex and Nowozin, Sebastian and Kohli, Pushmeet and Ba...

    2025

  25. [25]

    2026 , howpublished =

    Gao, Shanghua and Fang, Ada and Zitnik, Marinka , title =. 2026 , howpublished =

    2026

  26. [26]

    Agentomics: An Agentic System that Autonomously Develops Novel State-of-the-Art Solutions for Biomedical Machine Learning Tasks , journal =

    Martinek, Vlastimil and Gariboldi, Andrea and Tzimotoudis, Dimosthenis and Galea, Mark and Zacharopoulou, Elissavet and Alberdi Escudero, Aitor and Blake, Edward and. Agentomics: An Agentic System that Autonomously Develops Novel State-of-the-Art Solutions for Biomedical Machine Learning Tasks , journal =. 2026 , doi =

    2026

  27. [27]

    Prior Work on Closed-Loop Auto Research for Training Recipes , year =

  28. [28]

    Prior Work on Axis-Isolated Auto Research for Molecular Property Prediction , year =

  29. [29]

    2024 , howpublished =

    Jia, Shuyi and Zhang, Chao and Fung, Victor , title =. 2024 , howpublished =

    2024

  30. [30]

    , title =

    Ghafarollahi, Alireza and Buehler, Markus J. , title =. npj Computational Materials , volume =. 2026 , doi =

    2026

  31. [31]

    and Matanovic, Ivana and Li, Baikun and Holby, Edward F

    Rothfarb, Samuel and Davis, Megan C. and Matanovic, Ivana and Li, Baikun and Holby, Edward F. and Kort-Kamp, Wilton J. M. , title =. npj Computational Materials , volume =. 2026 , doi =

    2026

  32. [32]

    and Doherty, Tiarnan and Tigas, Panagiotis and Razzak, Muhammed and Roberts, Stephen and Walsh, Aron and Gal, Yarin , title =

    Malik, Shreshth A. and Doherty, Tiarnan and Tigas, Panagiotis and Razzak, Muhammed and Roberts, Stephen and Walsh, Aron and Gal, Yarin , title =. Proceedings of the 43rd International Conference on Machine Learning , year =

  33. [33]

    Huang, Ziyang and Cao, Yi and Shargh, Ali K. and Luo, Jing and Mei, Ruidong and Zaki, Mohd and Liu, Zhan and Bunstine, Wyatt and Jurayj, William and Goswami, Somdatta and McQueen, Tyrel and Shields, Michael and El-Awady, Jaafar and Clancy, Paulette and Van Durme, Benjamin and Andrews, Nicholas and Walden, William and Khashabi, Daniel , title =. 2026 , how...

    2026

  34. [34]

    and Rendy, Bernardus and Fei, Yuxing and Kumar, Rishi E

    Szymanski, Nathan J. and Rendy, Bernardus and Fei, Yuxing and Kumar, Rishi E. and He, Tanjin and Milsted, David and McDermott, Matthew J. and Gallant, Max and Cubuk, Ekin Dogus and Merchant, Amil and Kim, Haegyeom and Jain, Anubhav and Bartel, Christopher J. and Persson, Kristin and Zeng, Yan and Ceder, Gerbrand , title =. Nature , volume =. 2023 , doi =

    2023

  35. [35]

    and MacKnight, Robert and Kline, Ben and Gomes, Gabe , title =

    Boiko, Daniil A. and MacKnight, Robert and Kline, Ben and Gomes, Gabe , title =. Nature , volume =. 2023 , doi =

    2023

  36. [36]

    Proceedings of the 36th International Conference on Machine Learning , series =

    Recht, Benjamin and Roelofs, Rebecca and Schmidt, Ludwig and Shankar, Vaishaal , title =. Proceedings of the 36th International Conference on Machine Learning , series =. 2019 , publisher =

    2019

  37. [37]

    and Moore, Jason H

    Olson, Randal S. and Moore, Jason H. , title =. Proceedings of the Workshop on Automatic Machine Learning , series =. 2016 , publisher =

    2016

  38. [38]

    2020 , howpublished =

    Erickson, Nick and Mueller, Jonas and Shirkov, Alexander and Zhang, Hang and Larroy, Pedro and Li, Mu and Smola, Alexander , title =. 2020 , howpublished =

    2020

  39. [39]

    ACM Computing Surveys , volume =

    Zha, Daochen and Bhat, Zaid Pervaiz and Lai, Kwei-Herng and Yang, Fan and Jiang, Zhimeng and Zhong, Shaochen and Hu, Xia , title =. ACM Computing Surveys , volume =. 2025 , doi =

    2025

  40. [40]

    2026 , howpublished =

    Ning, Jingjie and Li, Xueqi and Yu, Chengyu , title =. 2026 , howpublished =

    2026

  41. [41]

    2026 , howpublished =

    2026

  42. [42]

    Zhong, Shanshan and Lu, Yi and Ning, Jingjie and Wan, Yibing and Feng, Lihan and Ao, Yuyi and Ribeiro, Leonardo F. R. and Dreyer, Markus and Ammirati, Sean and Xiong, Chenyan , title =. 2026 , howpublished =

    2026