Pith. sign in

REVIEW 3 major objections 5 minor 22 references

Idea Search: Guiding Tree Search with Ideas to Explore Diverse Scientific Methods

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Guiding Tree Search with a dynamic Idea Bank of decomposed method components reliably breaks its plateau on scRNA-seq batch integration, lifting the mean score from 0.678 to 0.697 and the best solution to 0.728.

desk verdict A plausible, honestly-reported method with a real missing control: the 'dynamic' idea bank is never compared to a frozen one, so the central mechanism is unproven despite a small but consistent score gain. read the letter →

arxiv 2608.08958 v1 pith:WRWN4YYE submitted 2026-08-09 cs.LG cs.AIq-bio.GNq-bio.QM

Xuefei Julie Wang , Hao Cui , Michael P. Brenner , Subhashini Venugopalan This is my paper · ORCID
classification cs.LGcs.AIq-bio.GNq-bio.QM
keywords IdeaSearchTreetest-timescalingLLMcodegenerationscRNA-seqbatchintegrationBankbanditsamplingautomatedscientificdiscovery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Idea Search claims that test-time scaling of LLMs by Tree Search can be systematically guided by a dynamic 'Idea Bank' of atomic method components, and that doing so breaks the performance plateau of an unguided strong Tree Search baseline. On single-cell RNA-sequencing batch integration, the framework raises the mean score from 0.678 to 0.697 (bandit and random sampling both reach about 0.697) and the best discovered solution from 0.694 to 0.728. The paper argues that the gains are attributable to the idea mechanism rather than to uniform extra computation, since design choices reshape where gains appear: augmenting the bank helps bandit sampling but not random sampling, an exploratory prompt surfaces rare best solutions, and increasing bandit exploration (alpha equals 4) hurts. The evidence is deliberately narrow: one task, one backbone LLM, validation-split scoring, and five trials per configuration.

What carries the argument

The load-bearing object is the Idea Bank: a growing set of short textual descriptions of atomic, recombinable design choices (for example, 'condition the decoder on batch ID'), each maintained with an execution-verified score. Sampling is either uniform or bandit-based using a rank-adjusted UCB score $S_i = 1/r_i + \alpha \sqrt{N \ln(1+T)/(1+t_i)}$, where $r_i$ is the rank of idea $i$ by mean score, $t_i$ is its implementation count, $T = \sum_j t_j$, and $\alpha$ controls exploration. The bank is initialized by decomposing expert methods (optionally augmented by LLM brainstorming) and expanded by decomposing every newly discovered solution. This machinery carries the argument because it turns code-level search into explicitly idea-level search, and because the paper's ablations vary exactly these components—sampling strategy, augmentation, and prompting—to locate the source of the gains.

What would settle it

Run the same Idea Search loop but replace the sampled textual ideas with semantically irrelevant or shuffled labels while holding prompts, bank size, and update dynamics fixed; if the plateau-breaking and outlier gains persist, the effect is not driven by idea content. Conversely, an ideal test would show score gains vanish when ideas are randomly misassigned to solutions.

Watch

Extended reading notes

Core claim

The central claim is that decomposing expert methods into reusable atomic ideas, storing them in a bank, and feeding sampled ideas into mutation prompts closes the loop between conceptual search and code search, enabling an automated framework to escape plateaus that pure Tree Search cannot. Idea Search runs a tree search in which each mutation is conditioned on one or more sampled ideas; after the child solution is executed and scored, an LLM decomposes it into ideas and updates the bank, with each idea's score stored as the average score of all solutions containing it. Under this loop, the paper reports reliable plateau-breaking: mean 0.678 ± 0.011 to 0.697, and best 0.728 versus the baseline's 0.694. The paper further claims that the design choices interact: the augmented bank benefits bandit sampling but not random sampling, exploratory prompting yields rare outlier solutions, and higher sampling-level exploration (alpha equals 4) is counterproductive.

Load-bearing premise

The load-bearing premise is that an LLM's decomposition of a method into atomic ideas is faithful enough that the average score of all solutions containing an idea measures that idea's true contribution; if the decomposition is arbitrary or biased, the bank's guidance degenerates into random prompt perturbation.

Editorial extensions

If this is right

  • If the claim is right, a pure Tree Search baseline's plateau is not a fixed ceiling; injecting execution-verified conceptual guidance from a dynamic bank shifts both the average and the best case.
  • The interaction findings imply that the best configuration is not simply 'more ideas' or 'more exploration': bank augmentation pays off only with score-guided sampling, and sampling-level exploration should be kept low while prompting-level exploration is increased.
  • Because gains concentrate in rare outliers, evaluations of such systems should track best-of-n and distributional tails, not just mean scores, to capture the discovery value.
  • The same loop could apply wherever a scientific method can be summarized as a recombination of ideas and scored by execution, making the framework a candidate general test-time architecture for automated method development.
  • The paper's own scope caveats mean these are directional effects on one validation benchmark, not calibrated effect sizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension beyond the paper would be to replace the coarse average-score credit with counterfactual or Shapley-style credit across co-occurring ideas, which could sharpen which idea actually drove a gain.
  • If idea-level guidance is what matters, the same bank could be reused across tasks or domains; a cheap test would be to run Idea Search on a second benchmark using the same bank without retraining.
  • The finding that exploratory prompting plus random sampling produced the best outlier suggests that novelty is generated by implementation-level freedom, not by more bandit exploration; an untested design would fix alpha very low and rely on prompts for diversity.
  • The mechanism resembles a form of representation learning over method space: the bank is a compressed, human-readable latent space where each dimension is an executable idea; one could test this by measuring whether bank growth correlates with diversity of discovered solutions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Idea Search, a tree-search framework for automated scientific code generation in which an explicit, human-readable "Idea Bank" is used to guide LLM mutations. Expert methods are decomposed into atomic ideas, an idea is sampled (randomly or via a rank-based UCB bandit) and injected into the mutation prompt, and the bank is dynamically updated by decomposing newly generated solutions and updating each idea's execution-verified score. The method is evaluated on scRNA-seq batch integration against a pure Tree Search baseline from Aygün et al. (2026). The central empirical claims are that Idea Search reliably breaks the baseline plateau (mean score from 0.678 to roughly 0.697, best score 0.728), and that design choices interact: bank augmentation helps bandit sampling but not random sampling, exploratory prompting produces rare high-scoring outliers, and increasing the bandit exploration parameter alpha is counterproductive. The manuscript includes an explicit Scope paragraph and a Limitations section acknowledging the small effect size, single task, single LLM, and coarse credit assignment.

Significance. If the central claim holds, Idea Search is a useful step toward making conceptual exploration a first-class component of test-time scaling: it maintains an explicit, executable, and dynamically updated representation of scientific ideas, and the paper includes enough detail in the appendix (prompts, full idea lists, scoring protocol reference) to reproduce the setup. The authors are commendably explicit about the narrowness of the evidence and the small effect size relative to trial-to-trial variance. The main value is the formulation and the clean identification of design-choice interactions, not a definitive effect-size estimate. However, the paper does not isolate the dynamic-update mechanism from static idea injection, and the statistical support for "reliably" is weak given five trials per configuration. If the dynamic mechanism is inert, the contribution shrinks to a static curated list plus prompt conditioning, so the missing control is load-bearing rather than cosmetic.

major comments (3)
  1. [Algorithm 1 and Section 5] The defining novelty of the method is the dynamic Idea Bank: Algorithm 1 lines 13-16 insert newly discovered ideas and update each idea's score and rank from executed solutions, and the Discussion credits "systematically injecting execution-verified ideas." Yet no experiment in Section 5 ablates these updates. Every reported configuration varies the sampling rule, the initial bank, the prompting style, or alpha, but none compares the full dynamic loop to a frozen initial bank with identical sampling and prompting. This matters because random sampling, which ignores all score feedback, reaches the same mean as the bandit in Section 5.1, and the augmented-bank random configuration produces the single best score of 0.728. A missing control for the mechanism is not an internal inconsistency, but it means the central claim that a dynamically updated bank drives the gain is not established by the reported experiments. I request an experiment that freezes the initial bank and disables score updates, matching all other components, so the contribution of new-idea insertion and execution-verified re-ranking can be separated from static idea prompting.
  2. [Sections 5.1, 5.2, and 5.3; Scope paragraph] The statistical support for the headline claim is weak. The mean improvement over baseline is about 0.02 while the reported standard deviations across five trials are 0.008-0.018, and the paper's own Scope paragraph concedes that the gain is comparable to trial-to-trial variance. The statement that Idea Search "reliably breaks the plateau" is not supported by any significance test, confidence interval, or power analysis, and with n=5 per configuration it is hard to distinguish a true shift from sampling noise. In addition, the alpha-comparison in Section 5.3 reports a default mean of 0.712 +/- 0.012, which matches the Random-sampling Expert-Only configuration in Section 5.2 rather than the bandit means reported in Sections 5.1 and 5.2; the precise configuration for the alpha sweep is not identified. Please report the exact configuration for each comparison, add confidence intervals or a paired test across matched trials, and, because the paper argues the signal is in rare outliers, report the distribution of best-of-n scores rather than only the mean.
  3. [Algorithm 1 lines 13-16 and Limitations] The execution-verified idea-score mechanism relies on the assumption that the mean score over all solutions containing an idea reflects that idea's contribution, but the paper's Limitations section correctly notes that credit is assigned coarsely when several ideas co-occur and that the decomposition inherits LLM bias. This is a genuine correctness risk for the bandit mechanism: if the decomposition is unfaithful or credit is diluted by co-occurrence, the bandit's rank-based scores may simply track prompt artifacts rather than the actual causal effect of the idea. The manuscript does not provide any sensitivity analysis, such as measuring decomposition stability across repeated LLM calls, ablating the score-update term while keeping new-idea insertion, or comparing rank-based UCB against a version that uses raw mean scores. I ask for at least one such analysis so that the reader can gauge how much of the reported effect is attributable to the idea-credit assignment rather than to the prompt's general encouragement to try new concepts.
minor comments (5)
  1. [Section 4.2, Eq. (1)] The exploration term is written as alpha/N times the square-root term; please clarify whether N is the current bank size and how the rank r_i is defined for ties and for unseen ideas, since the paper states that unseen ideas get maximum score but does not specify the corresponding rank used in the 1/r_i term.
  2. [Section 4 and Section 5] The experimental setup does not state the mutation budget b, the number of tree nodes or LLM calls per trial, the seed solution, or the exact tree-expansion policy; without these values the reported curves and the comparison across designs are not reproducible.
  3. [Appendix A.2] The Augmented Bank list contains duplicate entries (e.g., "Minimizing mutual information between latent space and source ID" and "Regressing out the source ID from principal components" each appear twice), which should be deduplicated because duplicates effectively change sampling weights in random sampling.
  4. [Section 5.1, Figure 2] The text says the baseline "typically plateaus early (around 300 nodes)" while Idea Search breaks the plateau "around 500 nodes," but the figure is not visible in the text; please either include the plot in the main text or describe the node-axis and the number of trials per curve in the caption.
  5. [Section 3, Algorithm 1] The prompt examples in Appendix A.3 are helpful, but the manuscript would benefit from stating whether the sampled ideas are always all injected into the prompt or whether the LLM is allowed to ignore incompatible ideas; this affects how faithfully "the idea is implemented" is tracked in the score update.

Circularity Check

0 steps flagged · score 2.0 of 10

No definitional or fitted-input circularity; the core comparison is externally scored, though the baseline and expert summaries come from group-internal prior work and the dynamic-update mechanism is never ablated.

full rationale

The central claim is an empirical comparison, not a derivation. Idea Search's scores (Algorithm 1 lines 13-16) are computed from executed solutions and then used to guide subsequent mutations; this is the intended closed-loop feedback, not an identity between input and output. Eq. 1 is a UCB sampling heuristic that only affects which idea is tried next; nothing in it defines the reported 0.697 mean or 0.728 max in terms of the bank contents. The baseline is run separately without idea injection (Section 5.1) and all scores follow the OpenProblems v2.0.0 validation protocol, so the measured gain is externally anchored. The main caveat is provenance rather than circularity: the pure-Tree-Search framework, the task preprocessing, and the expert method summaries are taken from Aygün et al. 2026, which shares an author with this paper (Hao Cui), making the plateau claim group-internal. Additionally, no experiment freezes the bank to test whether the dynamic updates contribute beyond static idea injection plus prompting; that is a missing control for the mechanism, not a step that reduces to its own inputs. The Limitations candidly acknowledge that idea scores are coarse averages over co-occurring ideas and inherit LLM decomposition bias, but this is an assumption about credit assignment, not circular reasoning. Under the rule that only constructional equivalences count as circularity, no such equivalence is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The method rests on the faithfulness of LLM decomposition, the usefulness of mean-score credit assignment, the strength of the group-internal baseline, and the sufficiency of five trials; the only free hyperparameter is the bandit exploration alpha, set a priori.

free parameters (1)
  • alpha (exploration parameter in bandit scoring) = 1 (default), 4 (tested)
    In Eq. (1), a hyperparameter controlling exploration-exploitation balance; set a priori to 1, and an alternate value 4 is tested to characterize sensitivity. It is not fitted to maximize scores.
assumptions (4)
  • domain assumption Expert methods can be decomposed by an LLM into atomic, recombinable, executable ideas.
    The entire mechanism relies on the decomposition (Fig. 1, Algorithm 1 line 1) yielding units that survive recombination in code mutations.
  • domain assumption The score of an idea, as the mean score of all solutions containing it, is a useful credit assignment for guiding subsequent samples.
    Algorithm 1 lines 14-16; acknowledged in Limitations as coarse when ideas co-occur.
  • domain assumption The pure Tree Search baseline from Aygün et al. (2026) is a strong and fairly reproduced baseline.
    All gains are measured relative to this baseline; the paper reuses the filtering and scoring pipeline from the reference.
  • domain assumption Five independent trials per configuration are sufficient to compare configurations.
    The paper's own 'Scope of the results' paragraph notes the standard deviation (0.008-0.018) is comparable to the mean improvement (~0.02), so comparisons only establish direction.
invented entities (1)
  • Idea Bank (atomic idea representation)
    purpose: Stores decomposed method components as short textual descriptions that are sampled and inserted into LLM mutation prompts.
    A new representational unit introduced by the paper; no independent falsifiable handle outside the reported benchmark. Its utility is defined by the paper's own measurements.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Idea Search: Guiding Tree Search with Ideas to Explore Diverse Scientific Methods." pith.science (2026). https://pith.science/paper/WRWN4YYE

@misc{pith2026260808958,
  author       = {Pith},
  title        = {Pith review of: Idea Search: Guiding Tree Search with Ideas to Explore Diverse Scientific Methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WRWN4YYE}},
  note         = {Machine review of arXiv:2608.08958}
}
read the original abstract

Tree Search-based test-time scaling of LLMs is a powerful tool for automated scientific coding. However, pure Tree Search sometimes struggles with systematic exploration, becoming trapped in local optima, or unproductive loops, especially in the vast search space of scientific methods. To address this limitation, we propose Idea Search, a framework that systematically integrates a dynamic "Idea Bank" into Tree Search. Idea Search involves three steps: (1) decomposing existing methods into atomic ideas, (2) sampling from this bank of ideas to guide branches of code mutations, and (3) dynamically updating the bank with new ideas discovered through execution. On single-cell RNA-sequencing (scRNA-seq) batch integration, Idea Search reliably breaks the plateau of a strong pure Tree Search baseline, improving the mean score from 0.678 to 0.697 and reaching a best score of 0.728. We then characterize which design choices drive these gains: bank augmentation helps bandit sampling but not random sampling, "Exploratory" prompting that prioritizes new ideas surfaces the rare best-performing solutions, while increasing sampling-level exploration is counterproductive.

Figures

Figures reproduced from arXiv: 2608.08958 by the authors.

Figure 1
Figure 1. Idea Search couples a dynamic Idea Bank with Tree Search. Expert methods are decomposed into reusable ideas that guide code mutations and are dynamically updated as new solutions are discovered. “condition the decoder on batch ID” or “merge per-batch nearest-neighbor sets.” We refer to such an atomic, executable unit as an idea, and treat a method as a particular combination of ideas that a search can mix and recomb… view at source ↗
Figure 2
Figure 2. Efficacy of Idea Search vs. Base￾line: Both Random and Bandit Sampling break the baseline’s plateau consistently. Both random and bandit idea sampling reliably break the plateau of the pure Tree Search baseline ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Effect of Idea Bank Augmenta￾tion: Bandit Sampling benefits from augmen￾tation, while Random Sampling gets higher score on average with the original “Expert￾Only” Idea Bank. (a) Mean (b) Max [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Effect of Prompting Strat￾egy: Exploratory Prompting is generally beneficial, and when combined with Ran￾dom Sampling, it leads to the discovery of an exceptional outlier solution. new ideas more aggressively–when combined with random sampling, creates a “high-risk, hi…
Figure 5
Figure 5. Figure 5: Effect of the Exploration Parame￾ter in Bandit Sam￾pling: An increased exploration parameter in Bandit Sampling surprisingly decreases performance. Scope of the results. Across all configurations, injecting ideas produces a consistent but small upward shift in the mean…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 6 canonical work pages

  1. [1]

    SWE-Search: Enhancing software agents with monte carlo tree search and iterative refinement.arXiv preprint arXiv:2410.20285,

    Antonis Antoniades, Albert ¨Orwall, Kexun Zhang, Yuxi Xie, Anirudh Goyal, and William Wang. SWE-Search: Enhancing software agents with monte carlo tree search and iterative refinement.arXiv preprint arXiv:2410.20285,

    arXiv

  2. [4]

    Tabvi: Leveraging lightweight transformer architectures to learn biolog- ically meaningful cellular representations.bioRxiv, pp

    Aditi Chandrashekar, Rohan Gala, Andreas Tj¨arnberg, Saniya Khullar, Grace Huynh, and Mariano Gabitto. Tabvi: Leveraging lightweight transformer architectures to learn biolog- ically meaningful cellular representations.bioRxiv, pp. 2025–02,

    work page 2025

  3. [6]

    The station: An open-world environment for ai-driven discovery.arXiv preprint arXiv:2511.06309,

    Stephen Chung and Wenyu Du. The station: An open-world environment for ai-driven discovery.arXiv preprint arXiv:2511.06309,

    work page arXiv

  4. [7]

    Laleh Haghverdi, Aaron TL Lun, Michael D Morgan, and John C Marioni

    Accessed: 2025-11-22. Laleh Haghverdi, Aaron TL Lun, Michael D Morgan, and John C Marioni. Batch effects in single-cell rna-sequencing data are corrected by matching mutual nearest neighbors. Nature biotechnology, 36(5):421–427,

    work page 2025

  5. [9]

    AIDE: AI-driven exploration in the space of code.arXiv preprint arXiv:2502.13138,

    Zhengyao Jiang, Dominik Schmidt, Dhruv Srikanth, Dixing Xu, Ian Kaplan, Deniss Jacenko, and Yuxiang Wu. AIDE: AI-driven exploration in the space of code.arXiv preprint arXiv:2502.13138,

    arXiv

  6. [11]

    Chain of ideas: Revolutionizing research via novel idea development with LLM agents.arXiv preprint arXiv:2410.13185,

    Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xingxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, et al. Chain of ideas: Revolutionizing research via novel idea development with LLM agents.arXiv preprint arXiv:2410.13185,

    arXiv

  7. [13]

    Ml-master: Towards ai-for-ai via integration of exploration and reasoning.arXiv preprint arXiv:2506.16499,

    Zexi Liu, Yuzhu Cai, Xinyu Zhu, Yujie Zheng, Runkun Chen, Ying Wen, Yanfeng Wang, Siheng Chen, et al. Ml-master: Towards ai-for-ai via integration of exploration and reasoning.arXiv preprint arXiv:2506.16499,

    arXiv

  8. [15]

    Krzysztof Pola ´nski, Matthew D Young, Zhichao Miao, Kerstin B Meyer, Sarah A Teich- mann, and Jong-Eun Park

    Accessed: 2025-11-22. Krzysztof Pola ´nski, Matthew D Young, Zhichao Miao, Kerstin B Meyer, Sarah A Teich- mann, and Jong-Eun Park. Bbknn: fast batch alignment of single cell transcriptomes. Bioinformatics, 36(3):964–965,

    work page 2025

Show all 22 references
  1. [16]

    Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers.arXiv preprint arXiv:2409.04109,

    Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers.arXiv preprint arXiv:2409.04109,

    arXiv

  2. [17]

    The ideation-execution gap: Execution outcomes of llm-generated versus human research ideas.arXiv preprint arXiv:2506.20803,

    Chenglei Si, Tatsunori Hashimoto, and Diyi Yang. The ideation-execution gap: Execution outcomes of llm-generated versus human research ideas.arXiv preprint arXiv:2506.20803,

    arXiv

  3. [18]

    Ai research agents for machine learning: Search, exploration, and generalization in mle-bench.arXiv preprint arXiv:2507.02554,

    Edan Toledo, Karen Hambardzumyan, Martin Josifoski, Rishi Hazra, Nicolas Baldwin, Alexis Audran-Reiss, Michael Kuchnik, Despoina Magka, Minqi Jiang, Alisia Maria Lu- pidi, et al. Ai research agents for machine learning: Search, exploration, and generalization in mle-bench.arXi...

  4. [19]

    Planning in natural language improves llm search for code generation.arXiv preprint arXiv:2409.03733, 2024a

    10 Accepted at LM4Sci, COLM 2026 Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean Hendryx, Summer Yue, and Hugh Zhang. Planning in natural language improves llm search for code generation.arXiv preprint arXiv:2409.03733, 2024a. Qi...

    2026 arXiv

  5. [22]

    And I would like you to analyze them, decompose those methods into a list of unitary, factored ideas so that I can sample and compose them to invent new methods

    11 Accepted at LM4Sci, COLM 2026 A Appendix A.1 Prompts for Generating and Augmenting the Idea Bank Generate the Idea Bank For a single-cell batch integration problem, there is a list of existing methods. And I would like you to analyze them, decompose those methods into a lis...

    2026

  6. [2007]

    Fast, sensitive and accurate integration of single-cell data with harmony.Nature methods, 16(12):1289–1296,

    9 Accepted at LM4Sci, COLM 2026 Ilya Korsunsky, Nghia Millard, Jean Fan, Kamil Slowikowski, Fan Zhang, Kevin Wei, Yuriy Baglaenko, Michael Brenner, Po-ru Loh, and Soumya Raychaudhuri. Fast, sensitive and accurate integration of single-cell data with harmony.Nature methods, 16(...

    2026

  7. [2018]

    Accessed: 2025-11-22. Alexander Novikov, Ng ˆan V ˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery. a...

    2025 arXiv

  8. [2019]

    Nova: An iterative planning and search approach to enhance novelty and diversity of llm generated ideas.arXiv preprint arXiv:2410.14255,

    Xiang Hu, Hongyu Fu, Jinge Wang, Yifeng Wang, Zhikun Li, Renjun Xu, Yu Lu, Yaochu Jin, Lili Pan, and Zhenzhong Lan. Nova: An iterative planning and search approach to enhance novelty and diversity of llm generated ideas.arXiv preprint arXiv:2410.14255,

    arXiv

  9. [2020]

    Pan, Alexander Du, Kurt Keutzer, Alvin Cheung, Alexandros G

    Shu Liu, Shubham Agarwal, Monishwaran Maheswaran, Mert Cemri, Zhifei Li, Qiuyang Mang, Ashwin Naren, Ethan Boneh, Audrey Cheng, Melissa Z. Pan, Alexander Du, Kurt Keutzer, Alvin Cheung, Alexandros G. Dimakis, Koushik Sen, Matei Zaharia, and Ion Stoica. Evox: Meta-evolution for...

  10. [2022]

    Thetaevolve: Test-time learning on open problems.arXiv preprint arXiv:2511.23473,

    Yiping Wang, Shao-Rong Su, Zhiyuan Zeng, Eva Xu, Liliang Ren, Xinyu Yang, Zeyi Huang, Xuehai He, Luyao Ma, Baolin Peng, Hao Cheng, Pengcheng He, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Thetaevolve: Test-time learning on open problems.arXiv preprint arXiv...

    arXiv

  11. [2023]

    Learning to discover at test time.arXiv preprint arXiv:2601.16175,

    Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, Jed McCaleb, Xiaolong Wang, Jan Kautz, Yejin Choi, James Zou, Carlos Guestrin, and Yu Sun. Learning to discover at test time.arXiv preprint arXiv:2601.16175,

    arXiv

  12. [2024]

    Adaevolve: Adaptive llm driven zeroth-order optimization.arXiv preprint arXiv:2602.20133,

    Mert Cemri, Shubham Agrawal, Akshat Gupta, Shu Liu, Audrey Cheng, Qiuyang Mang, Ashwin Naren, Lutfi Eren Erdogan, Koushik Sen, Matei Zaharia, Alex Dimakis, and Ion Stoica. Adaevolve: Adaptive llm driven zeroth-order optimization.arXiv preprint arXiv:2602.20133,

  13. [2025]

    Frontier-eng: Benchmarking self-evolving agents on real-world engineering tasks with generative optimization.arXiv preprint arXiv:2604.12290,

    Yizhe Chi, Deyao Hong, Dapeng Jiang, Tianwei Luo, Kaisen Yang, Boshi Zhang, Zhe Cao, Xiaoyan Fan, Bingxiang He, Han Hao, Weiyang Jin, Dianqiao Lei, Qingle Liu, Houde Qian, Bowen Wang, Situ Wang, Youjie Zheng, Yifan Zhou, Calvin Xiao, Eren Cai, and Qinhuai Na. Frontier-eng: Ben...

    arXiv

  14. [2026]

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang

    doi: 10.1038/s41586-026-10658-6. Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. ResearchAgent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738,

    arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.