REVIEW 19 cited by
Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideas
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Scientific innovation is pivotal for humanity, and harnessing large language models (LLMs) to generate research ideas could transform discovery. However, existing LLMs often produce simplistic and repetitive suggestions due to their limited ability in acquiring external knowledge for innovation. To address this problem, we introduce an enhanced planning and search methodology designed to boost the creative potential of LLM-based systems. Our approach involves an iterative process to purposely plan the retrieval of external knowledge, progressively enriching the idea generation with broader and deeper insights. Validation through automated and human assessments indicates that our framework substantially elevates the quality of generated ideas, particularly in novelty and diversity. The number of unique novel ideas produced by our framework is 3.4 times higher than without it. Moreover, our method outperforms the current state-of-the-art, generating at least 2.5 times more top-rated ideas based on 170 seed papers in a Swiss Tournament evaluation.
Forward citations
Cited by 19 Pith papers
-
The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
A randomized execution study with 43 experts shows that LLM-generated research ideas lose more of their appeal than human ideas when actually implemented, reversing part of their ideation-stage advantage.
-
ResearchStudio-Idea: An Evidence-Grounded Research-Ideation Skill Suite from ML Conference Outcomes
Conference accept/reject outcomes yield 15 operational ideation patterns that, as an LLM skill suite, improve automated-judged research-proposal quality over no-skill and generic-skill baselines.
-
LigBench: A Unified and Human-Aligned Benchmark for LLM-based Research Idea Generation
LigBench evaluates research ideas through formalization, pairwise comparison against 11k+ papers with debiased OpenReview scores, and Elo-based score aggregation, validated against human experts and paper acceptance outcomes.
-
Idea Search: Guiding Tree Search with Ideas to Explore Diverse Scientific Methods
Idea Search, a tree search variant that samples and updates a dynamically scored bank of atomic design ideas, modestly improves LLM-generated single-cell batch integration code over a pure tree search baseline.
-
Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation
LLM judges of scientific ideas are measurably swayed by writing style; a style-detecting module reduces but does not remove the bias.
-
Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
A new benchmark (IG-Bench) reveals that LLM-based scientists fail at compositional lineage reasoning, with the best system reaching only 27.3% exact accuracy.
-
HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discovery
In a small user study and two case studies, a hypothesis-chain workflow grounded in knowledge graphs helped biomedical researchers construct and validate hypotheses from machine-learning predictions more effectively t...
-
Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models
A new benchmark (TruthHypo) and a knowledge-grounded hallucination detector (KnowHD) show that grounding scores can partially select truthful LLM-generated biomedical hypotheses, but the result is at risk from knowled...
-
IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery
IRIS combines Monte Carlo Tree Search, fine-grained LLM review, and targeted literature retrieval in a human-in-the-loop system for generating research briefs.
-
AI Idea Bench 2025: AI Research Idea Generation Benchmark
The paper builds a post-cutoff benchmark dataset and a multi-metric evaluation framework for scoring LLM-generated research ideas against real paper motivations and experiment plans.
-
Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback
Dolphin closes the loop between idea generation, code implementation, and experimental feedback, improving on baselines and producing one 3D classification model comparable to a human-designed state-of-the-art.
-
Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details
For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.
-
Deep Researcher with Test-Time Diffusion
A test-time 'denoising' loop that repeatedly revises a draft report using fresh web retrieval, plus a component-wise self-evolution step, beats existing deep research agents on several benchmarks.
-
DeepResearch$^{\text{Eco}}$: A Recursive Agentic Workflow for Complex Scientific Question Answering in Ecology
A recursive agentic pipeline for literature synthesis showing a 21-fold source increase and 14.9-fold density gain when depth and breadth are raised.
-
LLM4SR: A Survey on Large Language Models for Scientific Research
A systematic review of LLM-based systems for hypothesis discovery, experiment planning, scientific writing, and peer review, including benchmarks, evaluation methods, and open challenges.
-
AIGS: Generating Science from AI-Powered Automated Falsification
Baby-AIGS is a multi-agent system that automates research through explicit falsification, outperforming its baseline on three ML tasks but lagging human experts.
-
AI for Auto-Research: Roadmap & User Guide
The paper delivers a stage-by-stage roadmap for AI in research, showing reliable assistance in retrieval and tool tasks but fragility in novelty and judgment, advocating human-governed collaboration.
-
How Far Are AI Scientists from Changing the World?
This survey proposes a four-level capability framework for AI Scientist systems and, using an AI reviewer, finds that current systems produce papers rated well below normal scientific standards.
-
AI Scientists Fail Without Strong Implementation Capability
AI scientist systems can propose ideas but cannot reliably implement and verify experiments, making the implementation gap, not idea generation, the current bottleneck.
Discussion (0). Continue with ORCID to comment.