Pith. sign in

REVIEW 5 major objections 5 minor 6 cited by

Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dolphin closes the research loop: LLM-generated ideas are implemented, run, and fed back, and one generated 3D classifier reaches 93.9% accuracy on ModelNet40.

desk verdict A genuinely closed-loop auto-research framework with real benchmark runs, but the headline SoTA claim rests on a single best-of-40 run with no seeds, so treat the numbers as promising, not established. read the letter →

arxiv 2501.03916 v3 pith:VQVUB5MU submitted 2025-01-07 cs.AI cs.CLcs.CV

Jiakang Yuan , Xiangchao Yan , Shiyang Feng , Bo Zhang , Tao Chen , Botian Shi , Wanli Ouyang , Yu Qiao
show 2 more authors
Lei Bai Bowen Zhou
This is my paper · ORCID
classification cs.AIcs.CLcs.CV
keywords closed-loopauto-researchLLM-drivenscienceideagenerationexperimentalverificationresultsfeedback3DpointcloudclassificationModelNet40MLE-bench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dolphin is a closed-loop framework in which a large language model generates research ideas, turns them into runnable code, executes the experiments, and feeds the measured results back into the next round of idea generation. The paper's central claim is that this cycle improves the input topic iteratively, and that it can produce methods comparable to human state of the art: on ModelNet40 the generated PointNet-CSR reaches 93.9% overall accuracy, slightly above GPSFormer's 93.8%. The framework also reports gains on CIFAR-100 and SST-2, and on three MLE-bench tasks it improves code templates from other pipelines. The point of the work is to replace novelty-only evaluation of AI-generated ideas with experimental verification and feedback, the loop that human researchers use.

What carries the argument

The load-bearing object is the three-stage research cycle: idea generation, experimental verification, and results feedback. Ideas are generated with reference to retrieved papers, which are filtered by an LLM-scored ranking that checks both topic relevance and task attributes such as model input and output; redundant ideas are removed by comparing summary embeddings against an idea bank, and novelty is screened by the LLM against the retrieved literature. Implementations are produced by modifying reference code, and failures are repaired by an exception-traceback-guided debugging step that extracts the function names, lines, and custom-code pieces from the traceback, asks the LLM to reconstruct the local code structure, and then plans the fix. After successful runs, results are classified as improvement, maintenance, or decline; declining or stagnant ideas are added to the idea bank to block similar proposals, while effective ideas are injected into the next idea-generation prompt. These two named mechanisms, task-attribute-guided paper ranking and exception-traceback-guided debugging, are what make the closed loop run at acceptable novelty and execution-success rates.

What would settle it

Reproduce the three baselines using the original authors' code and training schedules on ModelNet40, CIFAR-100, and SST-2, and compare them with the paper's Baseline† numbers; then rerun the Dolphin-generated models under the same evaluation. If PointNet-CSR's 93.9% OA does not reproduce, or if the reimplemented baselines are measurably weaker than the originals, the central claim of closed-loop improvement is not supported.

Watch

Extended reading notes

Core claim

The central claim is that an auto-research system can be closed: ideas are judged by whether they improve a reference implementation on a standard benchmark, and the outcome of each experiment changes what the next round of ideas is allowed to be. In two loops on ModelNet40, Dolphin generated 40 ideas, executed 21 that passed novelty and redundancy checks, and found 5 that improved the PointNet baseline; the best, PointNet-CSR, reaches 93.9% overall accuracy and 91.1% mean class accuracy, compared with 93.8% and 91.8% for GPSFormer under comparable settings (no pre-training, no voting). Across loops the average improvement over its own PointNet baseline is 1.0% OA; on CIFAR-100 the best idea adds 0.8% over WRN-28-10, and on SST-2 the best idea adds 1.5% over BERT-base. The paper also reports that 9 of 21 successfully executed ideas improved their baselines across three loops, with the improvement rate rising from 2/7 in loop 1 to 4/8 in loop 3.

Load-bearing premise

The load-bearing premise is that the paper's reimplementations of PointNet, WRN-28-10, and BERT-base faithfully match the original methods; if those baselines are weaker or differently tuned, the reported improvements are inflated.

Editorial extensions

If this is right

  • Given a topic, a reference model, and a dataset, Dolphin can keep proposing validated ideas across loops, with the per-loop improvement rate rising from 2/7 to 4/8 in the 3D classification experiments.
  • A generated method can match or exceed a human-designed state of the art on a standard benchmark: PointNet-CSR scores 93.9% OA versus 93.8% for GPSFormer on ModelNet40 without pre-training or voting.
  • Task-attribute filtering lifts the share of novel ideas from 8/20 to 19/20, and traceback-guided debugging roughly doubles the execution-success rate from 33.3% to 50.0%.
  • On MLE-bench tasks, the loop improves code templates produced by other pipelines, e.g., raising an insult-detection score from 81.0 to 84.7 and a toxic-comment score from 94.7 to 97.2.
  • The per-idea cost is about $0.20 including retrieval, generation, and novelty checks, making the loop cheap enough to run many iterations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the paper's own logic is to push the loop well past two iterations and measure when improvement plateaus or reverses; the feedback mechanism predicts continued gains until the idea space near the baseline is exhausted.
  • The 3D result suggests that a simple parameter-free contextual-semantic module can compete with learned graph convolutions; one could test whether that module transfers to other 3D backbones or to other modalities.
  • The paper's Limitations section concedes that only titles and abstracts are read and that project-level code is beyond current LLM coding ability; full-text grounding and hierarchical code-aware agents are natural next steps that the closed loop is compatible with.
  • Because the reported gains are computed against the paper's own reimplementations, a neutral reader should treat the improvement magnitudes as conditional on baseline fidelity until the baselines are matched against original published protocols.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Dolphin, a closed-loop LLM-driven framework for automated scientific research that cycles through idea generation, experimental verification, and result feedback. On ModelNet40, CIFAR-100, SST-2, and selected MLE-bench tasks, the authors report that Dolphin generates ideas that improve over their implemented baselines, with the best ModelNet40 run (93.9% OA) slightly surpassing the published GPSFormer result (93.8% OA). The paper also introduces task-attribute-guided paper ranking and exception-traceback-guided debugging, and reports ablation studies for these components.

Significance. If the central claims hold, the work is a useful step toward closed-loop automated research: it provides one of the first frameworks where LLM-generated ideas are validated by actually running code on standard benchmarks, and the proposed debugging and paper-ranking components are concrete, plausible engineering contributions. The reported accuracy numbers are externally obtained from executed experiments, not merely from model self-reporting. However, the current evidence does not support the strength of the headline claims: the SoTA comparison rests on a single best-of-40 run without seeds or error bars, the closed-loop improvement trend is not statistically distinguishable from noise, and the novelty evaluation is performed by the same LLM family that generates the ideas. These issues are load-bearing for the abstract and the main experimental sections.

major comments (5)
  1. [Table 1, Table 7, Section 4.2] The statement that Dolphin's methods are 'comparable to the state-of-the-art' rests on a single best-of-40 run (93.9% OA on ModelNet40, Table 7), with no seed-level replication, no standard deviation, and no correction for the fact that 40 ideas were attempted. Since the maximum of many noisy candidates systematically overestimates the reproducible gain, the paper must report per-idea distributions across multiple seeds or a pre-registered holdout before the SoTA claim can be evaluated.
  2. [Table 6, Section 4.3] The closed-loop 'continuous improvement' claim is based on improvement rates of 2/7, 3/6, and 4/8 over three loops, with no no-feedback control and no test of trend significance. As reported, the sequence is consistent with random fluctuation; a paired control using the same idea budget without feedback, together with an appropriate statistical test, is required to support the claim.
  3. [Section 3.1, Table 3] The novelty metric is gpt-4o's self-assessment of ideas generated by the same model, and the independence filter is a cosine threshold over summary embeddings. No human evaluation or external benchmark is provided to validate that the reported 19/20 novelty rate reflects genuine novelty. Please add a human-rated validation sample or an external novelty signal to break the circularity.
  4. [Appendix B.1.2, Table 1] The 'Baseline†' is labeled as 'our implementation' and described as 'partially refers to the codebase', but the paper does not show that the reimplemented PointNet, WRN-28-10, and BERT-base match their canonical training protocols, hyperparameters, and published accuracies. If the reimplemented baselines are weaker than the original implementations, the reported average and max improvements, as well as the comparisons to GPSFormer, ResNeXt, and BERT-large, are inflated.
  5. [Table 1, Section 4.2] Table 1 introduces two baselines for PointNet (89.2 and 91.0) without clearly explaining the relationship between 'Baseline' and 'Baseline†'. The text and the improvement numbers should state explicitly which baseline is used for the reported improvements, since the choice changes the magnitude and interpretation of the claims.
minor comments (5)
  1. [Appendix B.1.1] The scheduler name is misspelled as 'CosineAnuealing'; it should be 'CosineAnnealing'.
  2. [Appendix C.2] The framework name is misspelled as 'DOPLINE' in this section; it should be 'DOLPHIN'.
  3. [Section 1] The phrase 'We were supervised to observe' is unclear; likely 'surprised to observe' is intended.
  4. [Table 1] The header 'Number ideas' should read 'Number of ideas'.
  5. [Table 4] The keyword frequencies are not normalized by the number of papers in each retrieval condition, making the comparison across 'Naive' and 'Filter' hard to interpret; please report proportions or the underlying paper counts.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity in the novelty evaluation: the same gpt-4o model generates and judges the ideas, so the reported novelty rates are self-confirmation; the benchmark improvements are externally measured and not circular.

  1. self definitional [Section 3.1 (Ideas Generation and Filtering); Table 3 caption; Fig. 5 idea-generation prompt]
    "Following AI-Scientist (Lu et al., 2024), DOLPHIN simply prompt the LLMs to decide whether the idea is novel based on the searched papers by Semantic Scholar API. Only ideas identified as novel and independent will proceed to the subsequent experimental verification process. ... The novelty is evaluated by gpt-4o-2024-08-06. ... you should make sure your ideas are novel and distinct from the prior literature."

    The novelty judge and the idea generator are the same system (gpt-4o-2024-08-06), and the generator is explicitly prompted to produce ideas it will later be asked to certify as novel. The Table 3 counts (8/20 vs 19/20) therefore operationalize 'novel' as 'consistent with gpt-4o's own instructed generation', not as an independent external property. The paper uses these counts as evidence for its 'generate high-quality ideas' contribution and for the benefit of task-attribute paper ranking, so that part of the claimed derivation is self-confirming. The central benchmark claims (ModelNet40, CIFAR-100, SST-2, MLE-bench) rest on executed code and external data, so they are not circular.

full rationale

The main derivation chain is empirically grounded: DOLPHIN's claimed performance improvements are measured by executing generated code on public benchmarks (ModelNet40, CIFAR-100, SST-2, MLE-bench), so the accuracy numbers are not constructed from the paper's assumptions. The closed-loop feedback is a prompt-modification mechanism whose outputs are still externally evaluated; no parameter is fitted to make the target result true by construction. The self-citations (Yan et al. 2025; Qi et al. 2023/2024) are contextual and not load-bearing. The novelty evaluation is the one genuinely circular element: the LLM that generates ideas is also the LLM that certifies them as novel, and the generation prompt tells the model to make ideas novel, so Table 3's novelty counts are self-consistency scores rather than independent measurements. The paper itself acknowledges related leakage and verification limitations in its Limitations section, which further weakens the novelty evidence. The best-of-40 maximum improvement (93.9 OA) and the loop improvement rates without a no-feedback control are statistical-validity concerns about selection and noise, not derivation-by-construction, so they do not add to the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims depend on a handful of hand-chosen hyperparameters, on the fidelity of reimplemented baselines, and on self-judged novelty. The framework itself does not introduce new physical entities.

free parameters (5)
  • idea independence threshold = 0.8 cosine similarity
    Hand-chosen in Sec. 4.1; ideas above this similarity to any banked idea are discarded, so the threshold directly determines how many ideas are tested.
  • paper relevance cutoff = 8 out of 10
    Hand-chosen in Sec. 4.1 and 3.1; only papers scoring above 8 are used as references, shaping the idea pool.
  • ideas per loop = 20
    Hand-chosen in Sec. 4.1; the number of candidate ideas per cycle.
  • retrieved papers count = 50
    Hand-chosen in Sec. 4.1; retrieval window for Semantic Scholar API.
  • max debugging attempts = 5
    Hand-chosen in Sec. 4.1; caps the debugging loop and affects execution success rate.
assumptions (4)
  • domain assumption The reimplemented baselines (PointNet, WRN-28-10, BERT-base) match the original training and evaluation protocols.
    Needed for the reported improvements to be meaningful. The paper only says implementations "partially refer" to public codebases and does not compare its reimplemented baselines to published numbers in the same table.
  • ad hoc to paper gpt-4o novelty judgments are a valid measure of research novelty.
    Used as the evaluation metric in Table 3 and the novelty filter in Sec. 3.1; no human evaluation or external benchmark is provided.
  • domain assumption Single-run experiment results are representative of the method's performance.
    No seeds, repeats, or confidence intervals are reported for any accuracy number.
  • domain assumption Sentence embedding cosine similarity can detect whether two ideas are redundant.
    The independence check in Sec. 3.1 uses embeddings of idea summaries with a fixed threshold; this is assumed to capture redundancy without validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback." pith.science (2026). https://pith.science/paper/VQVUB5MU

@misc{pith2026250103916,
  author       = {Pith},
  title        = {Pith review of: Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQVUB5MU}},
  note         = {Machine review of arXiv:2501.03916}
}
read the original abstract

The scientific research paradigm is undergoing a profound transformation owing to the development of Artificial Intelligence (AI). Recent works demonstrate that various AI-assisted research methods can largely improve research efficiency by improving data analysis, accelerating computation, and fostering novel idea generation. To further move towards the ultimate goal (i.e., automatic scientific research), in this paper, we introduce Dolphin, a closed-loop LLM-driven framework to enhance the automation level of scientific research. Dolphin first generates novel ideas based on feedback from previous experiments and relevant papers ranked by the topic and task attributes. Then, the generated ideas can be implemented using a code template refined and debugged with the designed exception-traceback-guided local code structure. Finally, Dolphin automatically analyzes the results of each idea and feeds the results back to the next round of idea generation. Experiments are conducted on the benchmark datasets of different topics and a subset of MLE-bench. Results show that Dolphin can continuously improve the performance of the input topic in a loop. We highlight that Dolphin can automatically propose methods that are comparable to the state-of-the-art in some tasks such as 3D point classification.

Figures

Figures reproduced from arXiv: 2501.03916 by the authors.

Figure 1
Figure 1. Comparisons of the four stages in the evo [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. DOLPHIN first generates a set of ideas based on the retrieved papers. After filtering ideas, experimental plans will be generated for these filtered ideas. Then, codes can be generated and debugged using the proposed error-traceback-guided debugging process. Finally, the results of successfully executed experiments will be auto￾analyzed and reflected into the next round of ideas generation. to use LLMs to generate h… view at source ↗
Figure 3
Figure 3. Debugging with traceback-guided local code [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Prompts of paper retrieval, paper ranking, and ideas generation. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: An example of independence check. ### Code Structure Prompt You are an expert code analyst specializing in exception detection, debugging, and error handling patterns. Your task is to thoroughly analyze the provided code with a focus on potential errors below: \n {erro…
Figure 7
Figure 7. Figure 7: Prompts of local code structure and debugging. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Code structure without extracted traceback information. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Case studies for the ideas and codes generated by D [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Idea and codes generated by DOLPHIN which achieves 92.34% OA and 89.54% mAcc. on ModelNet40 (+1.34% OA and +1.94% mAcc. compared to our baseline). 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Idea and codes generated by DOLPHIN which achieves 92.30% OA and 88.96% mAcc. on ModelNet40 (+1.30% OA and +1.36% mAcc. compared to our baseline). 20 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Idea and codes generated by DOLPHIN which achieves 82.05% Acc. on CIFAR-100 (+0.85% Acc. compared to our baseline). 21 [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning

    cs.CV 2025-08 conditional novelty 6.0 of 10

    An RL framework that trains vision-language models to self-correct captions via a scene-graph-based reward outperforms SFT and DPO on caption quality.

  2. HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discovery

    cs.HC 2025-07 conditional novelty 6.0 of 10

    In a small user study and two case studies, a hypothesis-chain workflow grounded in knowledge graphs helped biomedical researchers construct and validate hypotheses from machine-learning predictions more effectively t...

  3. ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference

    cs.CL 2025-09 conditional novelty 5.0 of 10

    ResearchPulse extracts motivation-method chains and experimental trends from related papers, rendering them as mind maps and line charts, and releases a 100-cluster benchmark; the reported '7B beats GPT-4o' result is ...

  4. InternAgent: When Agent Becomes the Scientist -- Building Closed-Loop System from Hypothesis to Verification

    cs.AI 2025-05 conditional novelty 5.0 of 10

    A closed-loop LLM-agent framework that auto-generates research ideas and code, reported to improve baseline performance on all 12 tasks it was tested on.

  5. How Far Are AI Scientists from Changing the World?

    cs.AI 2025-07 conditional novelty 4.0 of 10

    This survey proposes a four-level capability framework for AI Scientist systems and, using an AI reviewer, finds that current systems produce papers rated well below normal scientific standards.

  6. AI Scientists Fail Without Strong Implementation Capability

    cs.AI 2025-06 conditional novelty 4.0 of 10

    AI scientist systems can propose ideas but cannot reliably implement and verify experiments, making the implementation gap, not idea generation, the current bottleneck.

Reference graph

Works this paper leans on

54 extracted references · 20 canonical work pages · cited by 6 Pith papers

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. URL: https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf

    work page 2024

  3. [3]

    Assafelovic. 2023. gpt-researcher. URL: https://github.com/assafelovic/gpt-researcher

    work page 2023

  4. [4]

    Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. 2024. Mle-bench: Evaluating machine learning agents on machine learning engineering. arXiv preprint arXiv:2410.07095

    arXiv 2024

  5. [5]

    Angelica Chen, David Dohan, and David So. 2024. Evoprompting: language models for code-level neural architecture search. Advances in Neural Information Processing Systems, 36

    work page 2024

  6. [6]

    Kang Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jing-Jia Luo, Xi Chen, Leiming Ma, Tianning Zhang, Rui Su, et al. 2023. Fengwu: Pushing the skillful global medium-range weather forecast beyond 10 days lead. arXiv preprint arXiv:2304.02948

    arXiv 2023

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics , pages 4171--4186

    work page 2019

  8. [8]

    Yuxiang Wu Dominik Schmidt, Zhengyao Jiang. 2024. Aide. URL: https://www.weco.ai/blog/technical-report

    work page 2024

Show all 54 references
  1. [9]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. In Internation...

    2020

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  3. [11]

    Tao Han, Zhenghao Chen, Song Guo, Wanghan Xu, and Lei Bai. 2024. Cra5: Extreme compression of era5 for portable global climate and weather research via an efficient variational transformer. arXiv preprint arXiv:2405.03376

    2024 arXiv

  4. [12]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778

    2016

  5. [13]

    Xiang Hu, Hongyu Fu, Jinge Wang, Yifeng Wang, Zhikun Li, Renjun Xu, Yu Lu, Yaochu Jin, Lili Pan, and Zhenzhong Lan. 2024. Nova: An iterative planning and search approach to enhance novelty and diversity of llm generated ideas. arXiv preprint arXiv:2410.14255

    2024 arXiv

  6. [14]

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Z \' dek, Anna Potapenko, et al. 2021. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583--589

    2021

  7. [15]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images

    2009

  8. [16]

    Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xinxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, et al. 2024. Chain of ideas: Revolutionizing research in novel idea development with llm agents. arXiv preprint arXiv:2410.13185

    2024 arXiv

  9. [17]

    Siyi Liu, Chen Gao, and Yong Li. 2024. Large language model agent for hyper-parameter optimization. arXiv preprint arXiv:2402.01881

    2024 arXiv

  10. [18]

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976--11986

    2022

  11. [19]

    Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. 2024. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery . ArXiv, abs/2408.06292

    2024 arXiv

  12. [20]

    Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics, 23(6):bbac409

    2022

  13. [21]

    OpenAI. 2024. Hello gpt-4o. https://openai.com/index/hello-gpt-4o/

    2024

  14. [22]

    Biqing Qi, Kaiyan Zhang, Haoxiang Li, Kai Tian, Sihang Zeng, Zhang-Ren Chen, and Bowen Zhou. 2023. Large language models are zero shot hypothesis proposers. arXiv preprint arXiv:2311.05965

    2023 arXiv

  15. [23]

    Biqing Qi, Kaiyan Zhang, Kai Tian, Haoxiang Li, Zhang-Ren Chen, Sihang Zeng, Ermo Hua, Hu Jinfang, and Bowen Zhou. 2024. Large language models as biomedical hypothesis generators: A comprehensive evaluation. arXiv preprint arXiv:2407.08940

    2024 arXiv

  16. [24]

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017 a . Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652--660

    2017

  17. [25]

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017 b . Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30

    2017

  18. [26]

    Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. 2022. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in neural information processing systems, 35:23192--23204

    2022

  19. [27]

    Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, et al. 2023. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. arXiv preprin...

    2023 arXiv

  20. [28]

    Justus Randolph. 2019. A guide to writing the dissertation literature review. Practical assessment, research, and evaluation, 14(1):13

    2019

  21. [29]

    Nils Reimers and Iryna Gurevych. 2020. Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation . In EMNLP

    2020

  22. [30]

    Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. 2025. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227

    2025 arXiv

  23. [31]

    Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. 2024. Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers. arXiv preprint arXiv:2409.04109

    2024 arXiv

  24. [32]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  25. [33]

    Haoyang Su, Renqi Chen, Shixiang Tang, Xinzhe Zheng, Jingzhe Li, Zhenfei Yin, Wanli Ouyang, and Nanqing Dong. 2024. Two heads are better than one: A multi-agent system has the potential to improve scientific idea generation. arXiv preprint arXiv:2410.09403

    2024 arXiv

  26. [34]

    Copilot Team. 2023 a . copilot. URL: https://github.com/features/copilot

    2023

  27. [35]

    Ollama Team. 2023 b . ollama. URL: https://ollama.com/

    2023

  28. [36]

    Changshuo Wang, Meiqing Wu, Siew-Kei Lam, Xin Ning, Shangshu Yu, Ruiping Wang, Weijun Li, and Thambipillai Srikanthan. 2024 a . Gpsformer: A global perception and local structure fitting-based transformer for point cloud understanding. In European Conference on Computer Vision...

    2024

  29. [37]

    Peng-Shuai Wang. 2023. Octformer: Octree-based transformers for 3d point clouds. ACM Transactions on Graphics (TOG), 42(4):1--11

    2023

  30. [38]

    Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. 2023. Scimon: Scientific inspiration machines optimized for novelty. arXiv preprint arXiv:2305.14259

    2023 arXiv

  31. [39]

    Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. 2024 b . SciMON: Scientific Inspiration Machines Optimized for Novelty . In ACL

    2024

  32. [40]

    Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, et al. 2024 c . Autosurvey: Large language models can automatically write surveys. arXiv preprint arXiv:2406.10252

    2024 arXiv

  33. [41]

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog), 38(5):1--12

    2019

  34. [42]

    Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912--1920

    2015

  35. [43]

    Xiangchao Yan, Shiyang Feng, Jiakang Yuan, Renqiu Xia, Bin Wang, Bo Zhang, and Lei Bai. 2025. Surveyforge: On the outline heuristics, memory-driven generation, and multi-dimensional evaluation for automated survey writing. arXiv preprint arXiv:2503.04629

    2025 arXiv

  36. [44]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024 a . Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  37. [45]

    Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and E. Cambria. 2024 b . Large Language Models for Automated Open-domain Scientific Hypotheses Discovery . ACL Findings

    2024

  38. [46]

    Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. 2023. Large language models for automated open-domain scientific hypotheses discovery. arXiv preprint arXiv:2309.02726

    2023 arXiv

  39. [47]

    Shunyu Yao, Qingqing Ke, Qiwei Wang, Kangtong Li, and Jie Hu. 2024. https://doi.org/10.1145/3689299.3689319 Lawyer gpt: A legal large language model with enhanced domain knowledge and reasoning capabilities . In Proceedings of the 2024 3rd International Symposium on Robotics, ...

    2024

  40. [48]

    Sergey Zagoruyko. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146

    2016 arXiv

  41. [49]

    Lei Zhang, Yuge Zhang, Kan Ren, Dongsheng Li, and Yuqing Yang. 2023 a . Mlcopilot: Unleashing the power of large language models in solving machine learning tasks. arXiv preprint arXiv:2304.14979

    2023 arXiv

  42. [50]

    Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mingyuan Zhou. 2023 b . Automl-gpt: Automatic machine learning with gpt. arXiv preprint arXiv:2305.02499

    2023 arXiv

  43. [51]

    Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, and Chenhao Tan. 2024. Hypothesis generation with large language models. arXiv preprint arXiv:2404.04326

    2024 arXiv

  44. [52]

    Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. 2024. Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:2406.11931

    2024 arXiv

  45. [53]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  46. [54]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.