The Ramon Llull's Thinking Machine for Automated Ideation

Boyuan Zheng; Chenglei Si; Haofei Yu; Ken Liu; Runlong Zhou; Ruochen Li; Tong Chen; Tongshuang Wu; Xiang Li; Xinran Zhao

REVIEW 4 major objections 5 minor 8 references

The Ramon Llull's Thinking Machine for Automated Ideation

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Most research ideas in machine learning decompose into theme, domain, and method.

desk verdict A modest, transparent workshop paper whose recombination baseline is worth having, but whose headline decomposability number is largely an artifact of the evaluation prompt and should be downgraded. read the letter →

arxiv 2508.19200 v3 pith:OUCQRLPH submitted 2025-08-26 cs.AI cs.CL

Xinran Zhao , Boyuan Zheng , Chenglei Si , Haofei Yu , Ken Liu , Runlong Zhou , Ruochen Li , Tong Chen

show 3 more authors

Xiang Li Yiming Zhang Tongshuang Wu

This is my paper · ORCID

classification cs.AIcs.CL

keywords RamonLlullArscombinatoriaautomatedideationlargelanguagemodelsresearchideagenerationtheme-domain-methoddecompositionscientificcreativitycombinatorial

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to show that a large part of machine-learning research ideation can be mechanized as combinatorial recombination across three axes: Theme (the motivation, such as efficiency), Domain (the problem setting, such as question answering), and Method (the technique, such as linear attention). It builds a modern version of Ramon Llull's thirteenth-century thinking machine, using three rotating disks of elements plus templates that combine them, and an LLM to rewrite raw combinations into research titles and abstracts. The authors mine elements from expert-written lists and from 7,483 accepted papers at major conferences, then evaluate the generated ideas with lexical diversity and relevance metrics. Their headline result is that 99.5% of paper titles can be decomposed into the three axes, while only 16.4% can be reconstructed from the elements alone, which they interpret as evidence that the axes capture near-universal structural building blocks but not the specific creative spark. If the claim holds, the pipeline offers a lightweight, interpretable baseline for automated ideation and a map of how different research communities distribute their attention across themes, domains, and methods.

What carries the argument

The central object is the 'thinking machine' itself: a triple (A, B, C) of element lists—Theme, Domain, Method—together with a set of templates T that specify how elements combine, and an LLM that rewrites the raw combination into a polished research idea. The machinery does three jobs: element mining (extracting A/B/C elements and templates from paper titles and abstracts via an LLM, then merging synonyms), combinatorial generation (sampling or enumerating combinations, optionally filtered by visit counts), and LLM rewriting (converting the raw idea into a title and abstract). The load-bearing evaluation device is the bijective coverage test: decomposition measures whether a title maps onto existing A, B, C elements; reconstruction measures whether those elements, fed to an LLM, can regenerate a title with at least 30% token Jaccard similarity to the original.

What would settle it

Take a random sample of papers from the coverage analysis, erase the mined element lists, and ask human annotators or a fresh LLM to propose Theme/Domain/Method triples for each title without seeing any predefined elements; if the share of titles that decompose drops far below 99.5%, the headline decomposability is an artifact of list-guided selection rather than an intrinsic property of research ideas. A second check is to have experts rate a sample of generated ideas for relevance and groundedness and correlate those ratings with the paper's BLEU and Jaccard scores; a near-zero correlation would show the metric-based evaluation does not measure the qualities claimed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the combinatorial structure of Ramon Llull's Ars combinatoria can be revived as a competitive, lightweight method for LLM-based research ideation. The authors define three disks of elements—Theme, Domain, Method—and a small set of templates, such as 'we did a in b with c' or 'compare c1 and c2 in b1 under a1'. Elements are harvested either from human experts or mined automatically from conference papers using an LLM, then merged by semantic similarity. Prompting a separate LLM to rewrite raw combinations produces titles and abstracts that, by lexical measures, are more diverse than prior single-pass or community-simulation baselines when elements are sampled randomly, and more similar to accepted ACL 2025 titles when top elements are enumerated. The bijective coverage analysis shows near-universal decomposability (99.5% across 7,483 papers) but limited reconstructibility (16.4% at a 30% Jaccard threshold), which the authors read as evidence that the three axes form a nearly complete descriptive vocabulary for ML research while the specific instantiation of an idea still demands human or model creativity beyond recombination.

Load-bearing premise

The load-bearing premise is that lexical measures—BLEU relevance to accepted titles, token Jaccard similarity, and distinct-1 diversity—together with a prompt that hands the LLM the element lists are valid proxies for whether research ideas are relevant, novel, and decomposable into Theme–Domain–Method; if those proxies are flawed, the coverage percentages and the comparative claims do not support the abstract's conclusion.

Editorial extensions

If this is right

If the 99.5% decomposability figure is accepted, Theme–Domain–Method is a nearly complete descriptive ontology for the surface structure of ML research papers across NLP, vision, and theory venues.
The pipeline provides a reproducible baseline: 'Llull (Top)' enumerates combinations of the most visited elements and achieves the highest similarity to accepted titles, while 'Llull (Random)' samples elements and achieves the highest diversity with lower relevance.
The released element lists and templates let researchers inspect community differences (ICLR yields more method elements than ACL; ACL domain elements are more stable across years than themes or methods) and track how field interests drift.
The decomposition–reconstruction gap delimits what pure recombination can accomplish: the axes supply structural raw material, while the paper's '4th axis', perturbation, and negation categories identify what still escapes the A+B+C frame.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A list-guided decomposition test may overstate decomposability; an unguided variant that asks annotators to propose A/B/C elements without seeing the mined lists would separate the claim that papers are inherently three-axis from the claim that an LLM can map any title onto a broad menu.
If the compositional model is right, mixing elements across conferences (an ICLR method with an ACL domain under a COLM theme) should produce ideas that expert judges rate as more novel on average than within-conference combinations; this is a testable prediction the paper does not run.
The 16.4% reconstruction ceiling suggests a quantitative definition of the 'non-combinatorial remainder' of an idea, and formalizing that remainder as a fourth axis or as perturbation and negation operators is a natural extension.
The paper's relevance metric is average BLEU against ACL 2025 titles; a semantic embedding-based measure would likely reorder the methods, so the reported diversity–relevance trade-off should be read as specific to lexical overlap.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this is a clearly written, modest workshop paper that builds a lightweight LLM ideation pipeline around three axes (Theme, Domain, Method) and mines those elements from ICLR/ACL/COLT/COLM papers. The genuinely new parts are the cross-conference element statistics, the resource of mined themes/domains/methods, and the decomposition/reconstruction analysis. The recombination idea itself is not new—Scideator and CHIMERA already do facet recombination—but the three-disk framing and the conference-level comparisons are a reasonable incremental step.

The paper does several things well. The pipeline is simple and reproducible in spirit, the authors are unusually honest in the limitations appendix (they explicitly say their quantitative metrics are not sufficient to judge scientific merit, that the comparison does not establish superior quality, and that the tool is not meant to attack peer review). That transparency is real and should be credited. The cross-conference statistics, e.g., similar domain counts but more method elements in ICLR than ACL, are interesting and could be useful to the community.

The soft spots are real but not fatal. The abstract claims ideas are diverse, relevant, and grounded, yet the reported relevance numbers (BLEU 0.11 for Top, 0.05 for Random) are well below the prior systems cited (0.28 and 0.18), and no groundedness metric is reported anywhere. That is an overclaim relative to the evidence. The bigger issue is the 99.5% decomposability result. The stress-test note is correct: the decomposition prompt hands Gemini the full mined lists (682 themes, 633 domains, 866 methods) and asks it to select at least one concept per axis. With lists that broad and populated by generic terms like "efficiency" and "reasoning," near-universal decomposability mostly measures vocabulary coverage, not an inherent A+B+C structure. The 16.4% reconstruction rate is also threshold-dependent (30% token Jaccard, a low bar, with candidate titles generated from the same broad lists). The authors do not hide the prompts, so the analysis is inspectable, but the headline claim should be reworded or the experiment redesigned.

Other issues are minor: no error bars, the baselines are sampled for different purposes so the comparison is only suggestive (the authors acknowledge this), and there is no quantitative comparison with Scideator or CHIMERA despite citing them. No code or data is released yet, though a public repo is promised.

Who this is for: people working on LLM-based scientific ideation and AI-for-science tools. A serious referee should engage with it as a workshop-quality contribution that deserves conditional acceptance with revisions. The central idea is defensible and the weaknesses are addressable. I would bring it to a reading group, but I would not cite it yet in my own work until the coverage analysis is reworked and the resource is actually released.

Referee Report

4 major / 5 minor

Summary. The paper proposes a combinatorial ideation pipeline inspired by Ramon Llull's Ars combinatoria. It defines three compositional axes — Theme, Domain, and Method — whose elements are mined from top-tier conference papers using Gemini, then combined with templates and rewritten by LLMs into research titles and abstracts. The authors report conference-level statistics of the mined elements, compare their generated ideas against prior ideation systems (Si et al. 2024; Yu et al. 2024) on diversity, similarity, and relevance, and perform a two-stage coverage analysis claiming 99.5% decomposability and 16.4% reconstructibility of research papers into the three-axis framework.

Significance. If the coverage claims were supported, this would be a lightweight, interpretable, and falsifiable baseline for LLM-driven ideation and a useful quantitative perspective on the combinatorial structure of machine-learning research. The paper is transparent about its limitations, plans to open-source code and data, and introduces no fitted parameters, which are notable strengths. However, the empirical support for the headline claims is currently partial: relevance scores in Table 4 are below those of prior systems, no groundedness metric is reported, and the decomposition protocol in Section 4.3 is vulnerable to a list-coverage artifact.

major comments (4)

[Section 4.3 and Appendix A.8] The decomposition protocol supplies the full mined element lists (682 themes, 633 domains, 866 methods; Table 2) and asks Gemini to select "the MOST SPECIFIC and ESSENTIAL concepts from these lists," with success defined as at least one selection per disk. With lists of this breadth and granularity, near-universal decomposability (99.5%) mostly measures vocabulary coverage rather than an inherent A+B+C structure. This undermines the claim that the three-disk design captures fundamental aspects of research ideation. I recommend a control condition: ask the model to propose elements without being shown the lists and then test membership, or compare against randomly sampled element lists, and report the decomposability rate under those conditions.
[Abstract and Table 4] The abstract claims the generated ideas are "diverse, relevant, and grounded in current literature," but Table 4 shows relevance of 0.11 (Top) and 0.05 (Random), well below the 0.28 and 0.18 of the prior systems, and no groundedness metric is reported anywhere. The relevance measure is lexical BLEU against ACL 2025 titles, which is a weak proxy for actual relevance, and the authors' own footnote in Section 4.2 states that the comparison does not suggest superior quality. The abstract should be softened or, preferably, a direct groundedness evaluation should be added — for example, checking whether generated ideas cite or are traceable to a specific relevant paper in the mined corpus.
[Section 4.3, reconstruction criterion] The reconstruction rate of 16.4% is threshold-dependent and the protocol is self-confirming: the five candidate titles are generated from the same elements by the same model that selected them, and the 30% Jaccard threshold is hand-chosen with no sensitivity analysis. Without a control — for example, reconstructing titles from random elements or from elements selected by a different model — the 16.4% figure is difficult to interpret. Please report how the rate varies with the Jaccard threshold and include a random-element baseline.
[Section 4.2, Table 4 comparison] The comparison in Table 4 is not matched on important dimensions: Si et al. (2024) uses human-filtered ideas restricted to seven NLP topics, and Yu et al. (2024) grounds ideas in seed papers, whereas the Ramón Llull variants use unfiltered ACL 2024 elements with no seed grounding. The paper acknowledges this in a footnote, but the table presentation invites direct numerical comparison. A matched condition — e.g., human-filtered Ramón Llull ideas, or topic-constrained sampling — would strengthen the diversity/relevance trade-off analysis.

minor comments (5)

[Section 4.3] The term "bijective coverage" is used for a one-way decomposition followed by a one-way reconstruction; this is not a bijection in the mathematical sense. Please clarify the terminology, e.g., "bidirectional coverage."
[Table 4] The table caption does not define "Similarity." The text explains it as average top-K Jaccard similarity with K equal to the number of generated ideas, but this should be stated in the caption for clarity.
[Appendix A.8] The reconstruction prompt instructs the model to generate five diverse titles but does not ask it to reconstruct the original title; reporting the maximum Jaccard over five unconstrained candidates may underestimate reconstructibility. Consider a prompt that explicitly asks for a title close in content to the original paper.
[Table 8] The table contains placeholder entries "Value A3," "Value B3," and "Value C3" in the RL Theory row; these appear to be unfinished and should be filled or removed.
[Throughout] There are several typographical inconsistencies, including "Ram´on" with inconsistent spacing and the use of non-ASCII apostrophes. A careful proofreading pass is recommended.

Circularity Check

1 steps flagged · score 6.0 of 10

The 99.5% decomposability result is built into the evaluation: success is defined as mapping titles onto element lists that the same mining pipeline already extracted from those titles.

fitted input called prediction [Section 4.3 (Decomposition) and Appendix A.8 (Bijective Coverage Evaluation Details)]
"We consider a research idea decomposable if Gemini 2.0 Flash successfully converts the paper title into theme, domain and method elements that our method already extracted. ... We consider a paper decomposable if it can be successfully mapped to at least one element from each disk (A, B, C)."

The element lists are not an external ground truth: they were produced by the same taxonomy and the same model from the same corpus in Section 3.2. The decomposition prompt then hands Gemini those exact mined lists and asks it to 'find the MOST SPECIFIC and ESSENTIAL concepts from these lists,' with success requiring only one selection per disk. With 682 themes, 633 domains, and 866 methods, many of them generic terms (e.g., 'efficiency,' 'reasoning,' 'LLMs'), near-universal decomposability mostly reflects list coverage and model compliance, not an independent confirmation that papers are inherently A+B+C.

full rationale

The ideation pipeline itself is not circular: mining elements with Gemini, combining them through templates, and rewriting with an LLM does not fit a parameter to the evaluation targets, and the comparisons to Si et al. (2024) and Yu et al. (2024) are external baselines, not self-citations bearing the argument. The circularity is confined to Section 4.3's decomposability validation. There, 'decomposable' is defined as the ability of Gemini to select, from the very element lists produced by the same mining pipeline from the same papers, at least one Theme, Domain, and Method. Because the lists are broad, merged, and generated to describe those papers, the 99.5% figure is a coverage/compliance artifact rather than a demonstration that the three-disk design captures fundamental aspects of research ideation. The reconstruction rate of 16.4% provides some independent content, but it is a negative result for the strong interpretation and is threshold-dependent. The paper's own limitations section (A.1) concedes that quantitative metrics are insufficient for judging idea quality, which is honest but does not repair the self-confirming decomposition design. No load-bearing self-citation chain or uniqueness argument appears. Overall, one headline supporting claim reduces by construction while the rest of the pipeline has independent content, so the appropriate score is 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central pipeline rests on the three-axis categorization, LLM-based extraction, and self-assessed lexical metrics. There are no fitted physical parameters, but the evaluation thresholds and sampling choices shape the headline numbers. No new physical or conceptual entities are postulated; the disks are organizational categories, not invented mechanisms.

free parameters (3)

Reconstruction success threshold = Jaccard similarity >= 0.30
The 16.4% reconstruction rate in Table 5 is computed against this hand-chosen cutoff; no justification is given, and the headline number moves with it.
Number of top elements per disk for idea generation = 20 per disk
Section 4.1 uses the top 20 most visited elements to build 4,000 raw ideas per conference; this choice shapes the t-SNE overlap and downstream diversity metrics.
Number of sampled ideas and similarity top-K = 100 ideas; K = number of generated ideas
Section 4.2 compares 100 ideas per method and averages Jaccard over the top-K ACL 2025 titles where K equals the sample size; changing K changes the similarity score.

assumptions (4)

domain assumption Theme, Domain, and Method are the minimum description of a research idea
Introduced in Section 3.1; the entire pipeline and coverage analysis assume these three axes capture ideation, and the authors acknowledge elements can be non-exclusive.
domain assumption Gemini 2.0 Flash extraction and merging faithfully produce reusable themes, domains, methods, and templates
Invoked in Section 3.2 and Appendix A.6; no human validation or error analysis of the mined elements is reported.
domain assumption Lexical metrics (distinct-1, BLEU, token Jaccard) are meaningful proxies for research idea diversity, relevance, and similarity
Used in Section 4.2; the authors themselves state in Appendix A.1 that such quantitative metrics can be insufficient for judging scientific merit.
ad hoc to paper Asking an LLM to select concepts from pre-supplied element lists measures whether a paper decomposes into A+B+C
The decomposition prompt in Appendix A.8 supplies the lists and asks Gemini to pick essential concepts, so the 99.5% decomposability partly reflects prompt design rather than an independent property of papers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Ramon Llull's Thinking Machine for Automated Ideation." pith.science (2026). https://pith.science/paper/OUCQRLPH

@misc{pith2026250819200,
  author       = {Pith},
  title        = {Pith review of: The Ramon Llull's Thinking Machine for Automated Ideation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OUCQRLPH}},
  note         = {Machine review of arXiv:2508.19200}
}

read the original abstract

This paper revisits Ramon Llull's Ars combinatoria - a medieval framework for generating knowledge through symbolic recombination - as a conceptual foundation for building a modern Llull's thinking machine for research ideation. Our approach defines three compositional axes: Theme (e.g., efficiency, adaptivity), Domain (e.g., question answering, machine translation), and Method (e.g., adversarial training, linear attention). These elements represent high-level abstractions common in scientific work - motivations, problem settings, and technical approaches - and serve as building blocks for LLM-driven exploration. We mine elements from human experts or conference papers and show that prompting LLMs with curated combinations produces research ideas that are diverse, relevant, and grounded in current literature. This modern thinking machine offers a lightweight, interpretable tool for augmenting scientific creativity and suggests a path toward collaborative ideation between humans and AI.

Figures

Figures reproduced from arXiv: 2508.19200 by the authors.

**Figure 2.** The density heatmap visualization of ideas generated from the basic [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** The illustration of the original Ramon Llull’s thinking machine. ´ You are a senior AI researcher. Given these research concepts, generate 5 different realistic paper titles that combine them: THEMES: {themes} DOMAINS: {domains} METHODOLOGIES: {methodologies} Generate 5 diverse paper titles that would be suitable for a top-tier conference like ACL/EMNLP/NeurIPS. Each title should: 1. Combine all the given concepts n… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 5 canonical work pages

[1]

This perpetuates harmful societal prejudices and limits the models’ ability to provide fair and inclusive responses across diverse user groups

Problem Statement: Large language models often generate outputs that reinforce existing stereotypes and social biases, even when attempting to be unbiased. This perpetuates harmful societal prejudices and limits the models’ ability to provide fair and inclusive responses across diverse user groups

work page
[2]

By prompting the model to generate adversarial examples that contradict stereotypes, we can encourage it to develop more nuanced and less biased representations

Motivation: Current approaches to reducing bias in language models typically focus on avoiding or counterbalancing stereotypes... By prompting the model to generate adversarial examples that contradict stereotypes, we can encourage it to develop more nuanced and less biased representations

work page
[3]

The prompt structure includes:

Proposed Method: We introduce Adversarial Stereotype Dissolution Prompting (ASDP) , a technique that challenges the model to actively generate counter-stereotypical examples. The prompt structure includes:

work page
[4]

Please provide an unbiased response

Step-by-Step Experiment Plan: Step 1: Dataset Preparation : Create a dataset of stereotype-sensitive queries across various domains (e.g., gender, race, age, profession), Collect 100-200 such queries for a comprehensive evaluation... Step 2: Baseline Methods Implementation : Implement the following baseline methods: a) Standard prompting (direct query). b...

work page 2025
[8]

methodology

The themes of the paper (As, e.g., few-shot, long-tail, less is more, in-the-wild, self-refine, look-ahead, hindsight, memory, self-, rethink, weak to strong, granularity, in-context learning, reference free, grokking, self-evolve, long-tail, compositionality, multi-hop, modular, etc.) 2. The domains of the paper (Bs, e.g., question answering, argument mi...

work page 1937
[2023]

AI Anthropic

URL https://api.semanticscholar.org/CorpusID:265150648. AI Anthropic. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card, 1:1, 2024. Christina Baek, Yiding Jiang, Aditi Raghunathan, and J Zico Kolter. Agreement-on-the-line: Predicting the performance of neural networks under distribution shift. In Alice H. Oh, Alekh Agarwal, Danielle Belg...

arXiv 2024
[2024]

Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal

URL https://openreview.net/forum?id=iAkhPz7Qt3. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, 631(8022): 755–759, 2024. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can llms generate novel research ideas? a large-scale human study wit...

arXiv 2023
[2025]

Haofei Yu, Zhaochen Hong, Zirui Cheng, Kunlun Zhu, Keyang Xuan, Jinwei Yao, Tao Feng, and Jiaxuan You

URL https://api.semanticscholar.org/CorpusID:276094819. Haofei Yu, Zhaochen Hong, Zirui Cheng, Kunlun Zhu, Keyang Xuan, Jinwei Yao, Tao Feng, and Jiaxuan You. Researchtown: Simulator of human research community. arXiv preprint arXiv:2412.17767, 2024. Hongming Zhang, Hantian Ding, and Yangqiu Song. SP-10K: A large-scale evaluation set for selectional prefe...

arXiv 2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

This perpetuates harmful societal prejudices and limits the models’ ability to provide fair and inclusive responses across diverse user groups

Problem Statement: Large language models often generate outputs that reinforce existing stereotypes and social biases, even when attempting to be unbiased. This perpetuates harmful societal prejudices and limits the models’ ability to provide fair and inclusive responses across diverse user groups

work page

[2] [2]

By prompting the model to generate adversarial examples that contradict stereotypes, we can encourage it to develop more nuanced and less biased representations

Motivation: Current approaches to reducing bias in language models typically focus on avoiding or counterbalancing stereotypes... By prompting the model to generate adversarial examples that contradict stereotypes, we can encourage it to develop more nuanced and less biased representations

work page

[3] [3]

The prompt structure includes:

Proposed Method: We introduce Adversarial Stereotype Dissolution Prompting (ASDP) , a technique that challenges the model to actively generate counter-stereotypical examples. The prompt structure includes:

work page

[4] [4]

Please provide an unbiased response

Step-by-Step Experiment Plan: Step 1: Dataset Preparation : Create a dataset of stereotype-sensitive queries across various domains (e.g., gender, race, age, profession), Collect 100-200 such queries for a comprehensive evaluation... Step 2: Baseline Methods Implementation : Implement the following baseline methods: a) Standard prompting (direct query). b...

work page 2025

[5] [8]

methodology

The themes of the paper (As, e.g., few-shot, long-tail, less is more, in-the-wild, self-refine, look-ahead, hindsight, memory, self-, rethink, weak to strong, granularity, in-context learning, reference free, grokking, self-evolve, long-tail, compositionality, multi-hop, modular, etc.) 2. The domains of the paper (Bs, e.g., question answering, argument mi...

work page 1937

[6] [2023]

AI Anthropic

URL https://api.semanticscholar.org/CorpusID:265150648. AI Anthropic. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card, 1:1, 2024. Christina Baek, Yiding Jiang, Aditi Raghunathan, and J Zico Kolter. Agreement-on-the-line: Predicting the performance of neural networks under distribution shift. In Alice H. Oh, Alekh Agarwal, Danielle Belg...

arXiv 2024

[7] [2024]

Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal

URL https://openreview.net/forum?id=iAkhPz7Qt3. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, 631(8022): 755–759, 2024. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can llms generate novel research ideas? a large-scale human study wit...

arXiv 2023

[8] [2025]

Haofei Yu, Zhaochen Hong, Zirui Cheng, Kunlun Zhu, Keyang Xuan, Jinwei Yao, Tao Feng, and Jiaxuan You

URL https://api.semanticscholar.org/CorpusID:276094819. Haofei Yu, Zhaochen Hong, Zirui Cheng, Kunlun Zhu, Keyang Xuan, Jinwei Yao, Tao Feng, and Jiaxuan You. Researchtown: Simulator of human research community. arXiv preprint arXiv:2412.17767, 2024. Hongming Zhang, Hantian Ding, and Yangqiu Song. SP-10K: A large-scale evaluation set for selectional prefe...

arXiv 2024