Pith. sign in

REVIEW 4 major objections 4 minor 4 cited by

Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Need

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a 32-billion-parameter model fine-tuned on 24,000 reasoning tasks distilled from medical knowledge-graph paths can outperform all evaluated open-source and proprietary reasoning models on its 15-category medical…

desk verdict A clever KG-curriculum pipeline with a benchmark that is too close to the training distribution and an unfair compute comparison; worth reviewing, but the 'superintelligence' claim is not yet established. read the letter →

arxiv 2507.13966 v2 pith:P3IOU2MD submitted 2025-07-18 cs.CL cs.AI

Bhishma Dedhia , Yuval Kansal , Niraj K. Jha This is my paper
classification cs.CLcs.AI
keywords knowledgegraphscurriculumlearningdomain-specificreasoningmedicalquestionansweringmulti-hopthinkingtracesICD-BenchUMLS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that deep domain expertise can be taught bottom-up: instead of exposing a model to more general text, one starts from primitive facts—single head-relation-tail edges of a knowledge graph—and trains on tasks that require composing those primitives into multi-hop reasoning chains. The authors build a pipeline that samples paths from the UMLS medical knowledge graph, turns each path into a multiple-choice clinical question, and distills a step-by-step thinking trace that explicitly walks along the path. Fine-tuning the QwQ-32B model on 24,000 such tasks yields QwQ-Med-3, which the paper reports significantly outperforms open-source and proprietary reasoning models on all 15 categories of its new ICD-Bench evaluation and improves the base model's accuracy on external medical QA benchmarks. If the claim holds, it suggests specialized superintelligence could be grown cheaply from structured domain knowledge rather than from ever-larger general-purpose models, and that networks of such specialists might compose into general intelligence. The load-bearing condition is that ICD-Bench measures genuine reasoning and not memorization of the training curriculum.

What carries the argument

The knowledge-graph path is the load-bearing object. A path is a chain of (head, relation, tail) triples—for example, diabetes predisposes to kidney disease, which causes anemia—and a path of length N encodes a higher-level concept that a model must recall and compose. The pipeline converts each sampled path into a closed-ended multiple-choice vignette using a template prompt, has a strong reasoning LLM produce a thinking trace explicitly anchored to the path, and filters items through two independent LLM graders. Training inserts the trace between special thinking delimiters, and inference scales compute by sampling parallel traces with majority voting or by iterative refinement.

What would settle it

Build an equivalent medical QA benchmark from an independent curated medical knowledge graph or from clinician-written board-style questions with no overlap with UMLS, then compare QwQ-Med-3, the base QwQ-32B, and the proprietary baselines under identical inference budgets; if the performance gap largely disappears, the claim that KG-grounded curriculum tuning elicits transferable domain reasoning collapses.

Watch

Extended reading notes

Core claim

The central discovery is that a model trained to acquire and explicitly compose knowledge-graph primitives exhibits domain-specific reasoning that the same model without that curriculum does not. Concretely, QwQ-Med-3—QwQ-32B fine-tuned with low-rank adapters on 24,000 question-and-thinking-trace pairs generated from one-, two-, and three-hop UMLS paths—outperforms the base model and the strongest open-source and proprietary reasoning baselines across every one of ICD-Bench's 15 medical categories, with the largest gains on the hardest tasks. The paper interprets the gap as evidence that the base model can recall medical entities but fails to compose them into multi-hop reasoning, whereas curriculum tuning closes that recall-reasoning gap.

Load-bearing premise

The results stand or fall on whether ICD-Bench, built with the same knowledge graph and LLM-based question-generation pipeline as the training data, measures genuine multi-hop reasoning rather than rewarded pattern recall; the paper's decontamination filters, which remove exact path overlaps and 18-gram text matches, are what protect against that artifact.

Editorial extensions

If this is right

  • A relatively small 32-billion-parameter model can exceed much larger general-purpose reasoning models on a specialized evaluation, so domain depth may not require model breadth or scale.
  • Deeper curriculum exposure, especially three-hop paths, is what lifts performance on the hardest tasks; easy tasks gain little from depth, so curriculum depth should be matched to target difficulty.
  • Models that can recall relevant facts often fail to reason over them; curriculum tuning specifically repairs this recall-reasoning gap rather than merely adding factual knowledge.
  • Knowledge-graph-grounded expertise transfers beyond the training paths: gains appear on external medical QA benchmarks such as MedQA, PubMedQA, MedMCQA, and the MMLU-Med subset.
  • The same bottom-up pipeline should apply to any domain with a reliable knowledge graph, which the paper frames as a path toward composable, domain-specific superintelligence rather than monolithic AGI.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The decisive test the paper leaves implicit is whether ICD-Bench, built from the same UMLS knowledge graph and the same LLM-based question-generation pipeline as the training curriculum, measures transferable reasoning rather than rewarded in-distribution patterns; an independent benchmark would settle that.
  • If the method is as general as claimed, a natural extension is to domains with weaker or less canonical knowledge graphs, such as law, finance, or chemistry, where the reliance on reliable primitives is the main open risk and is explicitly acknowledged in the paper's limitations.
  • The same KG primitives could plausibly serve as dense, verifiable rewards for reinforcement learning, turning the knowledge graph into a simulatable training environment; the paper mentions this only as a future direction.
  • Because the trained specialist is only 32 billion parameters, a practical consequence the authors raise in their discussion is that specialized superintelligence could be achieved at a fraction of the training and inference energy cost of a frontier generalist.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a bottom-up curriculum for language models in which multi-hop paths from a domain knowledge graph are converted into multiple-choice QA tasks with LLM-generated thinking traces, and the model is fine-tuned on this curriculum. The authors instantiate the approach in medicine using the UMLS KG, producing 24,000 training tasks, and introduce ICD-Bench, a 3,675-question benchmark across 15 ICD-derived categories. Fine-tuning QwQ-32B yields QwQ-Med-1/2/3, and the paper reports that QwQ-Med-3 outperforms open-source and proprietary reasoning baselines on ICD-Bench, that deeper curricula improve hardest-task accuracy, and that the model transfers to external medical QA benchmarks.

Significance. If the central claims hold, the paper offers a tractable data-centric recipe for injecting structured domain knowledge into small reasoning models, with a clearly described pipeline, a new benchmark, and careful ablations of path depth, diversity, and compute allocation. The authors also release the ICD-Bench suite and provide honest limitations. However, the validity of the headline 'domain-specific superintelligence' result depends critically on whether ICD-Bench is truly out-of-distribution relative to the training curriculum and on whether baseline comparisons are made at matched inference budgets; both of these conditions are currently in doubt.

major comments (4)
  1. [§4.1 and §5.1] ICD-Bench is generated with the same QA-generation pipeline (invoking §3.1.1), the same UMLS KG, and the same Gemini-2.0-Flash + dual-grader filtering as the training curriculum, and the decontamination in §5.1 excludes only exact KG-path matches and exact 18-gram text overlaps while explicitly permitting partial path overlaps. With ~10k nodes and ~44k edges (Appendix A.1), many 2- and 3-hop ICD-Bench paths necessarily share nodes, edges, and paraphrastic vignette structure with training paths, so the reported 10-20% ICD-Bench advantage (O2.1) may reflect learning the distribution of Gemini-generated questions rather than acquiring composable medical primitives. The central superintelligence claim therefore needs support from either a generator-shift evaluation (e.g., a different LLM or template for ICD-Bench), a held-out KG split, or an analysis showing that performance persists when the benchmark's surface style is varied.
  2. [§6.2 baseline setup] The comparison against proprietary models is confounded by inference budget: o3 and Gemini-2.5-Pro are evaluated at pass@1, while QwQ-Med models are evaluated with majority voting over up to 16 parallel samples, and the open-source baselines also receive parallel scaling. The claim that curriculum-tuned models 'significantly outperform' proprietary models (O2.1) is not established unless all models are compared at matched inference budgets or the advantage is shown to persist at pass@1 with comparable sampling. A matched-budget comparison, at least for the open-source models, should be reported.
  3. [Table 1 and §6.6 (O6.1)] The transfer claim is contradicted by the paper's own numbers: QwQ-Med-3 is below the base QwQ on MedQA (82.72 vs 85.62; with parallel scaling, 85.39 vs 87.09), yet O6.1 states that curriculum-tuned models 'reliably transfer acquired KG primitives.' The paper should acknowledge this regression explicitly and explain why the mixed external results still support the transfer claim, or temper the claim accordingly.
  4. [§6.3 and Appendix E] The paper repeatedly uses KG path length as a proxy for reasoning difficulty (Sections 3.1, O4.1, O4.2), but Appendix E reports no meaningful correlation between path length and base-model pass@1 rate, and the difficulty estimator in §6.3 is itself built from base-model pass@1 over 16 samples. This undercuts the interpretation that deeper paths correspond to a harder curriculum and that the gains from QwQ-Med-3 on 'hardest' tasks come from compositional multi-hop learning rather than from other properties of longer training examples. The authors should either provide direct evidence that path length tracks an independent measure of difficulty or reframe the curriculum claim in terms of the actual difficulty bins they define.
minor comments (4)
  1. [Abstract and §6.2 (O2.1)] The 10-20% improvement range is described without specifying whether it is absolute or relative accuracy; please clarify, and state whether the improvement is statistically significant after multiple-comparison correction across the 15 categories.
  2. [Figure 5] The bottom panel labeled 'Training Curriculum Thinking Trace Distribution' appears to show hop-wise trace lengths; the axes and caption should be made explicit so the reader can verify the claimed variance in reasoning complexity.
  3. [Table 1] The entries 'QwQ/parallel-scaling' and 'QwQ-Med-3/parallel-scal.' do not state the number of parallel samples used; please report K and the temperature so the results are reproducible.
  4. [§6.1 (O1.1)] The claim that 'parallel scaling outperforms iterative refinement with increasing curriculum' is supported mainly by visual inspection of Fig. 6; since the dotted and solid curves are close for several configurations, provide numerical deltas or error bars for the key comparisons.

Circularity Check

1 steps flagged · score 6.0 of 10

ICD-Bench is drawn from the same UMLS-KG QA pipeline as the 24,000-task training curriculum; with only exact-path/18-gram decontamination, the headline benchmark advantage is largely an in-distribution result.

  1. fitted input called prediction [Section 4.1 (ICD-Bench Construction) and Section 5.1 (Decontamination)]
    "We use the stratified KG to generate hop-controlled QA items per category, using our QA generation method (Section 3.1.1)... The final ICD-Bench evaluation suite comprises 3,675 high-quality QA items, systematically generated through the controlled pipeline described above. ... However, we allow partial path overlaps, as our goal is to allow models to learn and generalize from individual KG primitives."

    ICD-Bench is not independent: Section 4.1 constructs it 'using our QA generation method (Section 3.1.1)' — the same pipeline that produced the 24,000 training tasks (Section 5.1), with the same UMLS KG, the same Gemini 2.0 Flash QA generator, and the same filters. Decontamination only drops exact KG-path matches and exact 18-gram text overlaps, explicitly allowing partial path overlaps. On a ~10k-node, ~44.5k-edge KG, ICD-Bench paths share many nodes/edges with training paths, so models can match the training distribution's vignette style, option phrasing, and distractor patterns. The reported 10–20% advantage is thus substantially built into the construction. External benchmarks are mixed (MedQA: QwQ-Med-3 82.72 vs QwQ 85.62).

full rationale

Score is 6 rather than 0 because the central benchmark claim is evaluated on a test set built by the same generator and KG as the training curriculum; the exact-path/18-gram decontamination does not remove distributional overlap. This is a partial circularity: the model is fine-tuned on Gemini-generated KG tasks and then 'predicts' on Gemini-generated KG tasks. The external benchmarks (MedQA, MedMCQA, MMLU-Med, PubMedQA) are genuinely independent and would lower the score if they strongly and consistently supported transfer, but Table 1 shows QwQ-Med-3 underperforms the base QwQ on MedQA both with and without parallel scaling, so they do not. I found no load-bearing self-citation or imported uniqueness theorem; the circularity is in the benchmark construction, not in the citation chain.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the reliability of the UMLS KG, on the assumption that LLM-generated vignettes faithfully encode path-based reasoning, and on the validity of ICD-Bench as an independent evaluation. The benchmark and training data share the same generation pipeline, which is the main circularity burden. No external entity is introduced beyond the benchmark itself.

free parameters (5)
  • Maximum training path length N=3 = 3
    Paths longer than three hops were observed to be semantically unmeaningful (Section 5.1); this cutoff is an empirical choice that shapes the curriculum.
  • Decontamination n-gram threshold = 18
    An n-gram overlap threshold used to filter QA pairs similar to ICD-Bench (Section 5.1); chosen by hand, not derived.
  • Difficulty bin cutoffs = Empirical cutoffs on base-model pass@1 over 16 samples
    Five difficulty bins were segmented so that base accuracy declines roughly linearly (Appendix E); cutoffs are data-dependent.
  • SFT epochs per model = 24, 12, 8 for QwQ-Med-1/2/3
    Adjusted to match an equal FLOPs budget across datasets of size 8K/16K/24K (Section 5.2).
  • Grader agreement rule = Two-grader agreement
    QA items are retained only if both Gemini-2.0-Flash and Qwen-2.5-72B judge them correct; a design choice that defines the dataset.
assumptions (5)
  • domain assumption UMLS KG triples are medically correct and the relations are meaningful.
    The whole curriculum and benchmark are built on this KG from prior work [45]; if triples are wrong, tasks and answers are wrong.
  • ad hoc to paper An LLM-generated vignette from a KG path requires traversing that path to solve.
    The pipeline prompts Gemini to create a question testing the path (Section 3.1.1); there is no verification that the path is necessary, only that two graders say the answer follows.
  • domain assumption Two-LLM grader agreement guarantees factual correctness of the generated tasks.
    Graders may share systematic errors, and the prompt is binary; no human evaluation is reported (Section 3.1.2).
  • ad hoc to paper Base-model pass@1 over 16 samples is a valid difficulty oracle.
    Used to stratify ICD-Bench into difficulty bins (Section 6.3 and Appendix E).
  • ad hoc to paper Decontamination rules prevent benchmark contamination.
    The 18-gram and exact-path filters are heuristic; style and entity distribution overlap remain (Section 5.1).
invented entities (1)
  • ICD-Bench
    purpose: A 3,675-question benchmark across 15 ICD categories used to claim domain-specific superintelligence.
    Constructed with the same KG and the same LLM-based QA generation pipeline as the training data; no external validation that ICD-Bench performance predicts real clinical reasoning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Need." pith.science (2026). https://pith.science/paper/P3IOU2MD

@misc{pith2026250713966,
  author       = {Pith},
  title        = {Pith review of: Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Need},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P3IOU2MD}},
  note         = {Machine review of arXiv:2507.13966}
}
read the original abstract

Language models traditionally used for cross-domain generalization have recently demonstrated task-specific reasoning. However, their top-down training approach on general corpora is insufficient for acquiring abstractions needed for deep domain expertise. This may require a bottom-up approach that acquires expertise by learning to compose simple domain concepts into more complex ones. A knowledge graph (KG) provides this compositional structure, where domain primitives are represented as head-relation-tail edges and their paths encode higher-level concepts. We present a task generation pipeline that synthesizes tasks directly from KG primitives, enabling models to acquire and compose them for reasoning. We fine-tune language models on the resultant KG-grounded curriculum to demonstrate domain-specific superintelligence. While broadly applicable, we validate our approach in medicine, where reliable KGs exist. Using a medical KG, we curate 24,000 reasoning tasks paired with thinking traces derived from diverse medical primitives. We fine-tune the QwQ-32B model on this curriculum to obtain QwQ-Med-3 that takes a step towards medical superintelligence. We also introduce ICD-Bench, an evaluation suite to quantify reasoning abilities across 15 medical domains. Our experiments demonstrate that QwQ-Med-3 significantly outperforms state-of-the-art reasoning models on ICD-Bench categories. Further analysis reveals that QwQ-Med-3 utilizes acquired primitives to widen the performance gap on the hardest tasks of ICD-Bench. Finally, evaluation on medical question-answer benchmarks shows that QwQ-Med-3 transfers acquired expertise to enhance the base model's performance. While the industry's approach to artificial general intelligence (AGI) emphasizes broad expertise, we envision a future in which AGI emerges from the composable interaction of efficient domain-specific superintelligent agents.

Figures

Figures reproduced from arXiv: 2507.13966 by the authors.

Figure 1
Figure 1. We traverse paths on a medical KG to generate [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Generating QA tasks from a KG path. We explore KG paths to derive QA pairs grounded in the KG. We choose a KG path by starting from an initial node (left) and iteratively sampling (relation, entity) pairs from the current node’s neighbors to obtain an N-hop path p N (middle). The sampled path is mapped to a natural language QA task by prompting a backend LLM (right). The bottom of the diagram shows an example of a g… view at source ↗
Figure 3
Figure 3. Overview of our curriculum curation pipeline. Starting from KG-derived paths, we sample for node diversity and path complexity (Steps 1—2), followed by quality filtering of generated QA pairs (Step 3). We then generate thinking traces using a strong reasoning LLM grounded in the KG path (Step 4) and, finally, perform correctness filtering using two independent grader LLMs to ensure factual trace alignment to the KG … view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: ICD-Bench evaluation suite Top: The 15 medical sub-specialties derived from the ICD-10 taxonomy, each corresponding to a distinct category in the benchmark. Each node in the UMLS KG is mapped to one or more of these categories to guide domain-specific QA generation. Bo…
Figure 5
Figure 5. Figure 5: Distributional statistics of the curated training curriculum. The dataset spans 24,000 QA items grounded in UMLS KG paths. Left: Breakdown of entities in the sampled KG paths, across the ICD categories they belong to. Right: Distribution of relation types along sampled…
Figure 6
Figure 6. Figure 6: Inference-time scaling curves for curriculum-tuned models on ICD-Bench. Top row: Each plot displays per-model curves where solid lines denote pure parallel scaling and dotted lines denote iterative refinement augmentation. Deeper curriculum models (QwQ-Med-2, QwQ-Med-3…
Figure 7
Figure 7. Figure 7: Domain-specific performance of our curriculum-tuned models across ICD-Bench categories. Curriculum-tuned models significantly outperform proprietary and open-source reasoning baselines across medical sub-specialities. Moreover, curriculum-tuned models demonstrate impro…
Figure 8
Figure 8. Figure 8: Performance across task difficulty bins on ICD-Bench. We construct a task-difficulty estimator using the pass@1 rate of the base model. Left: All models show declining accuracy with increasing difficulty, validating our pass@1-based difficulty estimator. Curriculum-tun…
Figure 9
Figure 9. Figure 9: Disentangling the effects of KG path depth, complexity sampling, and diversity sampling on curriculum-tuned model performance. Left: Performance improves with increased thinking tokens as curricula incorporate deeper paths (3-hop), balanced path-length sampling, and gr…
Figure 10
Figure 10. Figure 10: Disentangling recall and reasoning performance across ICD-Bench tasks stratified by KG path length. Dot size denotes recall (fraction of path entities recovered in the reasoning trace) and saturation reflects reasoning accuracy. Curriculum-tuned models demonstrate hig…
Figure 11
Figure 11. Figure 11: Distributional statistics of the KG. Top row: On the left, we visualize the top-200 nodes with the highest degree in a word cloud. On the right, we plot the histogram of node degrees. The KG is fairly sparse, and the weight is significantly concentrated on single-degr…
Figure 12
Figure 12. Figure 12: Training run details of our curriculum-tuned models. [PITH_FULL_IMAGE:figures/full_fig_p035_12.png]
Figure 13
Figure 13. Figure 13: Top: Distribution of pass@1 score of the QwQ-Base-32B model over 16 samples, across ICD-Bench questions. Bottom: Mean KG-path length of questions across pass@1 scores. 44 [PITH_FULL_IMAGE:figures/full_fig_p044_13.png]
Figure 14
Figure 14. Figure 14: ICD-Bench performance of different models across difficulty levels within each ICD [PITH_FULL_IMAGE:figures/full_fig_p045_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Structured In-context Environment Scaling for Large Language Model Reasoning

    cs.CL 2025-09 conditional novelty 6.0 of 10

    SIE framework automatically constructs scalable, verifiable reasoning environments from structured data, improving in-domain performance and enabling generalization to out-of-domain math and logic tasks.

  2. Knowledge Graph-Driven Expert-Level Reasoning for Neuroscience

    cs.CL 2026-05 unverdicted novelty 4.0 of 10

    A textbook-derived neuroscience knowledge graph supplies synthetic multi-hop QA supervision and RL rewards to fine-tune a small LM claimed to exceed larger general models on expert reasoning.

  3. MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMs

    cs.CL 2026-02 unverdicted novelty 4.0 of 10

    MedXIAOHE is a medical MLLM that claims state-of-the-art benchmark performance through specialized pretraining to cover long-tail diseases and RL-based reasoning training.

  4. A Vision Toward Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents

    cs.AI 2025-10 unverdicted novelty 2.0 of 10

    A position paper proposing compact, domain-specific AI agents as the path to ≥1000× energy efficiency, without demonstrating the claim.

Reference graph

Works this paper leans on

139 extracted references · 32 canonical work pages · cited by 4 Pith papers

  1. [1]

    Hello GPT-4o, 2024

    OpenAI. Hello GPT-4o, 2024. System Card and Technical Overview. https://openai. com/index/hello-gpt-4o/

    2024

  2. [2]

    Introducing Claude 4

    Anthropic. Introducing Claude 4. Anthropic News, 2025. Accessed via Anthropic website. https://www.anthropic.com/news/claude-4

    2025

  3. [3]

    OLMo: Accelerating the Science of Language Models

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. OLMo: Accelerating the Science of Language Models. CoRR, abs/2402.00838, 2024

    arXiv 2024

  4. [4]

    Gemini 1.5: Unlocking Multimodal Under- standing Across Millions of Tokens of Context

    Gemini Team, DeepMind, and Google Research. Gemini 1.5: Unlocking Multimodal Under- standing Across Millions of Tokens of Context. CoRR, abs/2403.05530, 2024

    arXiv 2024

  5. [5]

    DeepSeek-V3 Technical Report

    DeepSeek-AI. DeepSeek-V3 Technical Report. CoRR, abs/2412.19437, 2024

    arXiv 2024

  6. [6]

    Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of Experts. CoRR, abs/2401.04088, 2024

    arXiv 2024

  7. [7]

    Qwen3 Technical Report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, et al. Qwen3 Technical Report. CoRR, abs/2505.09388, 2025

    arXiv 2025

  8. [8]

    The Llama 4 Herd: The Beginning of a New Era of Natively Multi- modal AI Innovation

    Meta AI. The Llama 4 Herd: The Beginning of a New Era of Natively Multi- modal AI Innovation. Meta AI Blog, Apr. 2025. https://ai.meta.com/blog/ llama-4-multimodal-intelligence/

    2025

Show all 139 references
  1. [9]

    Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L

    Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive Architectures for Language Agents. CoRR, abs/2309.02427, 2023. 20

    2023 arXiv

  2. [10]

    O’Brien, Carrie J

    Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith R. Morris, Percy Liang, and Michael S. Bernstein. Generative Agents: Interactive Simulacra of Human Behavior. CoRR, abs/2304.03442, 2023

    2023 arXiv

  3. [11]

    Claude Code: Deep Coding at Terminal Velocity

    Anthropic. Claude Code: Deep Coding at Terminal Velocity. Anthropic Engineering Blog,

  4. [12]

    Introducing Codex

    OpenAI. Introducing Codex. OpenAI Blog, 2025. Research preview; Codex-1 model. https://openai.com/index/introducing-codex/

    2025

  5. [13]

    Introducing Deep Research

    OpenAI. Introducing Deep Research. OpenAI Blog, February 2025. Launch of the Deep Research Feature Within ChatGPT, Powered by a Specialized Version of the o3 Model. https://openai.com/index/introducing-deep-research/

    2025

  6. [14]

    Gemini Deep Research: Your Personal Research Assistant

    Google Gemini Team. Gemini Deep Research: Your Personal Research Assistant. Google Gemini Website, 2024. Agentic Research Feature Using Large Context Window and Search. https://gemini.google/overview/deep-research/

    2024

  7. [15]

    Superintelligence: Paths, Dangers, Strategies

    Nick Bostrom. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, 2014

    2014

  8. [16]

    Weak-to-Strong Generalization: Eliciting Strong Capabilities with Weak Supervision

    Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschen- brenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeff Wu. Weak-to-Strong Generalization: Eliciting Strong Capabilities with Weak Supervision. CoRR, abs...

    2023 arXiv

  9. [17]

    Highly Accurate Protein Structure Prediction with AlphaFold

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, et al. Highly Accurate Protein Structure Prediction with AlphaFold. Nature, 596:583–589, 2021

    2021

  10. [18]

    Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, and Marco Selvi

    Daniel J. Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, and Marco Selvi. Faster Sorting Algorithms Discovered Using Deep Reinforcement Learning. Nature, 2023

    2023

  11. [19]

    Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J

    Alexander Novikov, Ngan Vu, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Z. Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet ...

    2025 arXiv

  12. [20]

    AlphaGenome: AI for Better Understanding the Genome

    Google DeepMind. AlphaGenome: AI for Better Understanding the Genome. Google DeepMind Blog, Jun. 2025. https://deepmind.google/discover/blog/ alphagenome-ai-for-better-understanding-the-genome/

    2025

  13. [21]

    Towards an AI Co-Scientist

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, Khaled Saab, Dan Popovici, Jacob Blum, Fan Zhang, and Katherine and Chou et al. Towards an AI Co-Scientist. CoRR, abs/2502.1...

    2025 arXiv

  14. [22]

    Schoenholz, Muratahan Aykol, Gowoon Cheon, Ekin Dogus Cubuk, et al

    Amil Merchant, Simon Batzner, Samuel S. Schoenholz, Muratahan Aykol, Gowoon Cheon, Ekin Dogus Cubuk, et al. Scaling Deep Learning for Materials Discovery. Nature, 614:1234– 1240, 2023

    2023

  15. [23]

    Durrant, Jerome Ku, Michael Poli, Greg Brockman, Daniel Chang, Gabriel A

    Garyk Brixi, Matthew G. Durrant, Jerome Ku, Michael Poli, Greg Brockman, Daniel Chang, Gabriel A. Gonzalez, Samuel H. King, David B. Li, Aditi T. Merchant, Mohsen Naghipourfar, Eric Nguyen, Chiara Ricci-Tam, David W. Romero, Gwanggyu Sun, Ali Taghibakshi, Anton V orontsov, Bra...

    2025

  16. [24]

    Rae, Oriol Vinyals, and Laurent Sifre

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osin...

    2022 arXiv

  17. [25]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, et al

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, et al. Language Models are Few-Shot Learners. CoRR, abs/2005.14165, 2020

    2005 arXiv

  18. [26]

    Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters. CoRR, abs/2408.03314, 2024

    2024 arXiv

  19. [27]

    Interpretable Contrastive Monte Carlo Tree Search Reasoning

    Zitian Gao, Boye Niu, Xuzheng He, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, and Lijie Wen. Interpretable Contrastive Monte Carlo Tree Search Reasoning. CoRR, abs/2410.01707, 2024

    2024 arXiv

  20. [28]

    s1: Simple Test-Time Scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple Test-Time Scaling. CoRR, abs/2501.19393, 2025

    2025 arXiv

  21. [29]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. CoRR, abs/2501.12948, 2025

    2025 arXiv

  22. [30]

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester J. V . Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Øyvind Tafjord, Chris Wilhelm, Luca ...

    2024 arXiv

  23. [31]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent Abilities of Large Language Models. C...

    2022 arXiv

  24. [32]

    Are Emergent Abilities of Large Language Models a Mirage? CoRR, abs/2304.15004, 2023

    Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are Emergent Abilities of Large Language Models a Mirage? CoRR, abs/2304.15004, 2023

    2023 arXiv

  25. [33]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. CoRR, abs/2305.04388, 2023

    2023 arXiv

  26. [34]

    What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

    Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What Can Transformers Learn In-Context? A Case Study of Simple Function Classes. CoRR, abs/2208.01066, 2023

    2023 arXiv

  27. [35]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the Middle: How Language Models Use Long Contexts. CoRR, abs/2307.03172, 2023

    2023 arXiv

  28. [36]

    Safety Alignment Should Be Made More Than Just a Few Tokens Deep

    Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety Alignment Should Be Made More Than Just a Few Tokens Deep. CoRR, abs/2406.05946, 2024

    2024 arXiv

  29. [37]

    ARC- AGI-2: A New Challenge for Frontier AI Reasoning Systems

    Francois Chollet, Mike Knoop, Gregory Kamradt, Bryan Landers, and Henry Pinkard. ARC- AGI-2: A New Challenge for Frontier AI Reasoning Systems. CoRR, abs/2505.11831, 2025

    2025 arXiv

  30. [38]

    The Origin of Concepts

    Susan Carey. The Origin of Concepts. Oxford University Press, New York, 2009

    2009

  31. [39]

    Tenenbaum, Charles Kemp, Thomas L

    Joshua B. Tenenbaum, Charles Kemp, Thomas L. Griffiths, and Noah D. Goodman. How to Grow a Mind: Statistics, Structure, and Abstraction. Science, 331, 2011

    2011

  32. [40]

    Lake, Tomer D

    Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. Building Machines That Learn and Think Like People. CoRR, abs/1604.00289, 2016

    2016 arXiv

  33. [41]

    Tenenbaum

    Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lucas Morales, Luke Hewitt, Armando Solar-Lezama, and Joshua B. Tenenbaum. DreamCoder: Growing Generalizable, Interpretable Knowledge with Wake-sleep Bayesian Program Learning. CoRR, abs/2006.08381, 2020. 22

    2006 arXiv

  34. [42]

    Efficient Probabilistic Logic Reasoning with Graph Neural Networks

    Yuyu Zhang, Xinshi Chen, Yuan Yang, Arun Ramamurthy, Bo Li, Yuan Qi, and Le Song. Efficient Probabilistic Logic Reasoning with Graph Neural Networks. CoRR, abs/2001.11850, 2020

    2001 arXiv

  35. [43]

    Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S. Yu. A Survey on Knowledge Graphs: Representation, Acquisition, and Applications. IEEE Transactions on Neural Networks and Learning Systems, 33, 2022

    2022

  36. [44]

    The Unified Medical Language System (UMLS): Integrating Biomedical Terminology

    Olivier Bodenreider. The Unified Medical Language System (UMLS): Integrating Biomedical Terminology. Nucleic Acids Research, 32:D267–D270, 2004

    2004

  37. [45]

    QA- GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering

    Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. QA- GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistic...

    2021

  38. [46]

    International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10)

    World Health Organization. International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10). World Health Organization, 1992. https://icd. who.int/browse10/2019/en

    1992

  39. [47]

    DBpedia: A Nucleus for a Web of Open Data

    Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. DBpedia: A Nucleus for a Web of Open Data. Lecture Notes in Computer Science (in- cluding subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 4...

    2007

  40. [48]

    Introducing the Knowledge Graph: Things, Not Strings, 2012

    Amit Singhal. Introducing the Knowledge Graph: Things, Not Strings, 2012. https: //blog.google/products/search/introducing-knowledge-graph-things-not/

    2012

  41. [49]

    Rindflesch

    Halil Kilicoglu, Dongwook Shin, Marcelo Fiszman, Graciela Rosemblat, and Thomas C. Rindflesch. SemMedDB: A PubMed-Scale Repository of Biomedical Semantic Predications. Bioinformatics, 28, 2012

    2012

  42. [50]

    Chen, Dexter Hadley, Ari Green, Pouya Khankhanian, and Sergio E

    Daniel Scott Himmelstein, Antoine Lizee, Christine Hessler, Leo Brueggeman, Sabrina L. Chen, Dexter Hadley, Ari Green, Pouya Khankhanian, and Sergio E. Baranzini. Systematic Integration of Biomedical Knowledge Prioritizes Drugs for Repurposing. eLife, 6, 2017

    2017

  43. [51]

    Wishart, Yannick D

    David S. Wishart, Yannick D. Feunang, An C. Guo, Elvis J. Lo, Ana Marcu, Jason R. Grant, et al. DrugBank 5.0: A Major Update to the DrugBank Database for 2018. Nucleic Acids Research, 46(D1):D1074–D1082, Jan. 2018

    2018

  44. [52]

    Gemini 2.5 Flash Model Card, 2025

    Google DeepMind. Gemini 2.5 Flash Model Card, 2025. https://deepmind.google/ models/gemini/flash/

    2025

  45. [53]

    Gemini 2.5 Pro Model Card, 2025

    Google DeepMind. Gemini 2.5 Pro Model Card, 2025. https://deepmind.google/ models/gemini/pro/

    2025

  46. [54]

    Qwen2.5 Technical Report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2025 arXiv

  47. [55]

    Self-Consistency Improves Chain of Thought Reasoning in Language Models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-Consistency Improves Chain of Thought Reasoning in Language Models. CoRR, abs/2203.11171, 2023

    2023 arXiv

  48. [56]

    QwQ-32B: Embracing the Power of Reinforcement Learning, 2025

    Qwen Team. QwQ-32B: Embracing the Power of Reinforcement Learning, 2025. https: //qwenlm.github.io/blog/qwq-32b/

    2025

  49. [57]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models.CoRR, abs/2106.09685, 2021. 23

    2021 arXiv

  50. [58]

    Introducing OpenAI o3 and o4-mini, 2025

    OpenAI. Introducing OpenAI o3 and o4-mini, 2025. https://openai.com/index/ introducing-o3-and-o4-mini/

    2025

  51. [59]

    Small Language Models Learn Enhanced Reasoning Skills from Medical Textbooks

    Hyunjae Kim, Hyeon Hwang, Jiwoo Lee, Sihyeon Park, Dain Kim, Taewhoo Lee, Chanwoong Yoon, Jiwoong Sohn, Jungwoo Park, Olga Reykhart, Thomas Fetherston, Donghee Choi, Soo Heon Kwak, Qingyu Chen, and Jaewoo Kang. Small Language Models Learn Enhanced Reasoning Skills from Medical...

    2025

  52. [60]

    MedGemma Model Card | Health AI Developer Foundations | Google for Develop- ers, 2025

    Google. MedGemma Model Card | Health AI Developer Foundations | Google for Develop- ers, 2025. https://developers.google.com/health-ai-developer-foundations/ medgemma/model-card

    2025

  53. [61]

    Sky-T1: Fully Open-Source Reasoning Model with o1-Preview Performance in $450 Budget, 2025

    NovaSky Team. Sky-T1: Fully Open-Source Reasoning Model with o1-Preview Performance in $450 Budget, 2025. https://novasky-ai.github.io/posts/sky-t1

    2025

  54. [62]

    What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams

    Di Jin, Eileen Pan, Nassim Oufattole, Wei Hung Weng, Hanyi Fang, and Peter Szolovits. What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams. Applied Sciences (Switzerland), 11, 2020

    2020

  55. [63]

    MedMCQA : A Large- Scale Multi-Subject Multi-Choice Dataset for Medical Domain Question Answering

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. MedMCQA : A Large- Scale Multi-Subject Multi-Choice Dataset for Medical Domain Question Answering. In Proceedings of Machine Learning Research, 174:248–260, Apr. 2022

    2022

  56. [64]

    Measuring Massive Multitask Language Understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring Massive Multitask Language Understanding. In Proceedings of the 9th International Conference on Learning Representations, 2020

    2020

  57. [65]

    PubMedQA: A Dataset for Biomedical Research Question Answering

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A Dataset for Biomedical Research Question Answering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural La...

    2019

  58. [66]

    Benchmarking Reasoning Robustness in Large Language Models

    Tong Yu, Yongcheng Jing, Xikun Zhang, Wentao Jiang, Wenjie Wu, Yingjie Wang, Wenbin Hu, Bo Du, and Dacheng Tao. Benchmarking Reasoning Robustness in Large Language Models. CoRR, abs/2503.04550, 2025

    2025 arXiv

  59. [67]

    Longwell, Ian Hirsch, Fernando Binder, Galileo Arturo Gonzalez Conchas, Daniel Mau, Raymond Jang, Rahul G

    Jack B. Longwell, Ian Hirsch, Fernando Binder, Galileo Arturo Gonzalez Conchas, Daniel Mau, Raymond Jang, Rahul G. Krishnan, and Robert C. Grant. Performance of Large Language Models on Medical Oncology Examination Questions. JAMA Network Open, 7, 2024

    2024

  60. [68]

    KGARevion: An AI Agent for Knowledge-Intensive Biomedical QA

    Xiaorui Su, Yibo Wang, Shanghua Gao, Xiaolong Liu, Valentina Giunchiglia, Djork-Arné Clevert, and Marinka Zitnik. KGARevion: An AI Agent for Knowledge-Intensive Biomedical QA. CoRR, abs/2410.04660, 2024

    2024 arXiv

  61. [69]

    Patterson, Matthew M

    Yanjun Gao, Ruizhe Li, Emma Croxford, Samuel Tesch, Daniel To, John Caskey, Brian W. Patterson, Matthew M. Churpek, Timothy Miller, Dmitriy Dligach, and Majid Afshar. Large Language Models and Medical Knowledge Grounding for Diagnosis Prediction. medRxiv, 2024

    2024

  62. [70]

    Selective UMLS Knowledge Infusion for Biomedical Question Answering

    Hyeryun Park, Jiye Son, Jeongwon Min, and Jinwook Choi. Selective UMLS Knowledge Infusion for Biomedical Question Answering. Scientific Reports, 13, 2023

    2023

  63. [71]

    AgentClinic: A Multimodal Agent Benchmark to Evaluate AI in Simulated Clinical Environments

    Samuel Schmidgall, Rojin Ziaei, Carl Harris, Ji Woong Kim, Eduardo Reis, Jeffrey Jopling, and Michael Moor. AgentClinic: A Multimodal Agent Benchmark to Evaluate AI in Simulated Clinical Environments. CoRR, abs/2405.07960:2025–2030, 2024

    2025 arXiv

  64. [72]

    Yu Chen, Lingfei Wu, and Mohammed J. Zaki. Toward Subgraph-Guided Knowledge Graph Question Generation with Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 35(9):12706–12717, Apr. 2023

    2023

  65. [73]

    Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering

    Jing Zhang, Xiaokang Zhang, Jifan Yu, Jian Tang, Jie Tang, Cuiping Li, and Hong Chen. Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 1, 2022. 24

    2022

  66. [74]

    Explore then Determine: A GNN- LLM Synergy Framework for Reasoning over Knowledge Graph

    Guangyi Liu, Yongqi Zhang, Yong Li, and Quanming Yao. Explore then Determine: A GNN- LLM Synergy Framework for Reasoning over Knowledge Graph. CoRR, abs/2406.01145, 2024

    2024 arXiv

  67. [75]

    A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal

    Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, Fuchun Sun, and Kunlun He. A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12...

    2024

  68. [76]

    AlpaCare: Instruction-tuned Large Language Models for Medical Application

    Xinlu Zhang, Chenxin Tian, Xianjun Yang, Lichang Chen, Zekun Li, and Linda Ruth Pet- zold. AlpaCare: Instruction-tuned Large Language Models for Medical Application. CoRR, abs/2310.14558, 2023

    2023 arXiv

  69. [77]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 1, 2022

    2022

  70. [78]

    HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs. CoRR, abs/2412.18925, 2024

    2024 arXiv

  71. [79]

    How to Design, Create, and Evaluate an Instruction-Tuning Dataset for Large Language Model Training in Health Care: Tutorial From a Clinical Perspective

    Wojciech Nazar, Grzegorz Nazar, Aleksandra Kami ´nska, and Ludmila Danilowicz- Szymanowicz. How to Design, Create, and Evaluate an Instruction-Tuning Dataset for Large Language Model Training in Health Care: Tutorial From a Clinical Perspective. Journal of Medical Internet Res...

    2025

  72. [80]

    KG-QAGen: A Knowledge-Graph-Based Framework for Systematic Question Generation and Long-Context LLM Evaluation

    Nikita Tatarinov, Vidhyakshaya Kannan, Haricharana Srinivasa, Arnav Raj, Singh Anand, Varun Singh, Aditya Luthra, Ravij Lade, Agam Shah, and Sudheer Chava. KG-QAGen: A Knowledge-Graph-Based Framework for Systematic Question Generation and Long-Context LLM Evaluation. CoRR, abs...

    2025

  73. [81]

    Curriculum Learning

    Yoshua Bengio, Jerome Louradour, Ronan Collobert, and Jason Weston. Curriculum Learning. ACM International Conference Proceeding Series, 382, 2009

    2009

  74. [82]

    Curriculum Learning: A Survey

    Petru Soviany, Tudor Radu, Paolo Rota, and Nicu Sebe. Curriculum Learning: A Survey. International Journal of Computer Vision, 130, 2022

    2022

  75. [83]

    Curriculum Learning: Theories, Approaches, Applications, Tools, and Future Directions in the Era of Large Language Models

    Xin Wang, Yuwei Zhou, Hong Chen, and Wenwu Zhu. Curriculum Learning: Theories, Approaches, Applications, Tools, and Future Directions in the Era of Large Language Models. Companion Proceedings of the ACM Web Conference, 2024

    2024

  76. [84]

    Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning

    Xuetao Ma, Wenbin Jiang, and Hua Huang. Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning. CoRR, abs/2502.15401, 2025

    2025

  77. [85]

    Let’s Be Self-Generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Models

    Kangyang Luo, Zichen Ding, Zhenmin Weng, Lingfeng Qiao, Meng Zhao, Xiang Li, Di Yin, and Jinlong Shu. Let’s Be Self-Generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Models. CoRR, abs/2410.21728, 2024

    arXiv

  78. [86]

    LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs

    Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ah- san, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, and Salman Khan. LlamaV-o1: Rethinking Step-by-step Visual Rea...

    2025 arXiv

  79. [87]

    Can an Easy-to-Hard Curriculum Make Reasoning Emerge in Small Language Models? Evidence from a Four-Stage Curriculum on GPT-2

    Xiang Fu. Can an Easy-to-Hard Curriculum Make Reasoning Emerge in Small Language Models? Evidence from a Four-Stage Curriculum on GPT-2. CoRR, abs/2505.11643, 2025

    2025 arXiv

  80. [88]

    Curriculum Learning for Small Code Language Models

    Marwa Nair, Kamel Yamani, Lynda Said Lhadj, and Riyadh Baghdadi. Curriculum Learning for Small Code Language Models. CoRR, abs/2407.10194, 2024

    2024 arXiv

  81. [89]

    Goedel-Prover: A Frontier Model for Open- Source Automated Theorem Proving

    Yong Lin, Shange Tang, Bohan Lyu, Jiayun Wu, Hongzhou Lin, Kaiyu Yang, Jia Li, Mengzhou Xia, Danqi Chen, Sanjeev Arora, and Chi Jin. Goedel-Prover: A Frontier Model for Open- Source Automated Theorem Proving. CoRR, abs/2502.07640, 2025. 25

    2025 arXiv

  82. [90]

    Trinh, Miroslav Olšák, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V

    Yuri Chervonyi, Trieu H. Trinh, Miroslav Olšák, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V . Le, and Thang Luong. Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2. CoRR, abs/2502.03544, 2025

    2025

  83. [91]

    Toward Scientific Rea- soning in LLMs: Training from Expert Discussions via Reinforcement Learning

    Ming Yin, Yuanhao Qu, Ling Yang, Le Cong, and Mengdi Wang. Toward Scientific Rea- soning in LLMs: Training from Expert Discussions via Reinforcement Learning. CoRR, abs/2505.19501, 2025

    2025 arXiv

  84. [92]

    Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data

    Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn. Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data. CoRR, abs/2211.04325, 2024

    2024 arXiv

  85. [93]

    Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? CoRR, abs/2504.13837, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? CoRR, abs/2504.13837, 2025

    2025 arXiv

  86. [94]

    Generative AI’s Crippling and Widespread Inability to Reason

    Gary Marcus. Generative AI’s Crippling and Widespread Inability to Reason. https:// garymarcus.substack.com/p/generative-ais-crippling-and-widespread , 2025. Accessed: 2025-08-30

    2025

  87. [95]

    AI 2027: We Predict That the Impact of Superhuman AI Over the Next Decade Will Be Enormous, Exceeding That of the Industrial Revolution, Apr

    Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean. AI 2027: We Predict That the Impact of Superhuman AI Over the Next Decade Will Be Enormous, Exceeding That of the Industrial Revolution, Apr. 2025.https://ai-2027.com/ai-2027. pdf

    2027

  88. [96]

    Society of Mind

    Marvin Minsky.The Society of Mind. Simon & Schuster, New York, 1986. First Comprehensive Presentation of the "Society of Mind" Theory

    1986

  89. [97]

    Let’s Verify Step by Step

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s Verify Step by Step. CoRR, abs/2305.20050, 2023

    2023 arXiv

  90. [98]

    The Path to Medical Superintelligence, 2025

    Microsoft AI. The Path to Medical Superintelligence, 2025. https://microsoft.ai/new/ the-path-to-medical-superintelligence/

    2025

  91. [99]

    Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards

    Jaehoon Yun, Jiwoong Sohn, Jungwoo Park, Hyunjae Kim, Xiangru Tang, Yanjun Shao, Yonghoe Koo, Minhyeok Ko, Qingyu Chen, Mark Gerstein, Michael Moor, and Jaewoo Kang. Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards. CoRR, abs/2506.11474, 2025

    2025

  92. [100]

    Capa- bilities of GPT-4 on Medical Challenge Problems

    Harsha Nori, Nicholas King, Scott Mayer Mckinney, Dean Carignan, and Eric Horvitz. Capa- bilities of GPT-4 on Medical Challenge Problems. CoRR, abs/2303.13375, 2023

    2023 arXiv

  93. [101]

    Sharing Google’s Med-PaLM 2 Medical Large Lan- guage Model, or LLM | Google Cloud Blog, 2024

    Google. Sharing Google’s Med-PaLM 2 Medical Large Lan- guage Model, or LLM | Google Cloud Blog, 2024. https: //cloud.google.com/blog/topics/healthcare-life-sciences/ sharing-google-med-palm-2-medical-large-language-model

    2024

  94. [102]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. CoRR, abs/2201.11903, 2023

    2023 arXiv

  95. [103]

    Zhaolong Wu, Abul Hasan, Jinge Wu, Yunsoo Kim, Jason P. Y . Cheung, Teng Zhang, and Honghan Wu. Chain-of-Thought (CoT) Prompting Strategies for Medical Error Detection and Correction. Association for Computational Linguistics, 2024

    2024

  96. [104]

    AI Achieves Silver-Medal Standard Solving International Mathematical Olympiad Problems - Google DeepMind, 2024

    Google Deepmind. AI Achieves Silver-Medal Standard Solving International Mathematical Olympiad Problems - Google DeepMind, 2024. https://deepmind.google/discover/ blog/ai-solves-imo-problems-at-silver-medal-level/

    2024

  97. [105]

    Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning

    Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning. CoRR, abs/2412.09078, 2024. 26

    2024 arXiv

  98. [106]

    Creative Beam Search: LLM-as-a-Judge For Improving Response Generation

    Giorgio Franceschelli and Mirco Musolesi. Creative Beam Search: LLM-as-a-Judge For Improving Response Generation. CoRR, abs/2405.00099, 2024

    2024 arXiv

  99. [107]

    belongs to the category of,

    Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling. CoRR, abs/2502.06703, 2025. 27 Appendices A Generating Grounded Question-Answering Tasks Using a KG A.1 UM...

    2025 arXiv

  100. [109]

    Be in multiple choice format (4 options)

  101. [110]

    Require clinical reasoning along the relationship

  102. [111]

    Include a brief clinical vignette

  103. [112]

    Not directly mention the relationship in the question stem

  104. [113]

    [Option] B

    Have one clearly correct answer Format: <Question> [Clinical Vignette] </Question> <Options> A. [Option] B. [Option] C. [Option] D. [Option] </Options> <Answer>: [Correct Option Letter] </Answer> A.3 Task Generation Pipeline Details Prompt 2 presents the template used to gener...

  105. [114]

    Detailed and include all the steps leading to the answer

  106. [115]

    You are to use the provided context to explain the relationship between the concepts

  107. [116]

    Prompt 3: Correctness Filtering Prompt Template You are a medical examiner

    Strictly do not mention that you are using a given context to generate the explanation. Prompt 3: Correctness Filtering Prompt Template You are a medical examiner. You are given a medical question along with an explanation and the answer. You have also been given a source context

  108. [117]

    If there is an explanation, also evaluate whether the explanation follows from the source to reach the correct answer

    Judge whether the question and answer are logically correct and medically accurate, and follow the source. If there is an explanation, also evaluate whether the explanation follows from the source to reach the correct answer

  109. [118]

    Yes" or

    Respond with only "Yes" or "No". Format your response exactly like this: “Correct: [Yes/No]” Question: {# insert question and options here} Explanation: {# insert thinking trace here} Answer: {# insert answer option letter here} Source Context: {# insert KG path here} 29 Algor...

  110. [119]

    • Mild developmental delay & subtle dysmorphic features: Suggests a possible underlying genetic syndrome

    Analyze the Patient’s Presentation: We have a 16-year-old female with a few key things going on: 30 • Bilateral ptosis (droopy eyelids): This has been there a long time. • Mild developmental delay & subtle dysmorphic features: Suggests a possible underlying genetic syndrome. •...

  111. [120]

    genetic syndrome

    Consider the Differential Diagnosis based on the key features: The combination of ptosis, short stature (relative), developmental delay, and dysmorphic features screams "genetic syndrome." Let’s look at the options through this lens: • A. Proximal muscle weakness and elevated ...

  112. [121]

    Both webbed neck (B) and increased carrying angle (D) are features of TS

    Choose the MOST Likely Finding: So, Turner Syndrome seems like a very strong candidate diagnosis given the patient’s features (female, ptosis, relative short stature, possible dysmorphism/DD). Both webbed neck (B) and increased carrying angle (D) are features of TS. • A webbed...

  113. [122]

    like a zombie

    Final Conclusion: Based on the patient’s constellation of symptoms pointing strongly towards Turner Syndrome as the most likely underlying diagnosis, an increased carrying angle of the elbows (cubitus valgus) is the most probable associated physical finding among the choices p...

  114. [123]

    like a zombie,

    Understand the Patient and Situation: We have a young adult male (24 years old) expe- riencing an acute psychotic episode (agitation, disorganized speech, paranoia, command hallucinations). This is likely an exacerbation of a known chronic mental illness, probably schizophreni...

  115. [124]

    • Minimizes the risk of EPS (like Parkinsonism, dystonia, akathisia, tardive dyskinesia)

    Identify the Goals of Treatment: The question specifically asks for a medication that: • Treats both positive symptoms (hallucinations, paranoia, disorganized thought) AND negative symptoms (which are common in chronic psychosis, like apathy, flat affect, alogia - though not e...

  116. [125]

    Haloperidol decanoate injection: This is a long-acting injectable (LAI) form of a first-generation antipsychotic (FGA)

    Evaluate the Options Based on Goals: • A. Haloperidol decanoate injection: This is a long-acting injectable (LAI) form of a first-generation antipsychotic (FGA). – Pros: LAI helps with adherence, which is good since he stopped meds before. Effective for positive symptoms. – Co...

  117. [126]

    perioral

    Final Decision: Comparing the options, Aripiprazole (D) best fits the requirements laid out in the question: efficacy for positive/negative symptoms, minimized EPS risk, and minimized weight gain risk, making it the most appropriate long-term strategy for this patient given hi...

    1974

  118. [127]

    We have a 14-year- old male with recurrent painless rectal bleeding (often bright red blood per rectum, or ‘painless hematochezia’) and iron deficiency anemia

    Identify the Syndrome: First, look at the patient’s presentation. We have a 14-year- old male with recurrent painless rectal bleeding (often bright red blood per rectum, or ‘painless hematochezia’) and iron deficiency anemia. The colonoscopy confirms numerous hamartomatous pol...

  119. [128]

    The classic features are: • A: Epistaxis (nosebleeds) – often severe and recurrent

    Understand HHT: HHT is an autosomal dominant disorder affecting the vascular system. The classic features are: • A: Epistaxis (nosebleeds) – often severe and recurrent. • B: Telangiectasias (on skin, lips, oral/nasal mucosa). • C: Gastrointestinal bleeding (from telangiectasia...

  120. [129]

    HHT is known to cause various vascular malformations through- out the body, including the brain

    Connect HHT to Neurovascular Abnormalities: The question specifically asks about neurovascular abnormalities associated with this constellation of findings (which we’ve identified as likely HHT). HHT is known to cause various vascular malformations through- out the body, inclu...

  121. [130]

    puff of smoke

    Evaluate the Options: • A. Moyamoya disease: This is a progressive narrowing (stenosis) and occlusion of the internal carotid arteries and their main branches, leading to the formation of a network of tiny collateral vessels (the "puff of smoke" or moyamoya appearance). While ...

  122. [131]

    MOST likely to develop as the condition progresses

    Conclusion: Based on the patient’s presentation (colonic hamartomatous polyps, mucocu- taneous telangiectasias) which strongly points towards HHT, the most likely associated neurovascular abnormality to consider is an intracranial arteriovenous malformation (A VM). Therefore, ...

  123. [132]

    We have a 55-year-old male with a pretty rapid decline over just 6 months

    Identify the Diagnosis: First off, we need to figure out what’s going on with this patient. We have a 55-year-old male with a pretty rapid decline over just 6 months. The key features are the neurological problems like trouble speaking (dysarthria), jerky movements (myoclonus)...

  124. [133]

    Mutations in PRNP cause a group of diseases called prion diseases, which are fatal neurodegenerative disorders

    What does the PRNP gene mutation mean? This gene codes for the prion protein (PrP). Mutations in PRNP cause a group of diseases called prion diseases, which are fatal neurodegenerative disorders. The most common form you might know is Creutzfeldt- Jakob disease (CJD). However,...

  125. [134]

    • Rapid Neurological Decline: This is a hallmark of prion diseases

    Connecting Symptoms to Prion Disease: The patient’s symptoms fit well within the spectrum of prion diseases. • Rapid Neurological Decline: This is a hallmark of prion diseases. • Myoclonus: This is extremely common in prion diseases, often being a key feature. • Dysarthria and...

  126. [135]

    We already know he has myoclonus and dysarthria

    Evaluating the Options: The question asks what additional neurological manifestation is most likely to develop as his condition progresses. We already know he has myoclonus and dysarthria. Let’s look at the choices in the context of prion disease progression: 40 • A. Lower mot...

  127. [136]

    controlled hormone levels,

    Conclusion: Based on the underlying diagnosis of a prion disease ( PRNP mutation) and the known spectrum of neurological signs that can develop, involuntary writhing movements (choreoathetosis) are a well-recognized additional feature. Therefore, this is the most likely sympto...

  128. [137]

    This is a rare cancer of the adrenal cortex

    Understand the Patient’s Situation: We have a 45-year-old female diagnosed with advanced ACC. This is a rare cancer of the adrenal cortex. Key things to note are her initial presentation (virilization, new-onset diabetes, high cortisol, androgens, estrogens) and the treatment ...

  129. [138]

    The direct effect of elevated cortisol on hepatic lipid metabolism: This is a very plausible mechanism

    Analyze the Options: • A. The direct effect of elevated cortisol on hepatic lipid metabolism: This is a very plausible mechanism. Chronic elevation of cortisol (a glucocorticoid) is well- known to mess with liver metabolism. Cortisol increases the production of Very Low-Densit...

  130. [139]

    external

    Synthesize and Conclude: We’ve ruled out dietary intake (D) because of the low- fat diet. Ectopic ApoB production (B) is too rare and specific. Elevated cortisol (A) could cause it, but her hormones are stated to be controlled. This leaves the medication side effect (C) as the...

  131. [2025]

    Agentic Coding Assistant Integrating with GitHub/GitLab and IDEs

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.