Pith. sign in

REVIEW 4 major objections 5 minor 79 references

HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discovery

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HypoChainer argues that a three-stage loop joining human experts, a retrieval-augmented LLM, and a knowledge graph improves hypothesis construction and validation, reporting that 4 of 6 users completed a drug-repurposing task versus 2 of…

desk verdict A well-designed LLM+KG visual analytics system whose user-study benchmark is too aligned with its own KG retrieval to support the headline effectiveness claim. read the letter →

arxiv 2507.17209 v1 pith:Y7HVU7NX submitted 2025-07-23 cs.HC cs.LG

Haoran Jiang , Shaohan Shi , Yunjie Yao , Chang Jiang , Quan Li This is my paper
classification cs.HCcs.LG
keywords LargeLanguageModelVisualAnalyticsIterativeHuman-AICollaborationKnowledgeGraphHypothesisConstructionDrugRepurposingSyntheticLethalityRetrieval-AugmentedGeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes HypoChainer, a collaborative system in which human experts, a retrieval-augmented large language model, and a knowledge graph work through three stages: exploring graph neural network (GNN) predictions, constructing a chain of linked hypotheses, and prioritizing predictions for validation. The central claim is that this three-way loop produces more coherent and more actionable scientific hypotheses than an LLM working alone. The paper supports the claim with case studies in two domains, expert interviews, and a user study in which 4 of 6 participants using HypoChainer completed a drug-repurposing task while 2 of 6 succeeded with an LLM-only baseline. If the claim holds, researchers can turn a flood of model predictions into a short, auditable list of experimental candidates with mechanistically explicit reasoning.

What carries the argument

The load-bearing object is the hypothesis chain: a structured reasoning path made of several interrelated hypotheses connected by logical links, for example drug-to-pathway-to-process-to-disease. The chain is both the human artifact and the retrieval query, letting experts state what they believe and then asking the knowledge-graph-grounded RAG engine to find predictions and triplets that align with it. The second mechanism is KG-grounded retrieval: the RAG engine draws on local knowledge-graph entities unless the expert deliberately invokes online retrieval, so LLM suggestions and hypothesis-aligned entities remain traceable. The Retrieval View's UpSet-style plot then shows how many triplets satisfy individual or combined hypotheses, exposing which links in the chain are weak and need refinement.

What would settle it

Re-score the same user-study task using reference mechanisms that were validated after the knowledge graph was assembled and are absent from DrugMechDB; if the 4-versus-2 completion advantage shrinks or vanishes, the reported gain is benchmark alignment rather than improved hypothesis construction.

Watch

Extended reading notes

Core claim

HypoChainer's central claim is that a chain of linked hypotheses, built jointly by an expert and an LLM but checked against knowledge-graph relations, makes GNN-based discovery both more coherent and more selective. The paper argues that raw predictions are too numerous for manual review, LLM-only filtering is unreliable because it can hallucinate, and knowledge graphs alone are too sparse to carry biological meaning; the system combines the three so that each compensates for the others. In the reported user study, 4 of 6 participants using HypoChainer completed the Hemophilia B drug-repurposing task within the time limit, versus 2 of 6 with an LLM-only baseline, and HypoChainer received significantly higher ratings for identifying research directions, hypothesis-chain coherence, rationality, flexibility, informativeness, and understandability (p < 0.05). The paper presents this as evidence that the collaborative loop, not the LLM alone, drives the improvement.

Load-bearing premise

The benchmark answers used to score success were chosen partly because they match a database built from the same kind of structured relationships the system is designed to retrieve, so the comparison could favor the system without proving better scientific reasoning.

Editorial extensions

If this is right

  • A researcher can move from thousands of GNN predictions to a shortlist of mechanistically coherent experimental candidates in one session, with the reasoning trail preserved as a hypothesis chain.
  • The workflow transfers across domains by swapping the knowledge graph and the predictive model: the paper demonstrates it on drug repurposing and synthetic-lethality cancer research.
  • LLM hallucinations are reduced because retrieval stays grounded in local knowledge-graph entities unless the expert deliberately enables online retrieval, and text-to-KG integration lets experts patch missing relationships.
  • Hypothesis-chain coherence becomes a measurable evaluation target for AI-assisted discovery systems, beyond raw prediction accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the completion-rate advantage is real, the biggest payoff may be auditability rather than raw discovery: hypothesis chains leave a visible reasoning trail that a lab head or reviewer can check before committing to experiments.
  • A stronger test would score the novelty of proposed candidates, not just recovery of known mechanisms, separating 'finds what the KG already encodes' from 'finds what the KG is missing.'
  • The same three-stage loop should transfer to any field with a predictive graph model and a domain ontology, such as materials discovery or protein function prediction, provided the reference answers are independent of the knowledge graph.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents HypoChainer, a visual analytics framework that couples LLM-driven reasoning, knowledge-graph retrieval, and human expert judgment for hypothesis construction and validation in GNN-based biomedical discovery. The authors derive six design requirements from a formative interview study with four domain experts, describe a three-stage pipeline (contextual exploration, hypothesis construction, validation selection) with seven coordinated views, and evaluate the system through an author-guided drug-repurposing case study and a 12-participant between-subjects user study comparing HypoChainer to an LLM-only baseline. The paper claims that HypoChainer enhances hypothesis construction and scientific discovery by making model predictions interpretable, supporting iterative construction of 'hypothesis chains', and prioritizing KG-supported validation candidates.

Significance. If the effectiveness claim could be supported, HypoChainer would be a useful contribution to the visual analytics and human-AI collaboration literature: the three-stage workflow is clearly motivated, the formative study provides concrete requirements, and the case study illustrates an elaborate and plausible discovery process. The authors also honestly discuss limitations such as KG quality dependence, RAG cost/accuracy trade-offs, and residual hallucinations. The main weakness is that the only quantitative evaluation, the user study in Section 5.2, does not isolate hypothesis-construction quality from KG-grounded retrieval of pre-selected reference paths, and the remaining evidence is qualitative and author-guided. The system's engineering and design-study value are credible, but the central effectiveness claim is not yet established.

major comments (4)
  1. [§5.2, User Study] The task success criterion is not a valid measure of the paper's central claim. Participants succeed by identifying at least 3 of 5 reference predictions, and those references were 'system-generated interpretative paths' selected for partial alignment with DrugMechDB-curated MOAs; HypoChainer's core mechanism is RAG retrieval over the same KG, and the baseline intentionally omits the Hypothesis View and KG-grounded workflow. The 4/6 vs 2/6 completion advantage can therefore reflect benchmark alignment rather than improved hypothesis construction or validation. I ask the authors to add a KG-independent outcome measure (e.g., expert scoring of the content, novelty, and mechanistic plausibility of submitted hypotheses) or a control condition whose reference answers are not derivable from the system's KG, and to demonstrate that the reference paths are not trivially returned by the RAG module alone.
  2. [§5.2, User Study] The quantitative results are statistically fragile. With six participants per arm, the completion rates 4/6 and 2/6 are not significantly different by Fisher's exact test, yet the paper presents this as a clear advantage. The questionnaire analyses test nine items without any multiple-comparison correction and report effect sizes for only some items; the familywise error rate is therefore inflated. The authors should report exact tests with effect sizes and corrected p-values, and should explicitly frame the study as a pilot, not as confirmatory evidence, if the sample size cannot be increased.
  3. [§5.2, User Study] The success threshold (≤300 retrieved predictions and ≥3 of 5 reference predictions) is not justified or pre-specified. Because the threshold directly determines the primary outcome, the authors should either justify it by reference to the study design or perform a sensitivity analysis across thresholds; otherwise the completion rate can be changed by small changes in the criterion.
  4. [§5.1, Case Study] The case study cannot carry the effectiveness claim independently: one author guided the session, E3 operated the system, and E4 discussed insights in real time, and all three were involved in the design process. This is an appropriate design-study illustration, but the conclusion should be worded as illustrating a plausible workflow rather than demonstrating effectiveness. An independent adoption study, or at least a session without author guidance, would be needed to support the stronger claim.
minor comments (5)
  1. [Manuscript metadata] The IEEE metadata (received 1 April 2024; accepted 15 July 2024; date of current version 18 July 2024) conflicts with the arXiv submission date of 23 July 2025; please correct the dates or venue information.
  2. [Byline and footnote] The byline contains a spacing/typo issue: 'Yunjie Y ao' should be 'Yunjie Yao', and the footnote lists 'Y. Jie' instead of 'Y. Yao' for the same author.
  3. [Appendix reference] The paper references Appendix Fig. 8 for the baseline system, but the appendix is not included in the manuscript; please ensure supplementary material is available to reviewers and readers.
  4. [Fig. 6] The questionnaire figure marks significance with asterisks but does not identify the statistical test used, and some Baseline means and SDs appear duplicated in the text (e.g., M = 4.17 appears for both 'Ease of identifying research focuses' and 'Logical coherence of hypothesis chains'), which makes the reporting hard to follow.
  5. [§2.2, Hypothesis View] The term 'hypothesis-aligned entities' is defined, but the RAG-based alignment judgment underlying it is treated as a black box; a brief description of the prompt or scoring procedure would improve reproducibility.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the empirical evaluation has construct-validity caveats but no derivation reduces to its inputs.

full rationale

HypoChainer is an empirical systems paper with no formal derivation chain; its effectiveness claims rest on a case study, expert interviews, and a small user study. The user-study reference paths are system-generated predictions selected for partial alignment with DrugMechDB-curated mechanisms of action, and the target drugs were excluded from training data; this is an external benchmark rather than a fitted target, and identifying the references still requires real user effort, so success is not forced by construction. The only notable self-citation is SLInterpreter [27], used to support the Voronoi treemap design choice; that design element is not load-bearing for the central effectiveness claim. The evaluation has genuine limitations (n=12, self-report questionnaire items, no multiple-comparison correction, and benchmark alignment with KG-grounded retrieval), but these are construct-validity and statistical-power concerns, not circularity under the definitions used here.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

This paper makes no fitted mathematical derivation, so the free-parameter count is small. The load-bearing premises are external: the quality of the biomedical KGs and of the GNN prediction models, and the reliability of the LLM's alignment judgments, all of which the paper itself flags in its Limitations section. The evaluation additionally depends on hand-chosen thresholds and benchmark construction choices. The only invented construct is the hypothesis chain, a design abstraction whose value rests on the paper's own studies.

free parameters (3)
  • RAG entity retrieval limit per query = Default value not reported
    Section 6 (Lessons Learned) states that increasing the retrieval limit markedly improved response accuracy at higher token cost. The operating point is a tuned choice that affects how many entities the system recommends and retrieves.
  • Task success thresholds = At most 300 retrieved predictions; at least 3 of 5 reference predictions
    Section 5.2 defines task success by these two thresholds without sensitivity analysis. The choice directly shapes the headline completion rates (66.7% vs 33.3%).
  • UMAP embedding configuration = Not reported
    The Embedding View's clusters depend on UMAP applied to entity features (gene descriptions, sequences, drug indications). The exact feature sets and hyperparameters are deferred to an appendix not present in the reviewed text, and the lasso-selection workflow relies on the clusters being semantically meaningful.
assumptions (4)
  • domain assumption The biomedical knowledge graphs (RTX-KG2c-derived graph, SynLethDB, ProteinKG25) and the GNN prediction models (KGML-xDTD, KR4SL) are accurate enough that KG-grounded retrieval constitutes reliable evidence for hypothesis validation.
    This is the paper's core grounding premise. The Limitations section explicitly concedes that 'the system's performance is largely dependent on the quality of the underlying KGs and the effectiveness of the associated predictive models.'
  • domain assumption KR4SL's reported 59% precision and KGML-xDTD's reported prediction gains are accepted as correct from the cited sources and appendix tables.
    Section 4.1 relies on upstream performance numbers without re-verification. They are external results taken from the literature (KR4SL, KGML-xDTD).
  • ad hoc to paper The RAG-based alignment judgment orders hypothesis-aligned entities and retrieved triplets correctly enough to guide expert reasoning.
    Section 4.2 describes entity ordering 'by their degree of alignment, which is judged by the RAG through a systematically designed prompt.' The prompt is not shown, and the Limitations section concedes that RAG hallucinations and inaccurate text-to-KG integration persist.
  • domain assumption The co-discovery protocol and the 12-participant sample yield measurements of workflow effectiveness that are not dominated by experimenter or sampling bias.
    Section 5.1 states that one author guided the case-study session, and Section 5.2 evaluates twelve graduate students. Both choices assume minimal bias, which is standard but untested in the paper.
invented entities (1)
  • Hypothesis chain
    purpose: A structured reasoning path linking multiple hypotheses, encoded like a triplet with textual descriptions, used to drive retrieval, alignment scoring, and validation selection.
    This is the paper's central design construct. There is no external benchmark or formal account establishing that hypothesis-chain structure improves reasoning or retrieval; its value is supported only by the paper's own case study and user study, and the paper does not compare chain-based retrieval against flat hypothesis retrieval.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discovery." pith.science (2026). https://pith.science/paper/Y7HVU7NX

@misc{pith2026250717209,
  author       = {Pith},
  title        = {Pith review of: HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7HVU7NX}},
  note         = {Machine review of arXiv:2507.17209}
}
read the original abstract

Modern scientific discovery faces growing challenges in integrating vast and heterogeneous knowledge critical to breakthroughs in biomedicine and drug development. Traditional hypothesis-driven research, though effective, is constrained by human cognitive limits, the complexity of biological systems, and the high cost of trial-and-error experimentation. Deep learning models, especially graph neural networks (GNNs), have accelerated prediction generation, but the sheer volume of outputs makes manual selection for validation unscalable. Large language models (LLMs) offer promise in filtering and hypothesis generation, yet suffer from hallucinations and lack grounding in structured knowledge, limiting their reliability. To address these issues, we propose HypoChainer, a collaborative visualization framework that integrates human expertise, LLM-driven reasoning, and knowledge graphs (KGs) to enhance hypothesis generation and validation. HypoChainer operates in three stages: First, exploration and contextualization -- experts use retrieval-augmented LLMs (RAGs) and dimensionality reduction to navigate large-scale GNN predictions, assisted by interactive explanations. Second, hypothesis chain formation -- experts iteratively examine KG relationships around predictions and semantically linked entities, refining hypotheses with LLM and KG suggestions. Third, validation prioritization -- refined hypotheses are filtered based on KG-supported evidence to identify high-priority candidates for experimentation, with visual analytics further strengthening weak links in reasoning. We demonstrate HypoChainer's effectiveness through case studies in two domains and expert interviews, highlighting its potential to support interpretable, scalable, and knowledge-grounded scientific discovery.

Figures

Figures reproduced from arXiv: 2507.17209 by the authors.

Figure 1
Figure 1. In the presented case study, the biologist’s analytical workflow unfolds as follows: [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of Traditional Practice A and HypoChainer Pipeline B : Both follow the I Contextual Exploration, II Hypothesis Con￾struction, and III Validation Selection workflow. Recent efforts [1, 69, 76] have extended to leveraging large language models (LLMs) to analyze and interpret large-scale predictive outcomes. LLMs offer unique advantages in integrating multimodal and hetero￾geneous data, enabling preliminary … view at source ↗
Figure 3
Figure 3. UpSet Plot: Visualization of entity triplets alignment organized by [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The pipeline comprises three main components: [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: 1 E3 no￾ticed a link to Hunt￾ington’s disease from Trinucleotide Repeat Expansion, a 1-hop entity of neurodegen￾erative disease. Hypothesis Challenge and Iterative Re￾finement. During hypothesis validation, E3 noted that the revised chain obscured the ma￾jority of the …
Figure 6
Figure 6. Figure 6: The questionnaire results of the two systems in terms of system [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

79 extracted references · 28 canonical work pages

  1. [1]

    Ahmed, M

    A. Ahmed, M. Saleem, M. Alzeen, et al. Leveraging large language models to enhance machine learning interpretability and predictive performance: A case study on emergency department returns for mental health patients,

    work page

  2. [2]

    Amer-Yahia, A

    S. Amer-Yahia, A. Bonifati, L. Chen, G. Li, X. Shim, Kyuseok, et al. From large language models to databases and back: A discussion on research and education. SIGMOD Rec., 52(3):49–56, 8 pages, Nov. 2023. doi: 10. 1145/3631504.3631518 3

    arXiv 2023

  3. [3]

    Asprino, C

    L. Asprino, C. Colonna, M. Mongiovì, M. Porena, and V . Presutti. Pattern- based visualization of knowledge graphs, 2021. doi: 10.48550/arXiv.2106 .12857 3

    doi:10.48550/arxiv.2106 2021

  4. [4]

    J. Baek, S. K. Jauhar, S. Cucerzan, and S. J. Hwang. ResearchAgent: Iter- ative research idea generation over scientific literature with large language models, 2025. doi: 10.48550/arXiv.2404.07738 3

    arXiv doi:10.48550/arxiv.2404.07738 2025

  5. [5]

    Balzer and O

    M. Balzer and O. Deussen. V oronoi treemaps. In IEEE Symposium on Information Visualization, 2005. INFOVIS 2005., pp. 49–56, 2005. doi: 10.1109/INFVIS.2005.1532128 3, 6

    arXiv 2005

  6. [6]

    M. R. Birtwistle. Analytical reduction of combinatorial complexity arising from multiple protein modification sites. Journal of The Royal Society Interface, 12(103):20141215, 2015. doi: 10.1098/rsif.2014.1215 2

    arXiv 2015

  7. [7]

    Boger, S

    T. Boger, S. B. Most, and S. L. Franconeri. Jurassic mark: Inattentional blindness for a datasaurus reveals that visualizations are explored, not seen. In 2021 IEEE Visualization Conference (VIS), pp. 71–75, 2021. doi: 10. 1109/VIS49827.2021.9623273 2

    arXiv 2021

  8. [8]

    J. Brooke. SUS: a retrospective. Journal of Usability Studies, 8:29–40, 01

    work page

Show all 79 references
  1. [9]

    Chang, X

    Y . Chang, X. Wang, J. Wang, Y . Wu, L. Yang, K. Zhu, et al. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1–45, 2024. doi: 10.1145/3641289 2

    2024 doi

  2. [10]

    S. Chen, S. Li, S. Chen, and X. Yuan. R-Map: A map metaphor for visual- izing information reposting process in social media. IEEE Transactions on Visualization and Computer Graphics, 26(1):1204–1214, 2020. doi: 10 .1109/TVCG.2019.2934263 3

    2020

  3. [11]

    Choudhary and C

    N. Choudhary and C. K. Reddy. Complex logical reasoning over knowl- edge graphs using large language models, 2024. doi: 10.48550/arXiv. 2305.01157 3

    2024 doi arXiv

  4. [12]

    Clarke and V

    V . Clarke and V . Braun. Thematic analysis. The journal of positive psychology, 12:297–298, 2017. doi: 10.1080/17439760.2016.1262613 3

    2017

  5. [13]

    T. Dang, P. Murray, and A. Forbes. BioLinker: Bottom-up exploration of protein interaction networks. In 2017 IEEE Pacific Visualization Sympo- sium (PacificVis), pp. 265–269. IEEE, 2017. doi: 10.1109/PACIFICVIS. 2017.8031603 2, 3

    2017

  6. [14]

    S. Deng, S. Wang, H. Rangwala, L. Wang, and Y . Ning. Cola-GNN: Cross-location Attention based Graph Neural Networks for Long-term ILI Prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, 10 pages, p. 245–254,

  7. [15]

    D. Edge, H. Trinh, N. Cheng, et al. From Local to Global: A graph RAG approach to query-focused summarization, 2025. doi: 10.48550/arxiv. 2404.16130 5

    2025 doi arXiv

  8. [16]

    Ghafarollahi and M

    A. Ghafarollahi and M. J. Buehler. SciAgents: Automating scientific discovery through bioinspired multi-agent intelligent graph reasoning. Advanced Materials, p. 2413523. doi: 10.1002/adma.202413523 2, 3

    doi

  9. [17]

    Gottweis, W.-H

    J. Gottweis, W.-H. Weng, A. Daryin, et al. Towards an AI co-scientist,

  10. [18]

    Gratzl, A

    S. Gratzl, A. Lex, N. Gehlenborg, H. Pfister, and M. Streit. LineUp: Visual analysis of multi-attribute rankings. IEEE Transactions on Visualization and Computer Graphics, 19(12):2277–2286, 2013. doi: 10.1109/TVCG. 2013.173 5

    2013 doi

  11. [19]

    Z. Guo, L. Xia, Y . Yu, T. Ao, and C. Huang. LightRAG: Simple and fast retrieval-augmented generation, 2024. doi: 10.48550/arXiv.2410.05779 5

    2024 doi arXiv

  12. [20]

    doi: 10.48550/arXiv.2502.18864 3

    doi arXiv

  13. [21]

    L. He, S. Zheng, T. Yang, and F. Zhang. KLMo: Knowledge graph enhanced pretrained language model with fine-grained relationships. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 4536–4542. Association for Computational Linguistics, Nov. 2021. doi:...

    2021 doi

  14. [22]

    Hecklau, S

    K. Hecklau, S. Mueller, S. P. Koch, et al. The effects of selective inhibition of histone deacetylase 1 and 3 in Huntington’s disease mice. Frontiers in molecular neuroscience, 14:616886, 2021. doi: 10.3389/fnmol.2021. 616886 8

    2021 doi

  15. [23]

    Hasin, M

    Y . Hasin, M. Seldin, and A. Lusis. Multi-omics approaches to disease. Genome biology, 18:1–15, 2017. doi: 10.1186/s13059-017-1215-1 3

    2017 doi

  16. [24]

    X. Hu, H. Fu, J. Wang, Y . Wang, Z. Li, R. Xu, Y . Lu, Y . Jin, L. Pan, and Z. Lan. Nova: An iterative planning and search approach to enhance novelty and diversity of LLM generated ideas, 2024. doi: 10.48550/arXiv. 2410.14255 3

    2024 doi arXiv

  17. [25]

    Huang, C

    D. Huang, C. Yan, Q. Li, and X. Peng. From large language models to large multimodal models: A literature review. Applied Sciences, 14(12),

  18. [26]

    Henry and B

    S. Henry and B. T. McInnes. Literature based discovery: models, methods, and trends. Journal of biomedical informatics, 74:20–32, 2017. doi: 10. 1016/j.jbi.2017.08.011 2

    2017

  19. [28]

    Jiang, F

    Z. Jiang, F. F. Xu, J. Araki, and G. Neubig. How can we know what lan- guage models know? Transactions of the Association for Computational Linguistics, 8:423–438, 2020. doi: 10.1162/tacl_a_00324 3

    2020 doi

  20. [29]

    Z. Jin, Y . Wang, Q. Wang, Y . Ming, T. Ma, and H. Qu. GNNLens: A visual analytics approach for prediction error diagnosis of graph neural networks. IEEE Transactions on Visualization and Computer Graphics, 29(6):3024–3038, 2023. doi: 10.1109/TVCG.2022.3148107 3

    2023

  21. [30]

    Izacard and E

    G. Izacard and E. Grave. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 874–880. Association for Computatio...

    2021 doi

  22. [31]

    Kehrer, F

    J. Kehrer, F. Ladstädter, P. Muigg, H. Doleisch, A. Steiner, and H. Hauser. Hypothesis generation in climate research with interactive visual data exploration. IEEE Transactions on Visualization and Computer Graphics, 14(6):1579–1586, 2008. doi: 10.1109/TVCG.2008.139 2

    2008 doi

  23. [32]

    Krenn, L

    M. Krenn, L. Buffoni, B. Coutinho, et al. Forecasting the future of artificial intelligence with machine learning-based link prediction in an exponentially growing knowledge network. Nature Machine Intelligence, 5(11):1326–1335, 2023. doi: 10.1038/s42256-023-00735-0 2

    2023 doi

  24. [33]

    Kumar, T

    S. Kumar, T. Ghosal, V . Goyal, and A. Ekbal. Can large language models unlock novel scientific research ideas?, 2024. doi: 10.48550/arXiv.2409. 06185 2

    2024 doi

  25. [34]

    Kaufmann, P

    T. Kaufmann, P. Weng, V . Bengs, and E. Hüllermeier. A survey of re- inforcement learning from human feedback, 2024. doi: 10.48550/arXiv. 2312.14925 3

    2024 doi

  26. [35]

    A. Lex, N. Gehlenborg, H. Strobelt, R. Vuillemot, and H. Pfister. Upset: Visualization of intersecting sets. IEEE Transactions on Visualization and Computer Graphics, 20(12):1983–1992, 2014. doi: 10.1109/TVCG.2014. 2346248 6

    1983 doi

  27. [36]

    H. Li, G. Appleby, and A. Suh. Linkq: An LLM-assisted visual interface for knowledge graph question-answering. In 2024 IEEE Visualization and Visual Analytics (VIS), pp. 116–120, 2024. doi: 10.1109/VIS55277.2024. 00031 7

    2024

  28. [37]

    Liang, M

    L. Liang, M. Sun, Z. Gui, et al. KAG: Boosting LLMs in professional domains via knowledge augmented generation, 2024. doi: 10.48550/arXiv .2409.13731 3

    2024 doi arXiv

  29. [38]

    V . Kumar. Understanding the role of histone deacetylase and their in- hibitors in neurodegenerative disorders: Current targets and future per- spective. Current Neuropharmacology, 20(1):158–178, 2022. doi: 10. 2174/1570159X19666210609160017 8

    2022

  30. [39]

    X. Lin, Z. Quan, Z.-J. Wang, T. Ma, and X. Zeng. KGNN: Knowledge graph neural network for drug-drug interaction prediction. In C. Bessiere, ed., Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 2739–2745. International Jo...

    2020 doi

  31. [40]

    C. Lu, C. Lu, R. T. Lange, J. Foerster, J. Clune, and D. Ha. The AI scientist: Towards fully automated open-ended scientific discovery, 2024. doi: 10.48550/arXiv.2408.06292 3

    2024 doi arXiv

  32. [41]

    C. Ma, Z. Zhou, H. Liu, and D. Koslicki. KGML-xDTD: a knowledge graph–based machine learning framework for drug treatment prediction and mechanism description. GigaScience, 12:giad057, 08 2023. doi: 10. 1093/gigascience/giad057 2, 5

    2023

  33. [42]

    K. H. Lim, L. M. Ward, and I. Benbasat. An empirical study of computer system learning: Comparison of co-discovery and self-discovery methods. Information Systems Research, 8(3):254–272, 1997. doi: 10.1287/isre.8.3. 254 8

    1997 doi

  34. [43]

    McInnes and J

    L. McInnes and J. Healy. UMAP: Uniform manifold approximation and projection for dimension reduction. ArXiv, abs/1802.03426, 2018. doi: 10. 21105/joss.00861 5

    2018 arXiv

  35. [44]

    Naveed, A

    H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian. A comprehensive overview of large language models, 2024. doi: 10.48550/arXiv.2307.06435 2

    2024 doi arXiv

  36. [45]

    Nigam, M

    H. Nigam, M. Patwardhan, L. Vig, and G. Shroff. Acceleron: A tool to accelerate research ideation, 2024. doi: 10.48550/arXiv.2403.04382 3

    2024 doi arXiv

  37. [46]

    Ma, T.-H

    P. Ma, T.-H. Wang, M. Guo, Z. Sun, J. B. Tenenbaum, D. Rus, C. Gan, and W. Matusik. LLM and simulation as bilevel optimizers: A new paradigm to advance physical scientific discovery, 2024. doi: 10.48550/arXiv.2405. 09783 3

    2024 doi

  38. [47]

    S. Pan, L. Luo, Y . Wang, C. Chen, J. Wang, and X. Wu. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 2024. doi: 10.48550/arXiv.2306. 08302 3

    2024 doi

  39. [48]

    C. Peng, F. Xia, M. Naseriparsa, and F. Osborne. Knowledge graphs: Opportunities and challenges.Artificial Intelligence Review, 56(11):13071– 13102, November 2023. doi: 10.1007/s10462-023-10465-9 2

    2023 doi

  40. [49]

    Pérez, M

    J. Pérez, M. Arenas, and C. Gutierrez. Semantics and complexity of sparql. ACM Trans. Database Syst., 34(3), article no. 16, 45 pages, Sept. 2009. doi: 10.1145/1567274.1567278 3

    2009

  41. [50]

    Paley, R

    S. Paley, R. Billington, J. Herson, M. Krummenacker, and P. D. Karp. Path- way tools visualization of organism-scale metabolic networks.Metabolites, 11(2):64, 2021. doi: 10.3390/metabo11020064 3

    2021 doi

  42. [51]

    K. Popper. The logic of scientific discovery . Routledge, 2005. doi: 10. 4324/9780203994627 2

    2005

  43. [52]

    K. Pu, K. J. K. Feng, T. Grossman, T. Hope, B. D. Mishra, M. Latzke, J. Bragg, J. C. Chang, and P. Siangliulue. IdeaSynth: Iterative research idea development through evolving and composing idea facets with literature- grounded feedback, 2024. doi: 10.48550/arXiv.2410.04025 2

    2024 doi arXiv

  44. [53]

    B. Qi, K. Zhang, H. Li, K. Tian, S. Zeng, Z.-R. Chen, and B. Zhou. Large language models are zero shot hypothesis proposers, 2023. doi: 10. 48550/arXiv.2311.05965 2

    2023 doi arXiv

  45. [54]

    Perkovi´c, A

    G. Perkovi´c, A. Drobnjak, and I. Botiˇcki. Hallucinations in LLMs: Un- derstanding and addressing challenges. In 2024 47th MIPRO ICT and Electronics Convention (MIPRO), pp. 2084–2088, 2024. doi: 10.1109/ MIPRO60963.2024.10569238 2, 3

    2024

  46. [55]

    Schimanski, J

    T. Schimanski, J. Ni, M. Kraus, E. Ash, and M. Leippold. Towards faithful and robust LLM specialists for evidence-based question-answering. In Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pp. 1913–1931. Asso...

    1913 doi

  47. [58]

    Qiang and Z

    W. Qiang and Z. Zhongli. Reinforcement learning model, algorithms and its application. In 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC), pp. 1143–1146, 2011. doi: 10 .1109/MEC.2011.6025669 3

    2011

  48. [59]

    H. Su, R. Chen, S. Tang, X. Zheng, J. Li, Z. Yin, W. Ouyang, and N. Dong. Two heads are better than one: A multi-agent system has the potential to improve scientific idea generation. IEEE Transactions on Cognitive and Developmental Systems, 2024. doi: 10.1109/TCDS.2025.3530945 3

    2024

  49. [60]

    K. M. Tolle, D. S. W. Tansley, and A. J. G. Hey. The Fourth Paradigm: Data-intensive Scientific Discovery. 2009. doi: 10.1109/JPROC.2011. 2155130 2

    2009 doi

  50. [61]

    D. R. Unni, S. A. T. Moxon, M. Bada, et al. Biolink model: A universal schema for knowledge graphs in clinical, biomedical, and translational science. Clinical and Translational Science, 15(8):1848–1855, 2022. doi: 10.1111/cts.13302 5

    2022 doi

  51. [62]

    C. Si, D. Yang, and T. Hashimoto. Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers, 2024. doi: 10.48550/arXiv.2409.04109 2

    2024 doi arXiv

  52. [63]

    Vrande ˇci´c and M

    D. Vrande ˇci´c and M. Krötzsch. Wikidata: a free collaborative knowl- edgebase. Communications of the ACM , 57(10):78–85, 2014. doi: 10. 1145/2629489 3

    2014

  53. [64]

    F. Wang, X. Zhou, W. Hu, Z. Luo, W. Luo, and X. Bai. LLM assists hypothesis generation and testing for deliberative questions. 13 pages, p. 424–436. Springer-Verlag, 2024. doi: 10.1007/978-981-97-9434-8_33 2

    2024 doi

  54. [65]

    Q. Wang, K. Huang, P. Chandak, M. Zitnik, and N. Gehlenborg. Extending the nested model for user-centric XAI: A design study on GNN-based drug repurposing. IEEE Transactions on Visualization and Computer Graphics, 29(1):1266–1276, 2023. doi: 10.1109/TVCG.2022.3209435 3

    2023

  55. [66]

    S. K. V ohra, P. Harth, Y . Isoe, A. Bahl, H. Fotowat, F. Engert, H.-C. Hege, and D. Baum. A visual interface for exploring hypotheses about neural circuits. IEEE Transactions on Visualization and Computer Graphics , 30(7):3945–3958, 2024. doi: 10.1109/TVCG.2023.3243668 2

    2024

  56. [67]

    X. Wang, T. Gao, Z. Zhu, Z. Zhang, Z. Liu, J. Li, and J. Tang. KEPLER: A unified model for knowledge embedding and pre-trained language repre- sentation. Transactions of the Association for Computational Linguistics, 9:176–194, 2021. doi: 10.1162/tacl_a_00360 3

    2021 doi

  57. [68]

    Y . Yan, Y . Hou, Y . Xiao, R. Zhang, and Q. Wang. KNOWNET: Guided health information seeking from LLMs via knowledge graph integration. IEEE Transactions on Visualization and Computer Graphics, 2024. doi: 10.1109/TVCG.2024.3456364 3

    2024

  58. [69]

    Y . Yang, Y . Wang, Y . Li, S. Sen, L. Li, and Q. Liu. Unleashing the potential of large language models for predictive tabular tasks in data science. 2024. doi: 10.18653/v1/2024.findings-emnlp.224 2

    2024 doi

  59. [70]

    T. Wang, S. Chen, Y . Wang, et al. From in Silico to in Vitro: A com- prehensive guide to validating bioinformatics findings, 2025. doi: 10. 48550/arXiv.2410.07076 2, 3

    2025 doi

  60. [71]

    J. Yuan, X. Yan, B. Shi, T. Chen, W. Ouyang, B. Zhang, et al. Dolphin: Closed-loop open-ended auto-research through thinking, practice, and feedback, 2025. doi: 10.48550/arXiv.2501.03916 3

    2025 doi arXiv

  61. [72]

    Zhang, M

    K. Zhang, M. Wu, Y . Liu, Y . Feng, and J. Zheng. KR4SL: knowledge graph reasoning for explainable prediction of synthetic lethality. Bioinformatics, 39:i158–i167, 06 2023. doi: 10.1093/bioinformatics/btad261 2, 5

    2023 doi

  62. [73]

    Zhang, B

    Y . Zhang, B. Hu, Z. Chen, L. Guo, Z. Liu, Z. Zhang, L. Liang, H. Chen, and W. Zhang. Multi-domain knowledge graph collaborative pre-training and prompt tuning for diverse downstream tasks, 2024. doi: 10.48550/ arXiv.2405.13085 2

    2024 doi arXiv

  63. [74]

    R. Yao, Z. Shen, X. Xu, G. Ling, R. Xiang, T. Song, F. Zhai, and Y . Zhai. Knowledge mapping of graph neural networks for drug discovery: a bibliometric and visualized analysis. Frontiers in Pharmacology, 15, 2024. doi: 10.3389/fphar.2024.1393415 2

    2024

  64. [75]

    Zheng, Y

    C. Zheng, Y . Zhang, Z. Huang, C. Shi, M. Xu, and X. Ma. Disciplink: Unfolding Interdisciplinary Information Seeking Process via Human-AI Co-Exploration. In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology , pp. 1–20, 2024. doi: 10. 1145/3...

    2024

  65. [76]

    Zheng, Z

    T. Zheng, Z. Deng, H. T. Tsang, W. Wang, J. Bai, Z. Wang, and Y . Song. From automation to autonomy: A survey on large language models in scientific discovery, 2025. 2

    2025

  66. [77]

    Y . Zhou, H. Liu, T. Srivastava, H. Mei, and C. Tan. Hypothesis generation with large language models. 2024. doi: 10.18653/v1/2024.nlp4science-1. 10 2, 3

    2024 doi

  67. [78]

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong, Y . Du, C. Yang, Y . Chen, Z. Chen, et al. A survey of large language models, 2025. doi: 10.48550/arXiv.2303.18223 2

    2025 doi arXiv

  68. [82]

    Z. Zhou, X. Feng, L. Huang, et al. From hypothesis to publication: A comprehensive survey of AI-driven research support systems, 2025. doi: 10.48550/arXiv.2310.05150 2

    2025 doi arXiv

  69. [2020]

    doi: 10.1145/3340531.3411975 2

  70. [2024]

    doi: 10.3390/app14125068 2

    doi

  71. [2025]

    doi: ttps://doi.org/10.48550/arXiv.2502.00025 2

    doi arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.