Pith. sign in

REVIEW 4 major objections 6 minor 44 references

Context-Aware Scientific Knowledge Extraction on Linked Open Data using Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that WISE, a tree-based LLM workflow, answers complex scientific queries with higher recall and more detail than search engines or chat LLMs.

desk verdict A sensible workflow architecture for LLM-based knowledge extraction, but the headline recall result is undermined by a circular evaluation that largely defines its own gold standard. read the letter →

arxiv 2506.17580 v1 pith:OEP662EE submitted 2025-06-21 cs.IR cs.AIcs.DLcs.ET

Sajratul Y. Rubaiat , Hasan M. Jamil This is my paper
classification cs.IRcs.AIcs.DLcs.ET
keywords knowledgeextractionlargelanguagemodelslinkedopendatagene-diseaseassociationtree-basedsearchinformationfilteringscientificsynthesisretrieval-augmentedgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces WISE, a tree-based workflow that uses an LLM to filter, score, and prune web sources while expanding a growing answer container, and argues this design solves a specific failure: search engines list sources but do not synthesize, while chat LLMs synthesize but are limited by context windows and stale knowledge. On an HBB gene-disease query, WISE reports recall of 0.842 versus 0.474 or lower for plain chat, search-augmented chat, a rival chat model, and web search, and an average detail level of 3.81 versus 3.42 or lower for the baselines. It also reports reducing the volume of processed text by more than 80%. The sympathetic reading is that the paper is trying to establish that a workflow, rather than a bigger model or a different query, is what unlocks complete and deep scientific answers from the open web.

What carries the argument

The load-bearing object is the recursive tree search around a growing knowledge container $K_l$. At layer $l$, each source's raw content $C(s_i)$ is reduced by an LLM filter $\Gamma(q, C(s_i))$ to the words relevant to query $q$; the filtered content $F(s_i)$ gets score $\Psi(F(s_i), K_l) = K(s_i)/\log(1 + w_{\text{filtered}}(s_i) + |K_l|)$, where $K(s_i)$ counts filtered words not already in the container. The top-$k$ sources are expanded into the next layer only if $\max_i \Psi(F(s_i), K_l) \ge T$, with $T = 20$ in the experiment, and their filtered text is merged into $K_l$. This scoring function simultaneously ranks sources, prunes low-yield branches, and decides when to stop, which is the whole of the claimed advantage.

What would settle it

Assemble an expert-curated list of HBB-associated diseases and phenotypes from clinical genetics records, run the paper's query through WISE and the four baselines, and compute recall against that fixed list; if WISE's recall drops toward the baseline range, the claimed comprehensiveness is an artifact of the self-constructed reference set.

Watch

Extended reading notes

Core claim

WISE's central contention is that comprehensiveness, not just relevance ranking, is the bottleneck in scientific question answering, and that a recursive progressive-deepening search can solve it. Starting from a small root set of linked resources, the system filters each source with an LLM, scores it by how many new words it would add to a growing answer container, keeps the most additive sources, follows their links, and stops when no source clears the threshold. On the HBB gene-disease query, the paper reports WISE identifies 16 of 19 diseases in the combined reference set (recall 0.842) versus 9 for plain ChatGPT, 7 for search-augmented ChatGPT, 3 for ordinary web search, and 2 for Gemini, and its average detail level of 3.81 also tops every baseline. The paper further claims WISE's answers are the most distinct by ROUGE and BLEU comparisons and that the workflow generalizes as a template for drug discovery, materials science, and social science queries.

Load-bearing premise

The recall result assumes the combined list of diseases produced by the five systems is a valid gold standard, yet WISE itself supplies most of that list, so the high recall is partly built into the measurement.

Editorial extensions

If this is right

  • In the paper's single HBB experiment, WISE identifies 16 HBB-associated diseases versus 9, 7, 3, and 2 for the four baselines, including rare conditions such as Hemoglobin S Oman and beta-type methemoglobinemia that the other systems miss.
  • WISE reaches an average detail level of 3.81 on the paper's 0-5 scale, versus 3.42 or lower for baselines, so users get mechanism, diagnosis, treatment, or research-link information rather than just disease names.
  • WISE reduces processed text by over 80% across the source set, which is what lets a large linked-data tree be searched within practical compute budgets and context windows.
  • Under ROUGE and BLEU comparisons, WISE's output is the least repetitive of the five systems, meaning much of its content is not found in the baseline answers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the recall denominator is the union of the five systems' own outputs, and WISE supplies 16 of the 19 entries in that union, so the reported margin is probably optimistic relative to an externally curated gold standard.
  • Editorial inference: the same pipeline is likely portable to other gene queries and non-biological corpora, but the fixed threshold and top-k depth will need retuning because source graphs and redundancy rates vary by domain.
  • Editorial inference: the scoring function treats novelty as new vocabulary, so conceptual novelty expressed in familiar terms can be undervalued; the paper's own sketch of knowledge-graph integration points toward a more semantic notion of contribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents WISE, a tree-structured workflow in which an LLM filters source content, a scoring function ranks sources by their unique word contribution to an evolving 'knowledge container', and a threshold plus top-k selection controls recursive expansion. The empirical section evaluates WISE on a single query about diseases associated with the HBB gene, comparing it against ChatGPT, ChatGPT with Search, Gemini, and Google Search. The headline results are recall 0.842 versus at most 0.474, lower ROUGE/BLEU overlap, and a higher average level of detail (3.81 versus at most 3.42). The paper also sketches applications in drug discovery, materials science, and social science.

Significance. If the empirical claims were valid, WISE would be a useful synthesis tool for complex domain queries, and the idea of explicit uniqueness scoring with adaptive stopping is a reasonable design direction. The system architecture and Algorithm 1 are described clearly, and the content-filtering reduction figures provide a useful engineering data point. However, this is an empirical systems paper with a single-query evaluation and no released code or data, and the main quantitative evidence is not currently trustworthy because of the self-referential recall protocol and uncontrolled manual depth annotation. The significance therefore rests on whether the evaluation can be repaired; the conceptual contribution alone is modest.

major comments (4)
  1. [Section IV-A.2, Table II] The recall denominator is the union of all unique diseases identified by any of the five systems, and Table II shows that WISE alone contributes 16 of the 19 diseases in that union. This makes recall self-referential: a system that emits more candidate names will automatically achieve higher recall, regardless of whether those names are genuinely HBB-associated diseases. The reported gap between 0.842 and 0.474 therefore does not by itself establish superior coverage of the true disease space. Please recompute recall against an externally curated reference (e.g., OMIM, ClinVar, or MedGen entries for HBB) or against a pooled set with independent relevance judgments, and report precision and false-discovery counts. The WISE-only row 'Malaria' in Table II illustrates the risk that overgenerous extraction is being counted as recall.
  2. [Section III-A and Section VIII] The manuscript states in Section III-A that the experiment 'started with an initial set of 24 sources' from HGNC, but Section VIII says that WISE successfully extracted content from 24 of 34 initial sources and that 10 were inaccessible. These statements are inconsistent. If 24 is the number of successfully accessed sources, the authors need to describe how the initial 34 were obtained and whether the 10 inaccessible sources could have changed the recall comparison; if 24 is the true initial set, Section VIII and Figure 8 are incorrect.
  3. [Section IV-A.3] The level-based depth metric relies on levels 0-5 assigned manually to each identified disease, but the paper does not report blinding, a detailed codebook, or inter-rater agreement. Because the annotators know which output came from which system, the 3.81 versus 3.33 gap could reflect expectation bias or output formatting rather than actual information depth. Please provide a blinded dual-annotation protocol with inter-rater reliability statistics and report the full level distribution for each system.
  4. [Section II-C and Section III-D] The threshold T (set to 20) and the top-k number of sources (top two per layer) are empirical choices, but the paper gives no sensitivity analysis and no ablation showing how the reported recall and depth scores vary with these parameters. Since the recall comparison is the central claim, the authors should show that the WISE advantage is not an artifact of a tuned stopping rule.
minor comments (6)
  1. [Table I and Table II] The recall values in Table I are reported without precision or the total number of candidate diseases emitted by each system; adding precision would help interpret the recall gap.
  2. [Title and Abstract] The title and abstract emphasize Linked Open Data, but the experimental pipeline is built from HTML pages and REST links from HGNC, UniProt, ClinVar, and similar sources; please clarify what makes these Linked Open Data or adjust the framing.
  3. [Figure 1] Figure 1 is difficult to read: the rightmost panel's source counts appear as '9 134 248', and the caption does not explain the three panels clearly.
  4. [Section V] Section V presents drug discovery, materials science, and social science as applications, but no experiments are reported there; this section should be framed as speculative future directions rather than demonstrated versatility.
  5. [Section VIII vs Section II-B] Section VIII mentions that frequent words are 'deprioritized based on term frequency (TF)', but Section II-B's scoring function uses exact word overlap counts with no TF weighting; the discrepancy should be fixed.
  6. [Reproducibility] The paper does not state whether code or the exact prompts used for the LLM filtering function Γ are available; this is important for reproducing an LLM-based workflow.

Circularity Check

2 steps flagged · score 7.0 of 10

The headline recall advantage is substantially by construction: the gold standard is the union of all five systems' outputs, and WISE itself supplies 16 of the 19 diseases in that union.

  1. self definitional [Section IV-A.2 (Recall), Table II]
    "To evaluate the comprehensiveness of each system, we calculated their recall based on a combined output created by taking the union of all unique diseases identified by any of the five systems. This combined output, representing a comprehensive collection of potentially relevant diseases, is detailed in Table II. WISE achieved a recall of 0.842, significantly outperforming the baseline systems, as shown in Figure 6."

    The recall denominator is defined as the union of all five systems' outputs, one of which is WISE itself. Table II lists 19 diseases in this union, and WISE contributed 16 of them (about 84%). Thus the reference set against which WISE is measured is largely composed of WISE's own extractions. Any system that emits more unique disease names will mechanically achieve higher recall against such a pooled reference, so the reported 0.842 versus 0.474/0.368/0.105/0.158 gap does not demonstrate agreement with an independent curated ground truth such as OMIM or ClinVar. The central recall claim is therefore partly by construction rather than an independent measurement of comprehensiveness.

  2. fitted input called prediction [Section III-D (Thresholding and Knowledge Container Construction) and Section IV-A.2]
    "In this experiment, a threshold value of 20 was empirically determined to effectively balance exploration and exploitation. ... The knowledge container, constructed by integrating the filtered content from the top two sources at each layer, evolves into a rich and contextually relevant repository of information."

    The threshold T=20 and the choice of the top two sources per layer were empirically selected on the same HBB query Q that is later used to report WISE's headline recall of 0.842. Because the threshold and top-k control when exploration stops and which sources enter the knowledge container, they directly determine how many diseases WISE emits. Reporting recall on the very query used to tune these parameters is therefore not a prediction on a held-out query; it is an evaluation of a configuration fitted to that test case. This amplifies the self-referential recall measurement and weakens the paper's claim of generalizable superiority.

full rationale

The paper's strongest quantitative claim is that WISE achieves markedly higher recall than four baselines for HBB-associated diseases. That claim depends on a recall denominator built as the union of all five systems' outputs, with WISE supplying 16 of the 19 diseases in Table II. Consequently, the high recall is substantially by construction: WISE is being measured against a reference list it largely generated itself, with no independent relevance judgments or curated external gold standard. Additionally, the system's threshold and top-k parameters were empirically tuned on the same HBB query used for evaluation, so the reported recall is also a fitted result rather than a held-out prediction. The level-based detail metric and ROUGE/BLEU uniqueness comparisons are not circular in the same way, though the level annotations are manually assigned and ROUGE/BLEU low overlap is only an indirect proxy for quality. There is no load-bearing self-citation chain in this paper; the circularity is entirely in the evaluation protocol. Overall, the central recall comparison reduces in part to the system's own output defining the target set, warranting a score of 7.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central evaluation rests on several hand-set or untested choices: the threshold T=20, top-k=2, manual level assignments, and the unvalidated assumption that LLM filtering loses no relevant content. The scoring function also assumes exact word overlap captures semantic redundancy. None of these are benchmarked independently, so the paper's reported gains are not robustly grounded.

free parameters (3)
  • Threshold T = 20
    Section III-D states the threshold was empirically determined to balance exploration and exploitation. It is tuned on the single HBB query and not varied.
  • Top-k sources per layer = 2
    Section III-D describes building the knowledge container from the top two sources at each layer. No sensitivity analysis is provided.
  • Manual level assignments = 0-5 per disease
    Section IV-A.3 describes manually assigned detail levels. The assignment is subjective and not blinded.
assumptions (5)
  • domain assumption The initial similarity search Phi(q) returns a relevant seed set of sources.
    WISE depends on S0 = Phi(q) to start the tree. The quality of this seed set is not evaluated.
  • domain assumption The LLM filter Gamma retains all query-relevant content and discards only irrelevant content.
    Section II-A defines Gamma. If the LLM drops relevant passages, recall is lost, but the filter's accuracy is never measured.
  • ad hoc to paper Exact word overlap between a filtered source and the knowledge container measures redundancy.
    Section II-B defines K(si) = |F(si) intersect Kl|, treating words as set elements. This ignores synonyms, paraphrases, and semantic similarity, a crude proxy for redundancy.
  • domain assumption The threshold termination criterion does not lose rare but important sources.
    Section II-C stops when max score < T. Rare conditions that appear only in low-scoring sources would be missed, and there is no evidence this did not happen.
  • domain assumption The LLM fusion function Lambda merges content without hallucination or information loss.
    Section II-D describes Lambda as an LLM-powered fusion function, but the fidelity of this merge is not audited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Context-Aware Scientific Knowledge Extraction on Linked Open Data using Large Language Models." pith.science (2026). https://pith.science/paper/OEP662EE

@misc{pith2026250617580,
  author       = {Pith},
  title        = {Pith review of: Context-Aware Scientific Knowledge Extraction on Linked Open Data using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OEP662EE}},
  note         = {Machine review of arXiv:2506.17580}
}
read the original abstract

The exponential growth of scientific literature challenges researchers extracting and synthesizing knowledge. Traditional search engines return many sources without direct, detailed answers, while general-purpose LLMs may offer concise responses that lack depth or omit current information. LLMs with search capabilities are also limited by context window, yielding short, incomplete answers. This paper introduces WISE (Workflow for Intelligent Scientific Knowledge Extraction), a system addressing these limits by using a structured workflow to extract, refine, and rank query-specific knowledge. WISE uses an LLM-powered, tree-based architecture to refine data, focusing on query-aligned, context-aware, and non-redundant information. Dynamic scoring and ranking prioritize unique contributions from each source, and adaptive stopping criteria minimize processing overhead. WISE delivers detailed, organized answers by systematically exploring and synthesizing knowledge from diverse sources. Experiments on HBB gene-associated diseases demonstrate WISE reduces processed text by over 80% while achieving significantly higher recall over baselines like search engines and other LLM-based approaches. ROUGE and BLEU metrics reveal WISE's output is more unique than other systems, and a novel level-based metric shows it provides more in-depth information. We also explore how the WISE workflow can be adapted for diverse domains like drug discovery, material science, and social science, enabling efficient knowledge extraction and synthesis from unstructured scientific papers and web sources.

Figures

Figures reproduced from arXiv: 2506.17580 by the authors.

Figure 1
Figure 1. Although capable of ranking and comparing content [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Number of sources, such as UniProt [16], that can be processed [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. WISE System Architecture, showcasing its four main components: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: WISE filtering and scoring in selected 4 sources, showing content [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 6
Figure 6. Figure 6: Recall scores of each system, demonstrating that WISE identifies a [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 4
Figure 4. Figure 4: Scores for Ranks 1, 2, and 3 across layers, highlighting their gradual [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Average ROUGE and BLEU scores for each system when used as a reference, demonstrating that WISE’s output is the most distinct and least [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Average level of detail for each system, demonstrating WISE’s superior ability to provide in-depth and comprehensive information about identified [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The number of sources that restrict data extraction by implementing [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 28 canonical work pages

  1. [1]

    Challenges and best practices for digital unstructured data enrichment in health research: A systematic narrative review,

    J. Sedlakova, P. Daniore, A. Horn Wintsch, M. Wolf, M. Stanikic, C. Haag, C. Sieber, G. Schneider, K. Staub, D. Alois Ettlin, O. Gr ¨ubner, F. Rinaldi, V . von Wyl, and for the University of Zurich Digital Society Initiative (UZH-DSI) Health Community, “Challenges and best practices for digital unstructured data enrichment in health research: A systematic...

    work page

  2. [2]

    Big data application in biomedical research and health care: A literature review,

    J. Luo, M. Wu, D. Gopukumar, and Y . Zhao, “Big data application in biomedical research and health care: A literature review,” Biomedical Informatics Insights , vol. 8, p. BII.S31559, 2016, pMID: 26843812. [Online]. Available: https://doi.org/10.4137/BII.S31559

    doi:10.4137/bii.s31559 2016

  3. [3]

    Large language models and future of information retrieval: Opportunities and challenges,

    C. Zhai, “Large language models and future of information retrieval: Opportunities and challenges,” in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 481–490

    work page 2024

  4. [4]

    Towards a search engine for machines: Unified ranking for multiple retrieval-augmented large language models,

    A. Salemi and H. Zamani, “Towards a search engine for machines: Unified ranking for multiple retrieval-augmented large language models,” in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , ser. SIGIR ’24. New York, NY , USA: Association for Computing Machinery, 2024, p. 741–751. [Online]. Av...

    arXiv 2024

  5. [5]

    Large Language Models are Built-in Autoregressive Search Engines

    N. Ziems, W. Yu, Z. Zhang, and M. Jiang, “Large language models are built-in autoregressive search engines,” arXiv preprint arXiv:2305.09612, 2023

    work page Pith review arXiv 2023

  6. [6]

    HBB Gene - Gene Symbol Report,

    HUGO Gene Nomenclature Committee (HGNC), “HBB Gene - Gene Symbol Report,” 2024, accessed: 2024-12-11. [Online]. Available: https: //www.genenames.org/data/gene-symbol-report/#!/hgnc id/HGNC:4827

    work page 2024

  7. [7]

    HBB Gene - Clinical Genome Knowledge Base,

    Clinical Genome Resource (ClinGen), “HBB Gene - Clinical Genome Knowledge Base,” 2024, accessed: 2024-12-11. [Online]. Available: https://search.clinicalgenome.org/kb/genes/HGNC:4827

    work page 2024

  8. [8]

    [Online]

    National Center for Biotechnology Information (NCBI), Sickle Cell Anemia - NCBI Bookshelf , 2024, accessed: 2024-12-11. [Online]. Available: https://www.ncbi.nlm.nih.gov/books/NBK1435/

    work page 2024

Show all 44 references
  1. [9]

    A review on scientific knowledge extraction using large language models in biomedical sciences,

    G. L. Garcia, J. R. R. Manesco, P. H. Paiola, L. Miranda, M. P. de Salvo, and J. P. Papa, “A review on scientific knowledge extraction using large language models in biomedical sciences,” 2024. [Online]. Available: https://arxiv.org/abs/2412.03531

    2024 arXiv

  2. [10]

    Harnessing the power of chatgpt for automating systematic review process: Methodology, case study, limitations, and future directions,

    A. Alshami, M. Elsayed, E. Ali, A. E. E. Eltoukhy, and T. Zayed, “Harnessing the power of chatgpt for automating systematic review process: Methodology, case study, limitations, and future directions,” Systems, vol. 11, no. 7, 2023. [Online]. Available: https://www.mdpi. com/2...

    2023

  3. [11]

    Large-Scale Knowledge Synthesis and Complex Information Retrieval from Biomedical Documents ,

    S. Saxena, R. Sangani, S. Prasad, S. Kumar, M. Athale, R. Awhad, and V . Vaddina, “ Large-Scale Knowledge Synthesis and Complex Information Retrieval from Biomedical Documents ,” in 2022 IEEE International Conference on Big Data (Big Data) . Los Alamitos, CA, USA: IEEE Compute...

    2022

  4. [12]

    Modelling stopping criteria for search results using Poisson processes,

    A. Sneyd and M. Stevenson, “Modelling stopping criteria for search results using Poisson processes,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNL...

    2019

  5. [13]

    Adaptive stopping algorithms based on concentration inequalities,

    M. Parmentier and A. Legay, “Adaptive stopping algorithms based on concentration inequalities,” in Bridging the Gap Between AI and Reality: Second International Conference, AISoLA 2024, Crete, Greece, October 30 – November 3, 2024, Proceedings . Berlin, Heidelberg: Springer-Ve...

    2024 doi

  6. [14]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023. [Online]. Available: https://arxiv.org/abs/1706.03762

    2023 arXiv

  7. [15]

    [Online]

    OpenAI, “Gpt-4o,” 2024, the GPT-4o model supports a context window of up to 128,000 tokens. [Online]. Available: https: //platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo

    2024

  8. [16]

    Hemoglobin subunit beta,

    U. Consortium, “Hemoglobin subunit beta,” 2024, accessed: 2024-12-

    2024

  9. [17]

    Mining knowledge at multiple concept levels,

    J. Han, “Mining knowledge at multiple concept levels,” in Proceedings of the fourth international conference on Information and knowledge management, 1995, pp. 19–24

    1995

  10. [18]

    A brief review on search engine optimization,

    D. Sharma, R. Shukla, A. K. Giri, and S. Kumar, “A brief review on search engine optimization,” in 2019 9th international conference on cloud computing, data science & engineering (confluence). IEEE, 2019, pp. 687–692

    2019

  11. [19]

    Available: https://www.uniprot.org/uniprotkb/P68871

    [Online]. Available: https://www.uniprot.org/uniprotkb/P68871

  12. [20]

    Putting gpt-4o to the sword: A comprehensive evaluation of language, vision, speech, and multimodal proficiency,

    S. Shahriar, B. D. Lund, N. R. Mannuru, M. A. Arshad, K. Hayawi, R. V . K. Bevara, A. Mannuru, and L. Batool, “Putting gpt-4o to the sword: A comprehensive evaluation of language, vision, speech, and multimodal proficiency,” Applied Sciences , vol. 14, no. 17, 2024. [Online]. ...

    2024

  13. [21]

    Introducing chatgpt search,

    OpenAI, “Introducing chatgpt search,” https://openai.com/index/ introducing-chatgpt-search/, 2024, accessed: 2024-12-27

    2024

  14. [22]

    Highly accurate protein structure prediction with alphafold,

    J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. ˇZ´ıdek, A. Potapenko et al., “Highly accurate protein structure prediction with alphafold,” nature, vol. 596, no. 7873, pp. 583–589, 2021

    2021

  15. [23]

    Gemini: a family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican et al., “Gemini: a family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  16. [24]

    Google search as an additional source in systematic reviews,

    J. Piasecki, M. Waligora, and V . Dranseika, “Google search as an additional source in systematic reviews,” Science and engineering ethics, vol. 24, pp. 809–810, 2018

    2018

  17. [25]

    Is chatgpt good at search? investigating large language models as re- ranking agents,

    W. Sun, L. Yan, X. Ma, S. Wang, P. Ren, Z. Chen, D. Yin, and Z. Ren, “Is chatgpt good at search? investigating large language models as re- ranking agents,” arXiv preprint arXiv:2304.09542 , 2023

    2023 arXiv

  18. [26]

    Recall-oriented understudy for gisting evaluation (rouge),

    C. Lin, “Recall-oriented understudy for gisting evaluation (rouge),” Retrieved August, vol. 20, p. 2005, 2005

    2005

  19. [27]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318

    2002

  20. [28]

    The google similarity distance,

    R. L. Cilibrasi and P. M. Vitanyi, “The google similarity distance,” IEEE Transactions on knowledge and data engineering , vol. 19, no. 3, pp. 370–383, 2007

    2007

  21. [29]

    Identifying protein-protein interaction using tree lstm and structured attention,

    M. Ahmed, J. Islam, M. R. Samee, and R. E. Mercer, “Identifying protein-protein interaction using tree lstm and structured attention,” in 2019 IEEE 13th International Conference on Semantic Computing (ICSC), 2019, pp. 224–231

    2019

  22. [30]

    Neural network-based approaches for biomedical relation classification: A review,

    Y . Zhang, H. Lin, Z. Yang, J. Wang, Y . Sun, B. Xu, and Z. Zhao, “Neural network-based approaches for biomedical relation classification: A review,” J. of Biomedical Informatics , vol. 99, no. C, Nov. 2019. [Online]. Available: https://doi.org/10.1016/j.jbi.2019.103294

    2019

  23. [31]

    Large language model based framework for automated extraction of genetic interactions from unstructured data,

    J. K. Gill, M. Chetty, S. Lim, and J. Hallinan, “Large language model based framework for automated extraction of genetic interactions from unstructured data,” PLOS ONE , vol. 19, no. 5, pp. 1–22, 05 2024. [Online]. Available: https://doi.org/10.1371/journal.pone.0303231

    2024 doi

  24. [32]

    Biomedical relation extraction: From binary to complex,

    D. Zhou, D. Zhong, and Y . He, “Biomedical relation extraction: From binary to complex,” Computational and Mathematical Methods in Medicine , vol. 2014, no. 1, p. 298473, 2014. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1155/2014/298473

    2014 doi

  25. [33]

    Hy- bridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction,

    B. Sarmah, D. Mehta, B. Hall, R. Rao, S. Patel, and S. Pasquali, “Hy- bridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction,” in Proceedings of the 5th ACM International Conference on AI in Finance , 2024, pp. 608– 616. 12

    2024

  26. [34]

    Biobert: a pre-trained biomedical language representation model for biomedical text mining,

    J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “Biobert: a pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020

    2020

  27. [35]

    Tree of science - tos: A web-based tool for scientific literature recommendation. search less, research more!

    M. Zuluaga, S. Robledo, O. Arbelaez-Echeverri, G. A. Osorio-Zuluaga, and N. Duque-M ´endez, “Tree of science - tos: A web-based tool for scientific literature recommendation. search less, research more!” Issues in Science and Technology Librarianship , no. 100, Aug

  28. [36]

    Embedding and extraction of knowledge in tree ensemble classifiers,

    W. Huang, X. Zhao, and X. Huang, “Embedding and extraction of knowledge in tree ensemble classifiers,” Mach. Learn. , vol. 111, no. 5, p. 1925–1958, May 2022. [Online]. Available: https: //doi.org/10.1007/s10994-021-06068-6

    1925 doi

  29. [37]

    Accelerating scientific discovery with generative knowledge extraction, graph-based representation, and multimodal intelligent graph reasoning,

    M. J. Buehler, “Accelerating scientific discovery with generative knowledge extraction, graph-based representation, and multimodal intelligent graph reasoning,” Machine Learning: Science and Technology, vol. 5, no. 3, p. 035083, sep 2024. [Online]. Available: https://dx.doi.or...

    2024 doi

  30. [38]

    Workflowllm: Enhancing workflow orchestration capability of large language models,

    S. Fan, X. Cong, Y . Fu, Z. Zhang, S. Zhang, Y . Liu, Y . Wu, Y . Lin, Z. Liu, and M. Sun, “Workflowllm: Enhancing workflow orchestration capability of large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2411.05451

    2024 arXiv

  31. [39]

    Challenges and advances in information extraction from scientific literature: a review,

    Z. Hong, L. Ward, K. Chard, B. Blaiszik, and I. Foster, “Challenges and advances in information extraction from scientific literature: a review,” JOM, vol. 73, no. 11, pp. 3383–3400, 2021. [Online]. Available: https://doi.org/10.1007/s11837-021-04902-9

    2021 doi

  32. [40]

    Bytescience: Bridging unstructured scientific literature and structured data with auto fine-tuned large language model in token granularity,

    T. Xie, H. Zhang, S. Wang, Y . Wan, I. Razzak, C. Kit, W. Zhang, and B. Hoex, “Bytescience: Bridging unstructured scientific literature and structured data with auto fine-tuned large language model in token granularity,” 2024. [Online]. Available: https://arxiv.org/abs/2411.12000

    2024 arXiv

  33. [41]

    Multi-source knowledge pruning for retrieval-augmented generation: A benchmark and empirical study,

    S. Yu, M. Cheng, J. Yang, J. Ouyang, Y . Luo, C. Lei, Q. Liu, and E. Chen, “Multi-source knowledge pruning for retrieval-augmented generation: A benchmark and empirical study,” 2024. [Online]. Available: https://arxiv.org/abs/2409.13694

    2024

  34. [2022]

    Available: https://journals.library.ualberta.ca/istl/index

    [Online]. Available: https://journals.library.ualberta.ca/istl/index. php/istl/article/view/2696

  35. [2023]

    Available: https://doi.org/10.1371/journal.pdig.0000347

    [Online]. Available: https://doi.org/10.1371/journal.pdig.0000347

    doi

  36. [2369]

    Available: https://doi.ieeecomputersociety.org/10.1109/ BigData55660.2022.10020725

    [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/ BigData55660.2022.10020725

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.