Pith. sign in

REVIEW 5 major objections 6 minor 35 references

ImmunoFOMO: Are Language Models missing what oncologists see?

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that domain-specific pre-trained language models can identify fine-grained 'hallmark of immunotherapy' concepts in breast cancer abstracts more accurately than large language models, while large models still capture…

desk verdict A useful new benchmark with a load-bearing evaluation flaw: Tier-II/III performance is measured via aggregation to Tier-I, so the headline claim that small models beat large ones on fine-grained concepts is not yet supported. read the letter →

arxiv 2506.11478 v1 pith:6MHIP3F5 submitted 2025-06-13 cs.CL

Aman Sinha , Bogdan-Valentin Popescu , Xavier Coubez , Marianne Clausel , Mathieu Constant This is my paper
classification cs.CL
keywords hallmarksofimmunotherapyconceptualgroundingbiomedicallanguagemodelsbreastcancerabstractszero-shotclassificationunsupervisedtextTier-ITier-IITier-IIIhierarchymodelevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether language models genuinely understand the conceptual hierarchies that oncologists use, rather than just retrieving surface text. It builds a three-tier taxonomy of hallmarks of immunotherapy—9 high-level categories, 27 intermediate sub-categories, and 177 fine-grained keywords—and tests ten models on labeling breast-cancer immunotherapy abstracts with the right hallmark tier. Against a single oncologist's gold labels, the paper finds a systematic split: domain-specific pre-trained language models such as PubMedBERT improve as concepts become finer and outperform large language models at the most specific Tier-III concepts, while large models such as Gemma-2-9B capture high-level Tier-I and Tier-II categories better. The paper's significance is that model size and domain specialization may matter less than the granularity of the concepts being extracted, a consideration for biomedical NLP applications. It also documents that LLMs hallucinate nearly 57% of the time at the finest tier even when given an explicit label list.

What carries the argument

The load-bearing object is the curated three-tier hallmark-of-immunotherapy hierarchy: nine Tier-I hallmarks such as 'Tumor genome and epigenome', 'Tumour microenvironment', and 'Systemic factors'; twenty-seven Tier-II subcategories; and 177 Tier-III keywords, adapted from published hallmark-of-response and hallmark-of-resistance taxonomies. The evaluation machinery runs each abstract through two unsupervised routes: embedding-similarity matching for the five pre-trained models and zero-shot prompting with a fixed prompt for the five large language models. For scoring, predictions at Tier-II and Tier-III are mapped back to their Tier-I parent using the curated hierarchy and compared with the oncologist's Tier-I gold labels using weighted F1 and agreement metrics.

What would settle it

Annotate the same 239 abstracts with Tier-II and Tier-III gold labels from multiple oncologists and score the models directly on those tiers; if large models match or exceed PubMedBERT at Tier-III under direct scoring, the paper's central claim is refuted.

Watch

Extended reading notes

Core claim

The central discovery is that conceptual granularity, not model scale, drives which language model class is more reliable for biomedical concept identification. In the paper's unsupervised evaluations, the SLM pool improves from Tier-I to Tier-III, with PubMedBERT reaching a weighted F1 of about 0.50 at Tier-III and 0.42 overall, while the LLM pool peaks at Tier-II and then drops sharply at Tier-III, with the best LLM, Gemma-2-9B, reaching 0.38 at Tier-III and 0.40 overall. The paper interprets this as smaller models retaining focused domain knowledge from smaller pretraining corpora, whereas large models absorb broader but shallower coverage that helps with coarse categories and hurts fine-grained distinctions. It also reports that both model pools have low agreement with the expert oncologist, with Cohen's kappa below 0.12 for SLMs and below 0.17 for LLMs across all tiers.

Load-bearing premise

The argument's load-bearing premise is that comparing lower-tier model outputs after mapping them up to Tier-I labels, measured against a single oncologist's Tier-I annotations, genuinely tests whether models understand fine-grained concepts; if the mapping or the single-expert gold standard distorts the comparison, the conclusion that small models beat large ones at low-level concepts falls apart.

Editorial extensions

If this is right

  • For tasks requiring fine-grained biomedical concept detection, domain-specific pre-trained models such as PubMedBERT merit serious consideration over large language models, despite the current shift toward LLM usage.
  • The 57% Tier-III hallucination rate shows that providing an explicit label list does not keep an LLM constrained; applications needing low-level labels must validate outputs against the allowed label set.
  • Both model pools showed low agreement with the expert oncologist, so neither class of models is yet a reliable stand-in for expert annotation in this task.
  • The opposite trends across tiers suggest that the choice of model family should be guided by the desired conceptual granularity, not by model size alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the pattern generalizes, the most promising architecture is a hybrid: a domain-specialized encoder that proposes fine-grained terms and a large model that reasons over them to decide high-level categories.
  • The single-expert gold standard likely understates the human-disagreement baseline; a multi-annotator Tier-II/III benchmark could reveal whether the reported SLM advantage reflects reproducibility or partly one clinician's labeling style.
  • A testable extension is to evaluate LLMs under constrained decoding or with a verification step that forces outputs into the list; if hallucinations drop and low-level F1 rises, the paper's low-level LLM deficit may be partly a decoding artifact rather than a conceptual-grounding gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces ImmunoFOMO, a study of whether small pre-trained language models (PLMs: BioBERT, SciBERT, PubmedBERT, ClinicalBERT, ClinicalBigBird) and large language models (LLMs: Llama-3-8B, Gemma-2-9B, Med-Qwen, DeepSeek-R1, BioMistral) can identify hallmarks of immunotherapy in breast cancer abstracts at three levels of conceptual granularity (Tier-I: 9 hallmarks; Tier-II: 27 subcategories; Tier-III: 177 keywords). A single oncologist annotated each of 188 abstracts with two Tier-I labels, and models were evaluated in an unsupervised setting: SLMs via similarity-based classification and LLMs via zero-shot prompting. The paper reports F1 scores and agreement metrics, and its central claim is that SLMs outperform LLMs on low-level (Tier-III) concepts while LLMs are better at high-level (Tier-I/II) concepts. The authors also report hallucination rates for LLMs and intra-model-pool agreement.

Significance. If the central claim were established, the paper would be a valuable contribution to biomedical NLP: it would provide a concrete benchmark for conceptual grounding across model families and a structured taxonomy of immunotherapy hallmarks. The paper curates a new dataset and a three-tier taxonomy, evaluates ten models, and includes a candid limitations section. It is also notable for reporting that SLMs produce no hallucinated labels in this setting. However, as detailed in the major comments, the evaluation design does not currently support the headline claim about low-level concept identification, because Tier-II/III performance is measured only after aggregation to Tier-I labels and against Tier-I gold. With appropriate additional validation or re-annotation, the study could become a useful reference for model selection in biomedical concept extraction.

major comments (5)
  1. [Section 3, footnote 1; Tables 3-4] The Tier-II and Tier-III F1 scores are computed by mapping model predictions to Tier-I labels and comparing against Tier-I gold, not by scoring against Tier-II/III gold. This aggregation cannot support the claim that SLMs are better at low-level concepts. With 177 Tier-III labels collapsed into 9 buckets, a model that hits the correct Tier-I bucket but the wrong specific concept is counted as correct; the PubmedBERT increase from 0.33 (Tier-I) to 0.50 (Tier-III) in Table 4 is exactly the kind of artifact this design permits. The Limitations section explicitly concedes the absence of Tier-II/III expert annotations. The central claim requires per-tier gold labels, or at minimum a validated aggregation mapping with chance-level baselines.
  2. [Section B and Figure 4] The LLM protocol asks each model to return the 'top three suited categories', while the SLM protocol selects a single most similar label. This asymmetry in prediction-set size biases the comparison in favor of LLMs on recall and inflates their F1 at every tier. The paper does not control for the number of allowed predictions or report matched-condition results, yet the comparison between pools is a central contribution.
  3. [Tables 3-4 and Section 4] All F1 scores are point estimates on a single dataset of 188 abstracts, with no confidence intervals, bootstrap resampling, or significance tests. The headline differences, such as PubmedBERT 0.50 versus Gemma-2-9B 0.38 at Tier-III overall, are small relative to the expected variance given the very low agreement values in Table 2. It is therefore not established that these differences are stable or meaningful.
  4. [Section 3, Expert Annotations] The gold standard is a single expert's Tier-I annotations, with no inter-annotator reliability assessment. The Limitations section itself notes that human annotators often disagree, which makes the absence of a reliability check for the gold standard a substantive problem: all F1 scores and agreement metrics are relative to an unvalidated reference.
  5. [Section B, Reproducibility] LLM raw outputs are post-processed with GPT-3.5-turbo to extract predictions, but the paper does not evaluate the fidelity of this extraction step. If GPT-3.5 alters, drops, or adds labels, the resulting F1 scores reflect the post-processing model as much as the LLMs under test, and the comparison is no longer clean.
minor comments (6)
  1. [Section 2] The sentence 'we access language models for identification' should read 'we assess language models for identification'.
  2. [Table 2] The text 'Cκ between 0.9 - 0.17' appears to be a typo for '0.09-0.17', given the values in the table.
  3. [Table 3] The row label 'DeepSeek-R10.50 0.410.18' is malformed; it should read 'DeepSeek-R1 0.50 0.41 0.18'.
  4. [Abstract and Section 5] The abstract and conclusion state that 'pre-trained language models have potential to outperform large language models' at low-level concepts, but the results show only PubmedBERT clearly doing so; other SLMs are not consistently better than the best LLM at Tier-III.
  5. [Figure 2] The pooled trends would be easier to interpret with individual model points or error bars; the current dashed lines aggregate over models with very different scores.
  6. [Section 5, Hallucinations] The paper reports hallucination percentages but does not quantify how hallucinations affect precision or F1 separately, so the claim that hallucinations 'lead to lower precision' is not directly tested.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central comparison is an external benchmark with human gold labels; the lower-tier aggregation is a stated evaluation limitation, not a circular derivation.

full rationale

The paper does not derive its central claim from an input by construction. The hallmark hierarchy (9 Tier-I, 27 Tier-II, 177 Tier-III labels) is curated from literature and clinical experience, and the gold standard is provided by a clinical expert as two Tier-I labels per abstract. Model predictions are produced independently: PLMs use similarity-based classification and LLMs use zero-shot prompting. No parameter is fitted to the expert labels, no uniqueness theorem from the authors is invoked, and no prior result by the same authors is load-bearing. The only design point that could look circular is the scoring of Tier-II/Tier-III predictions: as the Section 3 footnote states, 'we converted the lower-level labels (Tier-II or III) to Tier-I based on the aggregation criteria provided in the hallmark list.' This means reported Tier-III F1 scores measure whether a selected low-level keyword falls in the same Tier-I bucket as the expert's high-level label, so the headline claim that SLMs are better at low-level concepts is not fully supported by a direct low-level gold standard. However, this is an evaluation-validity limitation, explicitly conceded in the Limitations section ('One of the limitation of this study is the lack of expert annotation at tier-II and tier-III'), rather than a circularity: the predictions are not defined in terms of the target labels, and the conversion is disclosed instead of hidden. Under the stated rules, methodological weakness belongs to correctness risk, not circularity. Therefore no circular step is identified.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The benchmark relies on an author-curated hallmark taxonomy (9/27/177 labels), a single expert annotator for Tier-I labels, and an indirect aggregation step that maps lower-tier predictions to Tier-I for scoring. No parameters are fitted to data; the central uncertainties are domain assumptions about whether this taxonomy, the single-expert gold, and the aggregation mapping adequately capture clinical conceptual grounding.

assumptions (5)
  • domain assumption The single clinical expert's Tier-I annotations are a reliable gold standard for conceptual grounding.
    Section 3 'Expert Annotations': only one clinical expert annotated each abstract; no inter-annotator reliability is reported.
  • domain assumption Converting Tier-II/Tier-III predictions to Tier-I via the curated hallmark hierarchy preserves the meaning of the task.
    Section 3 'Evaluation': lower-level outputs are aggregated to Tier-I labels using the mapping in Table 5; no independent validation of this aggregation is provided.
  • domain assumption Similarity-based classification with sentence embeddings is a valid zero-shot conceptual grounding test for PLMs.
    Appendix B: PLMs are used as embedders and similarity scorers; no supervised fine-tuning or calibration is applied.
  • domain assumption Zero-shot prompting of LLMs with only the label list and abstract is a representative evaluation of their conceptual grounding.
    Section 3 'Task' and Figure 4: the prompt is minimal; no chain-of-thought, examples, or task-specific instructions are included.
  • domain assumption The curated hallmark taxonomy (9/27/177 labels) is a faithful operationalization of hallmarks of immunotherapy in breast cancer.
    Section C; adapted from Morad et al., Karasarides et al., and Cogdill et al.; the list is author-curated and not independently validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ImmunoFOMO: Are Language Models missing what oncologists see?." pith.science (2026). https://pith.science/paper/6MHIP3F5

@misc{pith2026250611478,
  author       = {Pith},
  title        = {Pith review of: ImmunoFOMO: Are Language Models missing what oncologists see?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6MHIP3F5}},
  note         = {Machine review of arXiv:2506.11478}
}
read the original abstract

Language models (LMs) capabilities have grown with a fast pace over the past decade leading researchers in various disciplines, such as biomedical research, to increasingly explore the utility of LMs in their day-to-day applications. Domain specific language models have already been in use for biomedical natural language processing (NLP) applications. Recently however, the interest has grown towards medical language models and their understanding capabilities. In this paper, we investigate the medical conceptual grounding of various language models against expert clinicians for identification of hallmarks of immunotherapy in breast cancer abstracts. Our results show that pre-trained language models have potential to outperform large language models in identifying very specific (low-level) concepts.

Figures

Figures reproduced from arXiv: 2506.11478 by the authors.

Figure 1
Figure 1. Illustration of conceptual hierarchical struc [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison between SLMs and LLMs against [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Intra-alignment (A%) between LMs over three tiers of HOI concepts (left to right). [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Zero-shot prompting template used for LLM based HOI identification. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 18 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  4. [4]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, and 29 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  5. [5]

    Simon Baker and Anna Korhonen. 2017. Initializing neural networks for hierarchical multi-label text classification. Association for Computational Linguistics

    work page 2017

  6. [6]

    Simon Baker, Anna Korhonen, and Sampo Pyysalo. 2017. Cancer hallmark text classification using convolutional neural networks

    work page 2017

  7. [7]

    Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. https://arxiv.org/abs/arXiv:1903.10676 Scibert: Pretrained language model for scientific text . In EMNLP

    arXiv 2019

  8. [8]

    Catherine Blake. 2010. https://doi.org/10.1016/j.jbi.2009.11.001 Beyond genes, proteins, and abstracts: Identifying scientific claims from full-text biomedical articles . Journal of Biomedical Informatics, 43(2):173--189

    work page doi:10.1016/j.jbi.2009.11.001 2010

Show all 35 references
  1. [9]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  2. [10]

    Alexandria P Cogdill, Miles C Andrews, and Jennifer A Wargo. 2017. Hallmarks of response to immune checkpoint blockade. British journal of cancer, 117(1):1--7

    2017

  3. [11]

    DeepSeek-AI. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning . Preprint, arXiv:2501.12948

    2025 arXiv

  4. [12]

    Pieter Fivez, Simon Suster, and Walter Daelemans. 2021. Conceptual grounding constraints for truly robust biomedical name representations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2440--2450

    2021

  5. [13]

    Dobbins, Namu Park, Michael Leu, Abby R

    Yujuan Fu, Giridhar Kaushik Ramachandran, Nicholas J. Dobbins, Namu Park, Michael Leu, Abby R. Rosenberg, Kevin Lybarger, Fei Xia, Ozlem Uzuner, and Meliha Yetisgen. 2024. https://api.semanticscholar.org/CorpusID:268820219 Extracting social determinants of health from pediatri...

    2024 arXiv

  6. [14]

    Alessandra Gennari, Fabrice Andr \'e , CH Barrios, Javier Cortes, Evandro de Azambuja, Angela DeMichele, R Dent, Deborah Fenlon, Joseph Gligorov, SA Hurvitz, and 1 others. 2021. Esmo clinical practice guideline for the diagnosis, staging and treatment of patients with metastat...

    2021

  7. [15]

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1--23

    2021

  8. [16]

    Douglas Hanahan and Robert A Weinberg. 2000. The hallmarks of cancer. cell, 100(1):57--70

    2000

  9. [17]

    Maria Karasarides, Alexandria P Cogdill, Paul B Robbins, Michaela Bowden, Elizabeth M Burton, Lisa H Butterfield, Alessandra Cesano, Christian Hammer, Cara L Haymaker, Christine E Horak, and 1 others. 2022. Hallmarks of resistance to immune-checkpoint inhibitors. Cancer immuno...

    2022

  10. [18]

    Mehrdad Khatir and Chandan K Reddy. 2024. Concept formation and alignment in language models: Bridging statistical patterns in latent space to concept taxonomy. arXiv preprint arXiv:2406.05315

    2024 arXiv

  11. [19]

    Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. https://arxiv.org/abs/2402.10373 Biomistral: A collection of open-source pretrained large language models for medical domains . Preprint, arXiv:2402.10373

    2024 arXiv

  12. [20]

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234--1240

    2020

  13. [21]

    Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.clinicalnlp-1.17 Pretrained language models for biomedical and clinical tasks: Understanding and extending the state-of-the-art . In Proceedings of the 3rd Clinical Natural Langua...

    2020 doi

  14. [22]

    Yikuan Li, Ramsey M Wehbe, Faraz S Ahmad, Hanyin Wang, and Yuan Luo. 2022. Clinical-longformer and clinical-bigbird: Transformers for long clinical sequences. arXiv preprint arXiv:2201.11838

    2022 arXiv

  15. [23]

    Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, and David A. Clifton. 2024. https://arxiv.or...

    2024 arXiv

  16. [24]

    Sibylle Loibl, F Andr \'e , T Bachelot, CH Barrios, J Bergh, HJ Burstein, MJ Cardoso, LA Carey, S Dawood, L Del Mastro, and 1 others. 2024. Early breast cancer: Esmo clinical practice guideline for diagnosis, treatment and follow-up. Annals of Oncology, 35(2):159--182

    2024

  17. [25]

    Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. https://doi.org/10.1093/bib/bbac409 BioGPT: generative pre-trained transformer for biomedical text generation and mining . Briefings in Bioinformatics, 23(6). Bbac409

    2022 doi

  18. [26]

    Golnaz Morad, Beth A Helmink, Padmanee Sharma, and Jennifer A Wargo. 2021. Hallmarks of response, resistance, and toxicity to immune checkpoint blockade. Cell, 184(21):5309--5337

    2021

  19. [27]

    Hambly, Zoe Triplett, Nahyan Asif, Anindya Susanto, Anamika Chowdhury, Amaya Azcoaga Lorenzo, Mark Dras, and Shlomo Berkovsky

    David Fraile Navarro, Enrico Coiera, Thomas W. Hambly, Zoe Triplett, Nahyan Asif, Anindya Susanto, Anamika Chowdhury, Amaya Azcoaga Lorenzo, Mark Dras, and Shlomo Berkovsky. 2025. https://api.semanticscholar.org/CorpusID:275385218 Expert evaluation of large language models for...

    2025

  20. [28]

    Tim Schopf, Daniel Braun, and Florian Matthes. 2022. https://doi.org/10.1145/3582768.3582795 Evaluating unsupervised text classification: Zero-shot and similarity-based approaches . In Proceedings of the 2022 6th International Conference on Natural Language Processing and Info...

    2022

  21. [29]

    Yangqiu Song and Dan Roth. 2014. On dataless hierarchical text classification. In Proceedings of the AAAI conference on artificial intelligence, volume 28

    2014

  22. [30]

    Yansen Su, Minglu Wang, Pengpeng Wang, Chunhou Zheng, Yuansheng Liu, and Xiangxiang Zeng. 2022. https://doi.org/10.1093/bib/bbac342 Deep learning joint models for extracting entities and relations in biomedical: a survey and comparison . Briefings in Bioinformatics, 23(6):bbac342

    2022 doi

  23. [31]

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi \`e re, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295

    2024 arXiv

  24. [32]

    Sappadla Prateek Veeranna, Jinseok Nam, Eneldo Loza Menc a, and Johannes F \"u rnkranz. 2016. Using semantic similarity for multi-label zero-shot classification of text documents. In Proceeding of european symposium on artificial neural networks, computational intelligence and...

    2016

  25. [33]

    Workum, Bas W

    Jessica D. Workum, Bas W. S. Volkers, Davy van de Sande, Sumesh Arora, Marco Goeijenbier, Diederik A.M.P.J. Gommers, and Michel E. van Genderen. 2025. https://api.semanticscholar.org/CorpusID:276255602 Comparative evaluation and performance of large language models on expert l...

    2025

  26. [34]

    Lawrence K. Q. Yan, Qian Niu, Ming Li, Yichao Zhang, Caitlyn Heqi Yin, Cheng Fei, Benji Peng, Ziqian Bi, Pohsun Feng, Keyu Chen, Tianyang Wang, Yunze Wang, Silin Chen, Ming Liu, and Junyu Liu. 2024. https://arxiv.org/abs/2410.21348 Large language model benchmarks in medical ta...

    2024

  27. [35]

    Xinlu Zhang, Chenxin Tian, Xianjun Yang, Lichang Chen, Zekun Li, and Linda Ruth Petzold. 2023. https://arxiv.org/abs/2310.14558 Alpacare:instruction-tuned large language models for medical application . Preprint, arXiv:2310.14558

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.