Pith. sign in

REVIEW 3 major objections 5 minor 57 references

Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned small LLMs classify biomedical topic relations at 86–92% F1, up 34 points on average.

desk verdict Fine-tuning small LLMs gives real F1 gains on a new MeSH relation benchmark, but the construction of the 'other' class is underspecified and could inflate the headline number. read the letter →

arxiv 2607.17902 v1 pith:AKPFCODS submitted 2026-07-20 cs.DL cs.CL

Tanay Aggarwal , Angelo Salatino , Francesco Osborne , Enrico Motta This is my paper
classification cs.DLcs.CL
keywords largelanguagemodelsbiomedicalontologiesontologygenerationMeSH-Rel-4Kfine-tuningchain-of-thoughtpromptingknowledgeorganizationsystemssemanticrelationclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that small, open-source language models (up to 9 billion parameters) can reliably generate biomedical ontology relations when fine-tuned on a targeted dataset, despite their poor performance under prompting. On MeSH-Rel-4K, a new 4,000-pair dataset built from Medical Subject Headings, fine-tuning raises average macro-F1 by 34.1 percentage points over chain-of-thought prompting, with all five models landing between 86.2% and 91.6%. The practical point is that manual ontology curation, a known bottleneck in knowledge organization, could be delegated to cheap, locally run models. The paper also shows that prompting alone tops out at 71.6%, and that the smallest model gains the most from fine-tuning.

What carries the argument

The load-bearing object is MeSH-Rel-4K, a dataset of 4,000 relation pairs sampled from MeSH: 1,000 broader, 1,000 narrower, 1,000 same-as, and 1,000 'other' pairs built from semantically disjoint topics. The experiment contrasts three adaptation strategies: standard prompting, chain-of-thought two-way prompting with bidirectional consistency and empirical referee rules, and LoRA-based fine-tuning with the simple template 'Classify the relationship between A and B'. The decisive mechanism is the fine-tuning step: training on 2,800 examples with a fixed output format teaches the models the label space, whereas prompting relies on in-context reasoning.

What would settle it

Take a random sample of the 1,000 'other' pairs and have two biomedical curators judge, using the full MeSH graph plus their own expertise, whether each pair is related through any chain of broader/narrower or synonym edges. If a substantial share (say more than 10%) is connected, the reported 'other' F1 of 94.3% is an artifact of mislabeled negatives.

Watch

Extended reading notes

Core claim

The central claim is that targeted fine-tuning exceeds the reasoning bottlenecks of smaller LLMs for classifying semantic relationships between biomedical concepts. Across five models with 3B–9B parameters evaluated on a held-out 800-pair test set from MeSH-Rel-4K, fine-tuning yields macro-F1 scores of 86.2–91.6%, while standard prompting reaches 25.7–66.9% and chain-of-thought prompting reaches 25.7–71.6%. The paper attributes the improvement to parameter-efficient fine-tuning, which re-anchors the models onto the task's label vocabulary and output format; prompting alone, including a bidirectional two-stage chain-of-thought, cannot close the gap. Error analysis traces residual confusion to

Load-bearing premise

The load-bearing premise is that the 'other' pairs in MeSH-Rel-4K are truly unrelated: the paper describes them only as 'semantically disjoint topics' with no stated criterion, so if some are actually connected through indirect MeSH paths or missing annotations, the high F1 on 'other' and the overall 34-point gain are inflated.

Editorial extensions

If this is right

  • If correct, fine-tuned sub-10B models can serve as automated ontology builders, easing the manual-curation bottleneck in knowledge organization systems.
  • The smallest model gains 60.5 percentage points from fine-tuning, suggesting model size is less limiting than adaptation method.
  • The 'other' category is easiest (94.3% average F1), so fine-tuned models can filter out hallucinated relations that would introduce cycles into an ontology.
  • The same-as relation is hardest (86.3% average F1), identifying equivalence disambiguation as the next target for ontology-generation pipelines.
  • The authors propose extending the same pipeline to other academic disciplines as the direct next step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline 34-point gap may shrink if the 'other' labels are noisy: since 'semantically disjoint' is not formally defined, pairs connected by indirect MeSH paths could be mislabeled as unrelated, inflating F1 on that class and the overall average.
  • Fine-tuned models likely capture the distribution of the training sample, which is drawn from MeSH's explicit hierarchy; on novel or emerging topics not represented by such edges, prompt-based reasoning may transfer better than fine-tuning.
  • A testable extension would compare fine-tuning against retrieval-augmented prompting that supplies MeSH definitions; if prompted models then match fine-tuned F1, the advantage is knowledge access rather than reasoning capacity.
  • The observed same-as confusion suggests a lightweight lexical-overlap heuristic, applied before or after the classifier, could recover several of the remaining errors at negligible cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MeSH-Rel-4K, a dataset of 4,000 relation pairs sampled from MeSH, labeled as broader, narrower, same-as, or other. It evaluates five small open-source LLMs (up to 9B parameters) under three adaptation strategies: standard prompting, chain-of-thought two-way prompting, and LoRA fine-tuning. The central empirical claim is that fine-tuning improves average macro-F1 by 34.1 percentage points over CoT prompting, with fine-tuned models reaching 86.2–91.6% F1 across the test set. The authors also release code and the dataset and provide confusion-matrix-based error analysis.

Significance. If the empirical findings hold, the paper provides a strong demonstration that resource-efficient LLMs fine-tuned on a modest dataset can perform biomedical relationship classification at a level useful for ontology curation and evolution. The main strengths are the public release of dataset/code, the held-out test set with concept-level leakage control, the systematic comparison of prompting versus fine-tuning across multiple models, and the confusion-matrix analysis. However, the benchmark's negative-class construction and the absence of statistical replicates are important correctness risks that need to be addressed before the 34.1-point gain can be fully accepted.

major comments (3)
  1. [Section 3.2] The construction of the 'other' class is under-specified: the paper states only that it contains '1,000 pairs of semantically disjoint topics' and gives no algorithm or verification procedure. If 'disjoint' was implemented as 'no direct mesh:broaderDescriptor / mesh:relatedConcept edge,' then pairs connected through longer paths or missing MeSH annotations would be mislabeled as unrelated. Because fine-tuned models are trained on these labels, such false negatives could inflate the 'other' F1 (average 94.3% in Table 4) and, in turn, the headline 34.1-point fine-tuning gain over CoT prompting, which is not trained on the same label distribution. Please specify the exact sampling algorithm, report whether transitive closure or expert validation was used, and quantify the label-noise risk (e.g., by analyzing how many 'other' pairs have shared ancestors or are connected at distance ≥2).
  2. [Section 5.2 / Table 4] All reported results are point estimates from what appear to be single fine-tuning runs. There are no repeated runs, confidence intervals, or significance tests. Given that LoRA fine-tuning is stochastic and the central claim is the exact magnitude of the F1 improvement, the authors should report mean ± standard deviation over at least three independent runs (or provide a statistical significance test) for the fine-tuning experiments in Table 4. This is necessary to establish that the 34.1-point gap is not an artifact of a particular seed.
  3. [Section 3.3 / Table 1] The LoRA hyperparameter tuning is described only as 'multiple training iterations to empirically adjust' r and alpha. For reproducibility and to rule out selection bias, the paper should report the search space, the number of configurations tried per model, the validation metric used for selection, and whether the same validation set was used across models. Table 1 lists only the final r/alpha values.
minor comments (5)
  1. [Section 5.2] The text says average F1-scores range 'from 86.6% to 94.3%,' but Table 4 shows same-as at 86.3%. The lower bound should be 86.3%.
  2. [Section 4.1] The prompt templates and the 'empirical referee rules' are only described by reference to [15]. Since this paper is self-contained in other respects, include the templates and rules in an appendix or in the repository (beyond a link) so reviewers can verify the CoT setup.
  3. [Section 5.3] The claim that the same-as/hierarchical confusion 'correlates with dense lexical overlap' is supported only by a single example (alitretinoin/tretinoin). Quantify this correlation, e.g., by measuring token overlap or embedding similarity for misclassified versus correctly classified pairs.
  4. [Section 5.3 / Conclusion] The phrase 'near-state-of-the-art models' is not supported by any comparison with state-of-the-art methods or larger LLMs. Either remove or qualify the statement.
  5. [Abstract / Section 3.2] The dataset is described as containing '4K semantic relationships,' but the 'other' class consists of unrelated pairs. Recommend calling them 'relation pairs' or 'semantic relation instances' to avoid terminological confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central F1 results are held-out measurements on an externally sourced benchmark; the 'other'-class construction is a validity risk, not a circular step.

full rationale

The paper's central claim is an empirical result: fine-tuned models are measured on a held-out test split of MeSH-Rel-4K (§5.2, Tables 3–4). There is no derivation chain in which an output is constructed from an input by definition. The LoRA hyperparameters are tuned on training and validation data (§3.2, §4.2), but the reported F1 scores are computed on the 800-pair test set, so the 34.1-point fine-tuning gain is a measurement, not a reduction. The self-citations [15,54] supply the CoT two-way prompt template and the choice of relation types; these are methodological choices, not load-bearing evidence for the F1 numbers, which are new measurements against an independently sourced benchmark (MeSH). The only notable fragility is the construction of the 'other' class: §3.2 says '1,000 pairs of semantically disjoint topics' without specifying the algorithm, so if 'disjoint' means only 'no direct mesh:broaderDescriptor/relatedConcept edge,' some negatives could be connected via longer MeSH paths and the 'other' F1 may be optimistic. That is a benchmark-validity and correctness risk, not a circularity: the labels are external to the model outputs and are not fitted parameters that later reappear as predictions. The central claim therefore does not reduce to its inputs by construction; score 0.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no theoretical entities. Its load-bearing assumptions concern the correctness and representativeness of the MeSH-Rel-4K labels, the integrity of the train/test split, and the adequacy of the classification task as a proxy for ontology construction.

free parameters (1)
  • LoRA rank r and scaling factor alpha for each model = mistral-7b: r=16, alpha=16; llama-3b: r=256, alpha=128; gemma-9b: r=256, alpha=128; phi-3: r=256, alpha=128; zephyr-7b:
    Table 1 reports only the optimal configurations after 'multiple training iterations' (Section 3.3). These choices affect all fine-tuned results, and the selection procedure is not fully specified.
assumptions (5)
  • domain assumption MeSH's mesh:broaderDescriptor and mesh:relatedConcept define ground-truth broader/narrower/same-as relations.
    Used to construct all positive labels in MeSH-Rel-4K (Section 3.2). If these MeSH properties are noisy or incomplete, the labels are noisy.
  • domain assumption The 'other' pairs, described as 'semantically disjoint topics,' are truly unrelated.
    Section 3.2 does not specify the disjointness algorithm. Absence of a direct MeSH edge does not guarantee absence of any semantic relation, which could make the class artificially easy.
  • ad hoc to paper Four-way single-label classification is an adequate proxy for ontology generation and evolution.
    The paper's broader claim about automated ontology construction (Abstract, Section 6) rests on treating this classification task as representative of the full curation workload.
  • domain assumption LoRA hyperparameters were selected using the validation set rather than the test set.
    Section 3.3 mentions empirical adjustment and 'optimal configurations' without stating the data used for selection. If test-set feedback occurred, the reported F1 scores would be optimistically biased.
  • domain assumption The balanced 1,000-per-class sampling reflects realistic ontology-editing workloads.
    Real ontology updates are highly imbalanced; the equal class balance in MeSH-Rel-4K may make macro-F1 more favorable than in deployment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field." pith.science (2026). https://pith.science/paper/AKPFCODS

@misc{pith2026260717902,
  author       = {Pith},
  title        = {Pith review of: Benchmarking Resource-Efficient LLMs for Research Topic Ontology Generation in the Biomedical Field},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AKPFCODS}},
  note         = {Machine review of arXiv:2607.17902}
}
read the original abstract

Knowledge Organization Systems like Ontologies and taxonomies are fundamental for structuring scientific knowledge, yet their manual curation presents a persistent bottleneck in knowledge management. While Large Language Models (LLMs) offer a scalable mechanism for automated ontology generation, their capacity to classify complex, domain-specific semantics requires systematic evaluation. In this paper, we assess the performance of five small, open-source LLMs (up to 9 billion parameters) in identifying semantic relationships between biomedical concepts. To support this evaluation, we introduce MeSH-Rel-4K, a dataset comprising 4K semantic relationships extracted from the Medical Subject Headings (MeSH). We analyse three adaptation strategies: standard prompting, Chain-of-Thought prompting, and fine-tuning. While parameter-constrained models traditionally struggle with the nuances of in-context logic, our results reveal that targeted fine-tuning increases the average F1-score by 34.1 percentage points. These results confirm that direct fine-tuning effectively exceeds the reasoning bottlenecks of smaller LLMs, providing an accurate, automated methodology for the construction and evolution of specialised biomedical ontologies.

Figures

Figures reproduced from arXiv: 2607.17902 by the authors.

Figure 1
Figure 1. Confusion matrices for LLMs fine-tuned and evaluated on MeSH-Rel-4K inversely with model parameter count. While gemma-9b minimises this confusion, the smallest model (llama-3b, see Fig. 1e) frequently predicts false hierarchical relations, misclassifying 40 same-as instances as broader (22) and narrower (18). The mid-sized models (zephyr-7b, mistral-7b, and phi-3) exhibit symmetric confusion, misclassifying hierarch… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 1 canonical work pages

  1. [1]

    Salatino, T

    A. Salatino, T. Aggarwal, A. Mannocci, F. Osborne, E. Motta, A survey of knowledge organization systems of research fields: Resources and challenges, Quantitative Science Studies 6 (2025) 567–610

    2025

  2. [2]

    Dunne, K

    E. Dunne, K. Hulek, Mathematics subject classification 2020, EMS Newsletter 2020–3 (2020) 5–6. URL: http://dx.doi.org/10.4171/NEWS/115/2. doi:10.4171/news/115/2

    work page doi:10.4171/news/115/2 2020

  3. [3]

    C. E. Lipscomb, Medical subject headings (mesh), Bulletin of the Medical Library Association 88 (2000) 265

    2000

  4. [4]

    Rous, Major update to acm’s computing classification system, Communications of the ACM 55 (2012) 12–12

    B. Rous, Major update to acm’s computing classification system, Communications of the ACM 55 (2012) 12–12

    2012

  5. [5]

    Palumbo, D

    E. Palumbo, D. Monti, G. Rizzo, R. Troncy, E. Baralis, entity2rec: Property-specific knowledge graph embeddings for item recommendation, Expert Systems with Applications 151 (2020) 113235

    2020

  6. [6]

    Cadeddu, A

    A. Cadeddu, A. Chessa, V. De Leo, G. Fenu, E. Motta, F. Osborne, D. R. Recupero, A. Salatino, L. Secchi, A comparative analysis of knowledge injection strategies for large language models in the scholarly domain, Engineering Applications of Artificial Intelligence 133 (2024) 108166

    2024

  7. [7]

    Gusenbauer, N

    M. Gusenbauer, N. R. Haddaway, Which academic search systems are suitable for systematic reviews or meta-analyses? evaluating retrieval qualities of google scholar, pubmed, and 26 other resources, Research synthesis methods 11 (2020) 181–217

    2020

  8. [8]

    Meloni, S

    A. Meloni, S. Angioni, A. Salatino, F. Osborne, D. R. Recupero, E. Motta, Integrating conversational agents and knowledge graphs within the scholarly domain, Ieee Access 11 (2023) 22468–22489

    2023

Show all 57 references
  1. [9]

    Angioni, A

    S. Angioni, A. Salatino, F. Osborne, D. R. Recupero, E. Motta, Aida: A knowledge graph about research dynamics in academia and industry, Quantitative Science Studies 2 (2021) 1356–1398. 17Metadata: Title, Abstract, Keywords, Introduction, and Conclusion

    2021

  2. [10]

    Osborne, E

    F. Osborne, E. Motta, Klink-2: integrating multiple web sources to generate semantic topic networks, in: The Semantic Web-ISWC 2015: 14th International Semantic Web Conference, Bethlehem, PA, USA, October 11-15, 2015, Proceedings, Part I 14, Springer, 2015, pp. 408–424

    2015

  3. [11]

    K. Han, P. Yang, S. Mishra, J. Diesner, Wikicssh: extracting computer science subject headings from wikipedia, in: ADBIS, TPDL and EDA 2020 Common Workshops and Doctoral Consortium: International Workshops: DOING, MADEISD, SKG, BBIGAP, SIMPDA, AIMinScience 2020 and Doctoral Co...

    2020

  4. [12]

    Osborne, E

    F. Osborne, E. Motta, Mining semantic relations between research areas, in: The Semantic Web– ISWC 2012: 11th International Semantic Web Conference, Boston, MA, USA, November 11-15, 2012, Proceedings, Part I 11, Springer, 2012, pp. 410–426

    2012

  5. [13]

    Sanderson, B

    M. Sanderson, B. Croft, Deriving concept hierarchies from text, in: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, 1999, pp. 206–213

    1999

  6. [14]

    Babaei Giglou, J

    H. Babaei Giglou, J. D’Souza, S. Auer, Llms4ol: Large language models for ontology learning, in: International Semantic Web Conference, Springer, 2023, pp. 408–427

    2023

  7. [15]

    Aggarwal, A

    T. Aggarwal, A. Salatino, F. Osborne, E. Motta, Large language models for scholarly ontology generation: An extensive analysis in the engineering field, Information Processing & Management 63 (2026) 104262

    2026

  8. [16]

    A. S. Lippolis, M. J. Saeedizade, R. Keskisärkkä, S. Zuppiroli, M. Ceriani, A. Gangemi, E. Blomqvist, A. G. Nuzzolese, Ontology generation using large language models, arXiv preprint arXiv:2503.05388 (2025)

    2025 arXiv

  9. [17]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, D. Zhou, Chain-of-thought prompting elicits reasoning in large language models, 2023. URL: https://arxiv.org/abs/2201.11903. arXiv:2201.11903

    2023 arXiv

  10. [18]

    Mangrulkar, S

    S. Mangrulkar, S. Gugger, L. Debut, Y. Belkada, S. Paul, B. Bossan, Peft: State-of-the-art parameter- efficient fine-tuning methods, https://github.com/huggingface/peft, 2022

    2022

  11. [19]

    M. L. Zeng, Knowledge organization systems (kos), KO Knowledge Organization 35 (2008) 160–182

    2008

  12. [20]

    D. Lee, N. de Keizer, F. Lau, R. Cornet, Literature review of snomed ct use, Journal of the American Medical Informatics Association 21 (2014) e11–e19

    2014

  13. [21]

    Ashburner, C

    M. Ashburner, C. A. Ball, J. A. Blake, D. Botstein, H. Butler, J. M. Cherry, A. P. Davis, K. Dolinski, S. S. Dwight, J. T. Eppig, et al., Gene ontology: tool for the unification of biology, Nature genetics 25 (2000) 25–29

    2000

  14. [22]

    A. A. Salatino, T. Thanapalasingam, A. Mannocci, F. Osborne, E. Motta, The computer science ontology: a large-scale taxonomy of research areas, in: The Semantic Web–ISWC 2018: 17th International Semantic Web Conference, Monterey, CA, USA, October 8–12, 2018, Proceedings, Part ...

    2018

  15. [23]

    Smith, Physics subject headings (physh), KO KNOWLEDGE ORGANIZATION 47 (2020) 257–266

    A. Smith, Physics subject headings (physh), KO KNOWLEDGE ORGANIZATION 47 (2020) 257–266

    2020

  16. [24]

    Chang, J

    E. Chang, J. Mostafa, The use of snomed ct, 2013-2020: a literature review, Journal of the American Medical Informatics Association 28 (2021) 2017–2026

    2013

  17. [25]

    G. O. Consortium, The gene ontology resource: 20 years and still going strong, Nucleic acids research 47 (2019) D330–D338

    2019

  18. [26]

    OpenAlex, Openalex: End-to-end process for topic classification, 2024

    2024

  19. [27]

    Maedche, S

    A. Maedche, S. Staab, Learning ontologies for the semantic web., in: SemWeb, 2001

    2001

  20. [28]

    Cimiano, J

    P. Cimiano, J. Völker, Text2onto: A framework for ontology learning and data-driven change discovery, in: International conference on application of natural language to information systems, Springer, 2005, pp. 227–238

    2005

  21. [29]

    Velardi, S

    P. Velardi, S. Faralli, R. Navigli, Ontolearn reloaded: A graph-based algorithm for taxonomy induction, Computational Linguistics 39 (2013) 665–707

    2013

  22. [30]

    C. Chen, K. Lin, D. Klein, Constructing taxonomies from pretrained language models, arXiv preprint arXiv:2010.12813 (2020)

    2010 arXiv

  23. [31]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.arXiv:1810.04805

    2019 arXiv

  24. [32]

    Grootendorst, Bertopic: Neural topic modeling with a class-based tf-idf procedure, arXiv preprint arXiv:2203.05794 (2022)

    M. Grootendorst, Bertopic: Neural topic modeling with a class-based tf-idf procedure, arXiv preprint arXiv:2203.05794 (2022)

    2022 arXiv

  25. [33]

    Fathallah, A

    N. Fathallah, A. Das, S. D. Giorgis, A. Poltronieri, P. Haase, L. Kovriguina, Neon-gpt: a large language model-powered pipeline for ontology learning, in: European Semantic Web Conference, Springer, 2024, pp. 36–50

    2024

  26. [34]

    Fathallah, S

    N. Fathallah, S. Staab, A. Algergawy, Llms4life: Large language models for ontology learning in life sciences, 2024. URL: https://arxiv.org/abs/2412.02035.arXiv:2412.02035

    2024 arXiv

  27. [35]

    M. J. Saeedizade, E. Blomqvist, Navigating ontology development with large language models, in: European Semantic Web Conference, Springer, 2024, pp. 143–161

    2024

  28. [36]

    Tsaneva, D

    S. Tsaneva, D. Dessì, F. Osborne, M. Sabou, Knowledge graph validation by integrating llms and human-in-the-loop, Information Processing & Management 62 (2025) 104145

    2025

  29. [37]

    Y. Sun, H. Xin, K. Sun, Y. E. Xu, X. Yang, X. L. Dong, N. Tang, L. Chen, Are large language models a good replacement of taxonomies?, arXiv preprint arXiv:2406.11131 (2024)

    2024 arXiv

  30. [38]

    K. Wang, Z. Shen, C. Huang, C.-H. Wu, Y. Dong, A. Kanakia, Microsoft academic graph: When experts are not enough, Quantitative Science Studies 1 (2020) 396–413

    2020

  31. [39]

    G. B. Jenset, P. J. Bevan, A. Jain, A large-scale, granular topic classification system for scientific documents (2025)

    2025

  32. [40]

    K. I. Kotis, G. A. Vouros, D. Spiliotopoulos, Ontology engineering methodologies for the evolution of living and reused ontologies: status, trends, findings and recommendations, The Knowledge Engineering Review 35 (2020) e4

    2020

  33. [41]

    Osborne, E

    F. Osborne, E. Motta, Pragmatic ontology evolution: reconciling user requirements and application performance, in: International Semantic Web Conference, Springer, 2018, pp. 495–512

    2018

  34. [42]

    S. Chow, L. Guo, J. Chow, C. Chia, S. Li, D.-Y. Huang, Semantic search using llm-aided topic generation on knowledge graphs for paper discovery, in: 2024 IEEE 14th International Symposium on Chinese Spoken Language Processing (ISCSLP), IEEE, 2024, pp. 353–357

    2024

  35. [43]

    Buscaldi, D

    D. Buscaldi, D. Dessí, E. Motta, M. Murgia, F. Osborne, D. R. Recupero, Citation prediction by leveraging transformers and natural language processing heuristics, Information Processing & Management 61 (2024) 103583

    2024

  36. [44]

    S. Auer, D. A. Barone, C. Bartz, E. G. Cortes, M. Y. Jaradeh, O. Karras, M. Koubarakis, D. Mouromtsev, D. Pliukhin, D. Radyush, et al., The sciqa scientific question answering benchmark for scholarly knowledge, Scientific Reports 13 (2023) 7240

    2023

  37. [45]

    Bolanos, A

    F. Bolanos, A. Salatino, F. Osborne, E. Motta, Artificial intelligence for literature reviews: Opportu- nities and challenges, Artificial Intelligence Review 57 (2024) 259

    2024

  38. [46]

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al., Mistral 7b, arXiv preprint arXiv:2310.06825 (2023)

    2023 arXiv

  39. [47]

    URL: https://github.com/meta-llama/llama3/blob/main/ MODEL_CARD.md

    AI@Meta, Llama 3 model card (2024). URL: https://github.com/meta-llama/llama3/blob/main/ MODEL_CARD.md

    2024

  40. [48]

    G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivière, M. S. Kale, J. Love, et al., Gemma: Open models based on gemini research and technology, arXiv preprint arXiv:2403.08295 (2024)

    2024 arXiv

  41. [49]

    Abdin, J

    M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari, J. Bao, H. Behl, et al., Phi-3 technical report: A highly capable language model locally on your phone, 2024, URL https://arxiv. org/abs/2404.14219 (2024)

    2024 arXiv

  42. [50]

    Han, Unsloth/zephyr-sft·hugging face, 2024

    D. Han, Unsloth/zephyr-sft·hugging face, 2024. URL: https://huggingface.co/unsloth/zephyr-sft

    2024

  43. [51]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, Lora: Low-rank adaptation of large language models. arxiv 2021, arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  44. [52]

    LoRA fine-tuning Hyperparameters Guide | Unsloth Documentation — unsloth.ai, https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/lora-hyperparameters-guide# lora-alpha-and-rank-relationship, 2025

    2025

  45. [53]

    Kojima, S

    T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, Y. Iwasawa, Large language models are zero-shot reasoners,

  46. [54]

    Aggarwal, A

    T. Aggarwal, A. Salatino, F. Osborne, E. Motta, et al., Identifying semantic relationships between research topics using large language models in a zero-shot learning setting, in: CEUR Workshop Proceedings, volume 3780, CEUR-WS, 2024

    2024

  47. [55]

    M. H. Daniel Han, U. team, Unsloth, 2023. URL: https://github.com/unslothai/unsloth

    2023

  48. [56]

    T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Fun- towicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, A. M. Rush, Transformers: State-of-the-art natural ...

    2020

  49. [2023]

    URL: https://arxiv.org/abs/2205.11916.arXiv:2205.11916

    arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.