Pith. sign in

REVIEW 5 major objections 4 minor 59 references

BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that BioPars, presented as the first Persian biomedical large language model, outperforms GPT-4, Llama, and Galactica on Persian medical question answering, reaching ROUGE-L 29.99 on its own 5,231-question…

desk verdict A real Persian medical QA dataset and a coherent architecture, but the central claim that BioPars is a trained model is unverifiable and the evaluation tables contain errors. read the letter →

arxiv 2506.21567 v2 pith:M7C6U6DF submitted 2025-06-12 cs.CL cs.AIcs.LG

Baqer M. Merzah , Tania Taami , Salman Asoudeh , Saeed Mirzaee , Amir reza Hossein pour , Amir Ali Bengari This is my paper
classification cs.CLcs.AIcs.LG
keywords PersianbiomedicalNLPlargelanguagemodelmedicalquestionansweringBioParsQAbenchmarkgatedattentionexponentialmovingaveragedomain-specificpretraininglow-resourcelanguages
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that Persian biomedical question answering cannot be handled well by general-purpose LLMs and needs a dedicated, language-specific model — and that such a model can in fact be built and measured. To make that case it introduces three artifacts: BIOPARS-BENCH, a Persian biomedical corpus drawn from over 10,000 scientific articles, textbooks, and medical websites; BioParsQA, 5,231 Persian medical question-answer pairs vetted by at least two specialist doctors; and BioPars, a pretrained Persian biomedical model whose attention is guided by a damped exponential moving average. Its central empirical claim is that BioPars outperforms ChatGPT (GPT-4), Llama 2-13B, and Galactica 30B on BioParsQA, reaching ROUGE-L 29.99 (1.0 over GPT-4), BERTScore 90.87, MoverScore 60.43, and BLEURT 50.78, while also posting the best numbers on three of the four English medical QA sets tested. If the claim holds, the practical consequence is that a purpose-built Persian model beats far larger general systems at generating usable medical answers, and the paper positions BioPars as the first Persian biomedical LLM for long-form medical question answering.

What carries the argument

The object that carries the argument is a transformer block whose self-attention is guided by a damped exponential moving average, extended to the complex plane as C-EMA: each input channel is lifted to an $h$-dimensional latent space, propagated through a complex-valued decay $h^j_t = \alpha^j(\cos\theta^j + i\sin\theta^j) u^j_t + (1-\alpha^j\delta^j)(\cos\theta^j + i\sin\theta^j) h^j_{t-1}$, and projected back to the token width. Around that core sit a causal TimestepNorm that stabilizes activations without revealing future tokens, a normalized attention that derives queries and keys from an $\ell^2$-normalized shared representation, a pre-norm two-hop residual that routes the feed-forward branch back to the block input, and chunk-wise attention plus parallelism so that only the final C-EMA state crosses devices during pretraining. The claimed role of this machinery is to give long-context, autoregressive stability during pretraining on Persian biomedical corpora, so the evaluation numbers are presented as the joint payoff of the architecture and the domain corpus.

What would settle it

Ask for the released weights and the exact pretraining and evaluation configuration, then rerun the BioParsQA evaluation on a split from which every reference answer has been confirmed absent from the training data; if the reported ROUGE-L 29.99 and BERTScore 90.87 cannot be reproduced, or if a contamination check finds BioParsQA texts inside the pretraining corpus, the claimed margin over GPT-4 is not established.

Watch

Extended reading notes

Core claim

The paper's central claim is that BioPars is the first Persian biomedical large language model, and that its pretraining on Persian biomedical text makes it better than larger or more general models at answering real Persian medical questions in free-form prose. On its own benchmark, BioParsQA, the authors report the best scores among the four tested systems on all four metrics — ROUGE-L 29.99 against 28.99 for GPT-4, BERTScore 90.87 under the MMR ranking method, MoverScore 60.43, and BLEURT 50.78 — with the advantage appearing where the model must generate an answer rather than pick one. The same model records the top ROUGE-L on LiveQA, MedicationQA, and the ExpertQA-Med subset of the English-language sets, while trailing GPT-4 and Galactica on ExpertQA-Bio. From the pattern that ChatGPT, Llama, and Galactica retrieve learned facts well but stumble on higher-level, real-world questions and fine-grained inferences, the paper concludes that targeted pretraining on authentic Persian biomedical data is necessary for viable applications, rather than relying on zero-shot or generic capabilities.

Load-bearing premise

The result depends on BioPars having actually been trained on Persian biomedical text and then tested on questions it had never seen, yet the paper provides no training setup, data splits, model size, or leakage controls.

Editorial extensions

If this is right

  • Persian medical question answering can be served by a dedicated, smaller model rather than a closed commercial API, with the reported numbers showing the largest gains where answers must be generated sentence by sentence rather than selected.
  • BioParsQA supplies the Persian biomedical community with a fixed, doctor-approved evaluation set of 5,231 question-answer pairs, so future Persian medical models can be compared on identical ground.
  • The reported shortfall of ChatGPT, Llama, and Galactica on higher-level and fine-grained questions is taken as evidence that zero-shot prompting of general LLMs is insufficient for Persian biomedical applications.
  • The architecture recipe — complex-domain damped EMA, TimestepNorm, and chunked normalized attention — is claimed to be a stable base for long-context autoregressive pretraining.
  • Because BioPars also posts the best scores on LiveQA and MedicationQA in the paper's tables, the authors' evidence suggests the gains are not confined to Persian.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the margin that matters is small — 1.0 ROUGE-L over GPT-4 — so the paper's strongest possible reading survives only if the benchmark is clean; the authors leave the leakage test implicit, and that test is the first thing an independent reader should run.
  • My inference: if the architecture, not just the corpus, is responsible for the gains, the gated-EMA block should transfer to other low-resource medical languages; a cheap experiment would be to pretrain the same block on Arabic or Urdu medical text and check BioParsQA-style benchmarks.
  • My inference: the paper's framing that general models 'remember' but cannot 'infer' is testable directionally — a diagnostic split of BioParsQA into fact-retrieval versus multi-step reasoning questions would show where the 29.99 ROUGE-L actually comes from.
  • My inference: the authors' own strongest unmet promise is clinical safety; a natural extension they mention but do not attempt is routing generated answers through the same two-doctor approval protocol used to build the dataset.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces three artifacts: BIOPARS-BENCH, a Persian biomedical corpus compiled from roughly 10,000 articles, textbooks, and websites; BioParsQA, a self-constructed set of 5,231 Persian medical question-answer pairs; and BioPars, which is described in the abstract and introduction as a benchmark or 'measure' but in Section 5 and the conclusion as a pretrained Persian biomedical large language model with a gated attention architecture based on damped exponential moving averages, C-EMA, TimestepNorm, normalized attention, and a pre-norm two-hop residual. The evaluation compares BioPars against GPT-4, LLaMA2-13b, and Galactica 30B on LiveQA, ExpertQA-Bio, ExpertQA-Med, MedicationQA, and BioParsQA using ROUGE-L, BERTScore, MoverScore, and BLEURT, reporting that BioPars achieves the best scores on several datasets, most notably ROUGE-L 29.99 on BioParsQA versus 28.99 for GPT-4.

Significance. If substantiated, a pretrained Persian biomedical LLM together with a Persian biomedical benchmark would fill a genuine gap in low-resource medical NLP, and BIOPARS-BENCH and BioParsQA could become useful community resources. The inclusion of external English medical QA datasets is also a positive partial check on the model's behavior. However, the manuscript does not establish that the claimed pretrained model exists as described: there is no training description, no data-split or leakage-control protocol, and the evaluation tables contain internally inconsistent values. The current strengths are therefore limited to a resource proposal; the central model and performance claims are not verifiable from the manuscript as written.

major comments (5)
  1. [§5, §8] The central claim that BioPars is a pretrained biomedical LLM is not supported by any training description. Section 5.1–5.4 presents architectural equations (Eqs. 1–12) but gives no training objective, token budget, model size, optimizer, learning-rate schedule, hardware, or procedure connecting the BIOPARS-BENCH corpus of Section 3 to pretraining. Section 5.3.1 explicitly lists 'Unknown Scalability' and states that stability at billion-parameter scale has not been tested, while Section 5.4.4 refers to 'even at 7 B parameters.' The conclusion's assertion that BioPars 'implements the entire training pipeline from pre-training to final testing' is therefore unsupported, and the paper's main claim cannot be independently checked.
  2. [§4, Table 7] The headline result is measured on BioParsQA, a dataset constructed by the same authors, yet Section 4 gives no train/test split, no deduplication against the pretraining corpus, no statement about whether BioParsQA was used for training, and no leakage-control procedure. Because BioPars's best ROUGE-L score is reported on this self-built dataset, the apparent improvement over GPT-4 may reflect overlap between training and evaluation data rather than domain-adaptive ability. The manuscript must state whether BioParsQA was seen during training and provide a split or contamination analysis before the BioParsQA comparison can be interpreted.
  3. [Table 7] Table 7 reports a ROUGE-L of 88.52 for LLaMA2-13b under the Sim method, while the same model's other ROUGE-L entries are between 27 and 28 and the competing models are in the 27–29 range. No explanation is offered for this value, and it is so far outside the range of the other rows that it cannot be a simple metric fluctuation. An unchecked or mislabeled value in the central comparison table undermines confidence in all the reported numbers.
  4. [Abstract, §1, §5] The object being evaluated is described inconsistently: the abstract calls BioPars 'a simple but accurate measure designed to assess LLMs,' Section 1 calls it an 'open but powerful testing platform,' while Section 5 and Section 8 describe it as a neural model with parameters and pretraining. These are incompatible descriptions. If BioPars is a benchmark or metric, comparing it against GPT-4 and Galactica in Section 7 is not meaningful; if it is a model, the paper needs to specify what was actually implemented and evaluated.
  5. [§7, Tables 3–6] The evaluation protocol for the external English datasets is not described. LiveQA, ExpertQA-Bio, ExpertQA-Med, and MedicationQA are all non-Persian datasets, but the manuscript does not state whether the questions were translated into Persian, whether BioPars was run in Persian or English, what prompting or few-shot settings were used, or what decoding parameters were applied. Without this information, the comparative results in Tables 3–6 cannot be reproduced or interpreted, and the claim that BioPars outperforms other models on these datasets is not verifiable.
minor comments (4)
  1. [Table 5] The caption of Table 5 says 'Automatic evaluation scores on ExpertQA-Bio,' but the table body, the surrounding text, and Figure 5 all refer to ExpertQA-Med; the caption should be corrected.
  2. [§7, MedicationQA paragraph] The text says BioPars achieved 'a maximum ERTScore=83.90'; this appears to be a typo for BERTScore, and the reported BERTScore values are expressed as percentages in the tables but as fractions (e.g., 'above 0.88') in the prose, so the unit should be stated consistently.
  3. [§4] The Mifepristone example states that the drug 'is usually provided to treat high blood sugar,' which is clinically inaccurate; since the paper is in the biomedical domain, such an error in a dataset example raises concerns about the curation quality and should be corrected or removed.
  4. [Abstract] The phrase 'an improvement over GPT-4 1.0' is unclear; it should be phrased as an improvement of 1.00 ROUGE-L points, and the relationship between the reported BioParsQA scores and the corresponding rows of Table 7 should be stated precisely.

Circularity Check

0 steps flagged · score 0.0 of 10

No demonstrated circularity; the paper's main failures are missing training and leakage controls, not an equation-level reduction of prediction to input.

full rationale

The paper does not exhibit a circular derivation that meets the evidentiary bar of this review. The central claim that BioPars is a pretrained Persian biomedical LLM is asserted, but the manuscript never describes a training objective, token budget, model size, optimizer, or procedure linking BIOPARS-BENCH to pretraining, and Section 5.3.1 explicitly lists 'Unknown Scalability: Stability in multi-million token and billion-parameter contexts has not yet been tested.' These are severe verifiability and reproducibility failures, not circular reductions. No parameter is fitted and then reported as a prediction, and no uniqueness theorem or load-bearing self-citation is invoked. The architecture equations in Section 5 are definitions, not fitted quantities. The evaluation on the self-built BioParsQA is concerning because no train/test split, deduplication against the pretraining corpus, or leakage control is reported; however, the paper never states that BioParsQA was used for training or fine-tuning, and the comparison also includes external datasets (LiveQA, ExpertQA, MedicationQA). Thus the headline ROUGE-L/BERTScore gains on BioParsQA are unverified and potentially memorization-based, but they do not reduce by construction to the paper's own stated inputs. Additional problems, such as the internally inconsistent LLaMA2-13b ROUGE-L value of 88.52 in Table 7 and the mislabeled caption in Table 5, are correctness and auditing concerns rather than circularity. Accordingly, no circular step can be quoted and exhibited, and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 4 invented entities

The central claim rests on unverified assumptions: the BioPars model exists and was trained, BioParsQA answers are correct, and automatic metrics are valid for Persian. No free parameters are fit in the paper, but the absence of any training or evaluation protocol makes these assumptions load-bearing.

assumptions (4)
  • domain assumption The BioParsQA dataset contains correct, expert-approved answers to Persian medical questions.
    The authors state at least two specialist doctors approved the Q&A pairs (Section 4), but no verification, inter-annotator agreement, or source records are provided.
  • domain assumption Automatic metrics (ROUGE, BERTScore, MoverScore, BLEURT) are valid for evaluating Persian medical long-form answers.
    These metrics were developed for English; the paper provides no validation or calibration for Persian, which can distort tokenization and semantic matching.
  • ad hoc to paper BioPars, the model described in Section 5, was actually implemented, pretrained on Persian biomedical data, and used to generate the evaluation outputs.
    The paper gives a high-level architecture but no train/test setup, compute, hyperparameters, or model weights, making the existence of the evaluated model an unsupported assumption.
  • domain assumption The compared baseline models (GPT-4, LLaMA2-13b, Galactica 30B) were evaluated under fair, comparable conditions.
    No prompting template, decoding parameters, or API settings are reported for any model, so fairness of comparison is assumed.
invented entities (4)
  • BioPars pretrained model
    purpose: Central claimed contribution, a Persian biomedical LLM for medical QA.
    Paper provides no training details, weights, or reproducible artifacts, so its existence is unverified.
  • BioParsQA dataset
    purpose: Evaluation benchmark for Persian medical QA.
    No download link or data sample is given in the paper, and the GitHub link is unverified.
  • BIOPARS-BENCH corpus
    purpose: Pretraining corpus from Persian medical sources.
    No corpus release or preprocessing details; called a benchmark but never used in evaluation.
  • C-EMA and TimestepNorm architecture components
    purpose: Proposed improvements for long-context training and stability.
    Not compared against existing EMA/state-space models, and no ablations are shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining." pith.science (2026). https://pith.science/paper/M7C6U6DF

@misc{pith2026250621567,
  author       = {Pith},
  title        = {Pith review of: BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M7C6U6DF}},
  note         = {Machine review of arXiv:2506.21567}
}
read the original abstract

Large Language Models (LLMs) have recently gained attention in the life sciences due to their capacity to model, extract, and apply complex biological information. Beyond their classical use as chatbots, these systems are increasingly used for complex analysis and problem-solving in specialized fields, including bioinformatics. First, we introduce BIOPARS-BENCH, a dataset from over 10,000 scientific articles, textbooks, and medical websites. BioParsQA was also introduced to evaluate the proposed model, which consists of 5,231 Persian medical questions and answers. This study then introduces BioPars, a simple but accurate measure designed to assess LLMs for three main abilities: acquiring subject-specific knowledge, interpreting and synthesizing such knowledge, and demonstrating proper evidence. Comparing ChatGPT, Llama, and Galactica, our study highlights their ability to remember and retrieve learned knowledge but also reveals shortcomings in addressing higher-level, real-world questions and fine-grained inferences. These findings indicate the need for further fine-tuning to address the capabilities of LLM in bioinformatics tasks. To our knowledge, BioPars is the first application of LLM in Persian medical QA, especially for generating long answers. Evaluation of four selected medical QA datasets shows that BioPars has achieved remarkable results compared to comparative approaches. The model on BioParsQA achieved a ROUGE-L score of 29.99, which is an improvement over GPT-4 1.0. The model achieved a BERTScore of 90.87 with the MMR method. The MoverScore and BLEURT values were also higher in this model than the other three models. In addition, the reported scores for the model are MoverScore=60.43 and BLEURT=50.78. BioPars is an ongoing project and all resources related to its development will be made available via the following GitHub repository: https://github.com/amirap80/BioPars.

Figures

Figures reproduced from arXiv: 2506.21567 by the authors.

Figure 1
Figure 1. Statistical analysis of the research data set, 8/24 [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. A sample of questions and answers provided in BioparsQA. 9/24 [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Comparative bar plot of different approaches on the LiveQA . 16/24 [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: comparative bar plot of different approaches on the ExpertQA-Bio. The evaluation of the models on the ExpertQA-Bio dataset is also shown in [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Comparative bar plot of different approaches on the ExpertQA-Med. achieved maximum ROUGE-L of 26.49 and 27.97, respectively. In terms of BERTScore, the GPT-4 model also recorded better results. The GPT-4 model also achieved a maximum MoverScore = 57.32 and BLEURT = 47.…
Figure 6
Figure 6. Figure 6: comparative bar plot of different approaches on the MedicationQA. achieved ROUGE-L=18.14, which was a 1.19 improvement over GPT-4. This model achieved a maximum ERTScore=83.90 using the RR method. 54.32 and 39.87 were recorded in the maximum mode for MoverScore and BLE…
Figure 7
Figure 7. Figure 7: The comparative bar plot of different approaches on the BioParsQA. BERTscore=90.87 with the MMR method. The MoverScore and BLEURT values were also higher than the other three models. The model achieved MoverScore=60.43 and BLEURT=50.78. The comparative bar plot of diff…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 35 canonical work pages

  1. [1]

    Hendrycks, D. et al. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020)

    arXiv 2020

  2. [2]

    Luo, H. et al. Bigbench: A unified benchmark for social bias in text-to-image generative models based on multi-modal llm. arXiv preprint arXiv:2407.15240 (2024)

    arXiv 2024

  3. [3]

    C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models

    Huang, Y .et al. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. Adv. Neural Inf. Process. Syst. 36, 62991–63010 (2023)

    work page 2023

  4. [4]

    Deng, C. et al. K2: A foundation language model for geoscience knowledge understanding and utilization. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, 161–170 (2024)

    work page 2024

  5. [5]

    Demner-Fushman, D. et al. Preparing a collection of radiology examinations for distribution and retrieval. J. Am. Med. Informatics Assoc. 23, 304–310 (2016)

    work page 2016

  6. [6]

    Bi, Z. et al. Oceangpt: A large language model for ocean science tasks. arXiv preprint arXiv:2310.02031 (2023)

    arXiv 2023

  7. [7]

    Chen, M. et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    arXiv 2021

  8. [8]

    Singhal, K. et al. Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138 (2022)

    arXiv 2022

Show all 59 references
  1. [9]

    Benchmarking medical large language models

    Bakhshandeh, S. Benchmarking medical large language models. Nat. Rev. Bioeng. 1, 543–543 (2023)

    2023

  2. [10]

    Zhang, S. et al. Applications of transformer-based language models in bioinformatics: a survey. Bioinforma. Adv. 3, vbad001 (2023). 21/24

    2023

  3. [11]

    Applications of natural language models in bioinformatics: A literature review

    Buzatu, R. Applications of natural language models in bioinformatics: A literature review. Utrecht Univ. Stud. Theses Repos. (2023)

    2023

  4. [12]

    R., Denny, P., Luxton-Reilly, A., Payne, S

    Piccolo, S. R., Denny, P., Luxton-Reilly, A., Payne, S. H. & Ridge, P. G. Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLOS Comput. Biol. 19, e1011511 (2023)

    2023

  5. [13]

    M., Ahmad, F

    Li, Y ., Wehbe, R. M., Ahmad, F. S., Wang, H. & Luo, Y . A comparative study of pretrained language models for long clinical text. J. Am. Med. Informatics Assoc. 30, 340–347 (2023)

    2023

  6. [14]

    & Deng, C

    Yang, Y ., Zhao, N., Ma, T., Yuan, Z. & Deng, C. ‘paperdemic’during the covid-19 pandemic. Eur. journal internal medicine 108, 111–113 (2023)

    2023

  7. [15]

    & Singh, H

    Jha, K., Saha, S. & Singh, H. Prediction of protein–protein interaction using graph neural networks. Sci. Reports 12, 8360 (2022)

    2022

  8. [16]

    & Sun, M

    Zeng, Z., Yao, Y ., Liu, Z. & Sun, M. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals. Nat. communications 13, 862 (2022)

    2022

  9. [17]

    & Luo, L

    Luo, H., Shan, W., Chen, C., Ding, P. & Luo, L. Improving language model of human genome for dna–protein binding prediction based on task-specific pre-training. Interdiscip. Sci. Comput. Life Sci. 15, 32–43 (2023)

    2023

  10. [18]

    Le, N. Q. K. Leveraging transformers-based language models in proteome bioinformatics. Proteomics 23, 2300011 (2023)

    2023

  11. [19]

    & Rives, A

    Rao, R., Meier, J., Sercu, T., Ovchinnikov, S. & Rives, A. Transformer protein language models are unsupervised structure learners. Biorxiv 2020–12 (2020)

    2020

  12. [20]

    & Gleghorn, J

    Hallee, L. & Gleghorn, J. P. Protein-protein interaction prediction is achievable with large language models. bioRxiv 2023–06 (2023)

    2023

  13. [21]

    & Barati Farimani, A

    Guntuboina, C., Das, A., Mollaei, P., Kim, S. & Barati Farimani, A. Peptidebert: A language model based on transformers for peptide property prediction. The J. Phys. Chem. Lett. 14, 10427–10434 (2023)

    2023

  14. [22]

    Science, medicine, and the future: Bioinformatics

    Bayat, A. Science, medicine, and the future: Bioinformatics. BMJ (Clinical research ed.) 324, 1018–1022 (2002)

    2002

  15. [23]

    & Irankhah, E

    Farhadi Nia, M., Ahmadi, M. & Irankhah, E. Transforming dental diagnostics with artificial intelligence: advanced integration of chatgpt and large language models for patient care. Front. Dental Medicine (2025)

    2025

  16. [24]

    Norcéide, F. S. et al. Positional tracking of physical objects in an augmented reality environment using neuromorphic vision sensors. 2024 Int. Conf. on Mach. Learn. Appl. (ICMLA) (2024)

    2024

  17. [25]

    Nia, M. F.et al. Experiential learning for interdisciplinary education on vestibular system models. ASEE annual conference exposition (2023)

    2023

  18. [26]

    Explore cross-codec quality-rate convex hulls relation for adaptive streaming

    Farhadi Nia, M. Explore cross-codec quality-rate convex hulls relation for adaptive streaming. arXiv preprint arXiv:2408.09044 (2024)

    2024 arXiv

  19. [27]

    Pollard, T. J. et al. The eicu collaborative research database, a freely available multi-center database for critical care research. Sci. data 5, 1–13 (2018)

    2018

  20. [28]

    Zeng, G. et al. Meddialog: Large-scale medical dialogue datasets. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 9241–9250 (2020)

    2020

  21. [29]

    & Liu, S

    Zhang, S., Zhang, X., Wang, H., Guo, L. & Liu, S. Multi-scale attentive interaction networks for chinese medical question answer selection. IEEE Access 6, 74061–74071 (2018). 22/24

    2018

  22. [30]

    Ahmadi, M. et al. Comparative analysis of segment anything model and u-net for breast tumor detection in ultrasound and mammography images. arXiv preprint arXiv:2306.12510 (2023)

    2023 arXiv

  23. [31]

    Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023)

    2023

  24. [32]

    Luo, Y . et al. Biomedgpt: Open multimodal generative pre-trained transformer for biomedicine. arXiv preprint arXiv:2308.09442 (2023)

    2023 arXiv

  25. [33]

    Luu, R. K. & Buehler, M. J. Bioinspiredllm: Conversational large language model for the mechanics of biological and bio-inspired materials. Adv. Sci. 11, 2306724 (2024)

    2024

  26. [34]

    Biomistral: A collection of open-source pretrained large language models for medical domains

    Labrak, Y .et al. Biomistral: A collection of open-source pretrained large language models for medical domains. arXiv preprint arXiv:2402.10373 (2024)

    2024 arXiv

  27. [35]

    Guo, J. et al. Automated chemical reaction extraction from scientific literature. J. chemical information modeling 62, 2035–2045 (2021)

    2021

  28. [36]

    Gupta, T., Zaki, M., Krishnan, N. A. & Mausam. Matscibert: A materials domain language model for text mining and information extraction. npj Comput. Mater. 8, 102 (2022)

    2022

  29. [37]

    A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing

    Shetty, P.et al. A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing. npj Comput. Mater. 9, 52 (2023)

    2023

  30. [38]

    Zhao, Z. et al. Chemdfm: Dialogue foundation model for chemistry. arXiv preprint arXiv:2401.14818 (2024)

    2024 arXiv

  31. [39]

    Zhang, D. et al. Chemllm: A chemical large language model. arXiv preprint arXiv:2402.06852 (2024)

    2024 arXiv

  32. [40]

    N., Chen, Z., Ning, X

    Yu, B., Baker, F. N., Chen, Z., Ning, X. & Sun, H. Llasmol: Advancing large language models for chemistry with a large-scale, comprehensive, high-quality instruction tuning dataset. arXiv preprint arXiv:2402.09391 (2024)

    2024 arXiv

  33. [41]

    Chen, L. et al. Pharmgpt: Domain-specific large language models for bio-pharmaceutical and chemistry. arXiv preprint arXiv:2406.18045 (2024)

    2024 arXiv

  34. [42]

    Sun, L. et al. Scieval: A multi-level large language model evaluation benchmark for scientific research. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 19053–19061 (2024)

    2024

  35. [43]

    Gu, Z. et al. Xiezhi: An ever-updating benchmark for holistic domain knowledge evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 18099–18107 (2024)

    2024

  36. [44]

    C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models

    Huang, Y .et al. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. Adv. Neural Inf. Process. Syst. 36 (2024)

    2024

  37. [45]

    Zhong, W. et al. Agieval: A human-centric benchmark for evaluating foundation models. arXiv preprint arXiv:2304.06364 (2023)

    2023 arXiv

  38. [46]

    & Deng, C

    Chen, Q. & Deng, C. Bioinfo-bench: A simple benchmark framework for llm bioinformatics skills evaluation. bioRxiv 2023–10 (2023)

    2023

  39. [47]

    Lu, P. et al. Learn to explain: Multimodal reasoning via thought chains for science question answering. Adv. Neural Inf. Process. Syst. 35, 2507–2521 (2022)

    2022

  40. [48]

    Clark, P. et al. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 (2018). 23/24

    2018 arXiv

  41. [49]

    Zhang, T., Kishore, V ., Wu, F., Weinberger, K. Q. & Artzi, Y . Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 (2019)

    2019 arXiv

  42. [50]

    Rouge: A package for automatic evaluation of summaries

    Lin, C.-Y . Rouge: A package for automatic evaluation of summaries. InText summarization branches out, 74–81 (2004)

    2004

  43. [51]

    Zhao, W. et al. Moverscore: Text generation evaluating with contextualized embeddings and earth mover distance. arXiv preprint arXiv:1909.02622 (2019)

    2019 arXiv

  44. [52]

    & Parikh, A

    Sellam, T., Das, D. & Parikh, A. P. Bleurt: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04696 (2020)

    2020 arXiv

  45. [53]

    Y ., Miao, B

    Williams, C. Y ., Miao, B. Y . & Butte, A. J. Evaluating the use of gpt-3.5-turbo to provide clinical recommendations in the emergency department. medRxiv 2023–10 (2023)

    2023

  46. [54]

    Touvron, H. et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  47. [55]

    Taylor, R. et al. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085 (2022)

    2022 arXiv

  48. [56]

    Yang, R. et al. Kg-rank: Enhancing large language models for medical qa with knowledge graphs and ranking techniques. arXiv preprint arXiv:2403.05881 (2024)

    2024 arXiv

  49. [57]

    B., Agichtein, E., Pinter, Y

    Abacha, A. B., Agichtein, E., Pinter, Y . & Demner-Fushman, D. Overview of the medical question answering task at trec 2017 liveqa. In TREC, 1–12 (2017)

    2017

  50. [58]

    Malaviya, C. et al. Expertqa: Expert-curated questions and attributed answers. arXiv preprint arXiv:2309.07852 (2023)

    2023 arXiv

  51. [59]

    Abacha, A. B. et al. Bridging the gap between consumers’ medication questions and trusted answers. In MEDINFO 2019: Health and Wellbeing e-Networks for All, 25–29 (IOS Press, 2019). 24/24

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.