Pith. sign in

REVIEW 4 major objections 5 minor 29 references

ReFINE: A Reward-Based Framework for Interpretable and Nuanced Evaluation of Radiology Report Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A reward-model metric for radiology reports, trained on GPT-4-generated examples, correlates with radiologist judgments better than existing evaluation metrics while also producing per-criterion sub-scores.

desk verdict Overall-score correlation on ReXVal is genuinely strong, but the per-criterion sub-scores that the paper sells as interpretability are too weak to support the 'nuanced evaluation' claim. read the letter →

arxiv 2411.17301 v2 pith:43UZM7IM submitted 2024-11-26 cs.CL cs.AI

Yunyi Liu , Yingshu Li , Zhanyu Wang , Xinyu Liang , Lingqiao Liu , Lei Wang , Luping Zhou This is my paper · ORCID
classification cs.CLcs.AI
keywords radiologyreportgenerationrewardmodelevaluationmetricmargin-basedenforcementlossinterpretabilityLLMReXValMRScore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ReFINE is an automatic metric for evaluating AI-generated radiology reports, built as a reward model that learns to rank reports by quality. The paper's central claim is that a small LLM fine-tuned with a margin-based reward enforcement loss on GPT-4-scored report pairs can reproduce radiologist judgments more closely than existing metrics, while also returning a score for each evaluation criterion. On the ReXVal benchmark, ReFINE reaches a Kendall's Tau of 0.751 and Spearman correlation of 0.910 with radiologist error scores, above the best compared metric GREEN at 0.640. The practical payoff is that model developers could replace costly human evaluation and opaque n-gram scores with an interpretable, locally run metric that says why a report is penalized.

What carries the argument

The central object is the margin-based reward enforcement (MRE) loss, which trains a LoRA-tuned Llama3 model with a linear multi-reward head to output N criterion rewards in one forward pass. For each accepted/rejected report pair, the loss penalizes the model when a predicted sub-reward difference disagrees with the GPT-4-assigned per-criterion margin, allowing margins to be negative when a 'better' report is worse on one criterion, and it separately penalizes total-reward disagreement using a total-score margin. This paired sample design, where accepted and rejected reports are derived from the same ground-truth report with known score margins, is what lets the model learn fine-grained, human-aligned quality judgments instead of a single global preference.

What would settle it

If one reruns the ReXVal comparison after training ReFINE only on real R2Gen outputs with radiologist scores instead of GPT-4-generated tiers, and the correlation gap with GREEN disappears, that would falsify the claim that the synthetic-data recipe is what drives alignment; a simpler check is whether the 'incorrect location or position' sub-criterion, where ReFINE scores only 0.246 Kendall's Tau, climbs substantially when trained on data that oversamples spatial errors.

Watch

Extended reading notes

Core claim

The paper argues that report-quality evaluation can be treated as a reward-modeling problem rather than a text-similarity problem. Taking two GPT-4-generated reports about the same ground-truth radiology report—one marked 'accepted' and one 'rejected' under a chosen scoring system—ReFINE fine-tunes Llama3 with a multi-reward head and a margin-based reward enforcement loss. The loss enforces that predicted sub-rewards preserve the per-criterion margins of the paired scores, and that their sum preserves the total-score margin. At inference, the sum of the predicted sub-rewards is the ReFINE score; the sub-rewards themselves identify which criteria (for example, false prediction of a finding or incorrect severity) drive the score. The authors establish the claim by showing higher correlation with human radiologist ratings than BLEU-4, ROUGE-L, METEOR, CIDEr, BERTScore, RadGraphF1, RadCliQ-v1, and GREEN on ReXVal, and by demonstrating the same training procedure transfers to the MRScore criteria on the Rad-100 dataset.

Load-bearing premise

The training signal comes from GPT-4-generated reports with hand-assigned error tiers, and the paper assumes those synthetic reports look like real radiology-report-generator outputs in their errors and score distribution; if that assumption fails, the reward model's human alignment at test time is not guaranteed.

Editorial extensions

If this is right

  • Radiology report generation systems can be evaluated and compared without collecting new human error annotations for every candidate output, since ReFINE is trained once and runs locally.
  • Users of report generators can see which evaluation criterion is failing (for example, omission of findings versus incorrect location), turning a single quality number into actionable diagnostic feedback.
  • The same reward-model recipe can be retrained for different scoring systems, as demonstrated by transferring from RadCliQ criteria to MRScore criteria.
  • Because ReFINE correlates better with radiologists than GREEN while using far fewer training resources, it offers a cheaper and faster metric for model selection in R2Gen research.
  • The per-criterion sub-scores make it possible for users to re-weight or customize evaluation emphasis without retraining the whole pipeline, since the sub-scores are individually meaningful.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to use ReFINE's sub-rewards as dense reward signals for reinforcement learning or preference optimization of report generators, something the paper does not test.
  • The weak location/position sub-score correlation (Kendall's Tau 0.246 on ReXVal) suggests the GPT-4-generated training data may under-represent spatial errors; testing with a training set that oversamples such errors would reveal whether that gap is a data artifact or a model limitation.
  • The same GPT-4-to-reward-model pipeline could generalize to other clinical note types or structured reporting tasks, provided the scoring criteria can be expressed in a prompt.
  • Replacing human annotation with GPT-4-generated tiers is the main epistemic risk; a direct distributional comparison between synthetic and real R2Gen outputs would tell whether the metric's alignment holds outside the training distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ReFINE, a trainable reward-model-based metric for evaluating radiology report generation. The method uses GPT-4 to generate synthetic training reports with injected errors and corresponding scores under two scoring systems (RadCliQ and MRScore), then fine-tunes Llama3 with LoRA using a new margin-based reward enforcement (MRE) loss. The model outputs multiple sub-scores for individual criteria plus a total score. ReFINE is evaluated on the ReXVal dataset (200 candidate reports) and a newly constructed Rad-100 dataset (100 reports), and is compared with NLG, clinical, and LLM-based metrics. The paper reports strong total-score correlations on ReXVal (Kendall's Tau 0.751) and much weaker correlations on Rad-100 (0.230), with mixed sub-score performance on both datasets.

Significance. If the results hold, ReFINE would be a useful, locally deployable evaluation metric that aligns with human judgments better than existing metrics while providing per-criterion sub-scores. The paper's strengths include the use of two human-annotated test sets, ablations of the loss components and LLM backbones, and a computational efficiency advantage over GREEN. However, the central interpretability claim is only weakly supported: several sub-score correlations and accuracies are close to chance, and the Rad-100 total correlation is low. The paper also lacks uncertainty quantification and statistical comparisons between metrics. The contribution is therefore promising but not yet fully validated.

major comments (4)
  1. [Tables 1 and 3; Sections 1 and 5] The per-criterion sub-scores, which are the paper's central differentiator over scalar metrics, are not reliable for several criteria. On ReXVal, Kendall's Tau is 0.246 for 'Incorrect location or position of a finding' and 0.267 for 'Omission of a comparison that notes a change' (Table 1). On Rad-100, binary accuracies are 0.380 for Completeness and 0.410 for Clinical History (Table 3). Without base rates or inter-rater agreement, these values are near chance. A high total-score correlation does not validate the sub-scores because the total can be dominated by frequent error categories. The claims in the Abstract and Section 1 that users can 'pinpoint the reasons for a report's poor quality' and that ReFINE provides 'nuanced evaluation' are therefore not supported by the reported evidence. The paper should provide additional evidence of sub-score validity (e.g., inter-rater agreement, base rates, confidence intervals) or temper the interpretability claims accordingly.
  2. [Section 4.2, Table 2] The claim that ReFINE 'outperforms' GREEN and other metrics is based on point estimates without confidence intervals, variance across training seeds, or a significance test for the difference between correlated correlation coefficients. The p-values reported in Table 2 test each correlation against zero, not the difference between two metrics. A bootstrap confidence interval or a test such as Williams's procedure should be reported before claiming superiority, especially since the margin over GREEN (0.751 vs. 0.640) could be within sampling noise for a dataset of 200 reports.
  3. [Section 3.1] The training data are GPT-4-generated reports with hand-injected error tiers (0-2, 3-4, 5-6 errors for RadCliQ; 0-40, 40-70, 70-100 for MRScore). The paper assumes these synthetic reports are representative of real R2Gen outputs in error type, severity, and distribution, so the reward model trained on them will generalize to actual model outputs. The only validation described is a 50-sample check of score agreement with a radiologist, which does not verify distributional similarity. Without such evidence, the generalization premise is unverified. The paper should analyze the error distributions of synthetic versus real outputs or otherwise justify this assumption.
  4. [Section 4.3, Table 4] The Rad-100 results are weak: the overall Kendall's Tau is 0.230 and several sub-score accuracies are at or below 0.5. Although these values are statistically significant, the paper describes ReFINE as demonstrating 'superior performance' without discussing the absolute weakness of the correlation. Since Rad-100 is a second dataset intended to validate the human-alignment claim, the low correlation should be addressed directly, for example by analyzing error patterns, comparing with training data distribution, or discussing the limitations of the MRScore-based evaluation.
minor comments (5)
  1. [Section 4.3, heading] 'Dataest' is a typo for 'Dataset'.
  2. [Table 4 and Section 4.3] The metric is referred to as 'ER²Score' instead of 'ReFINE'; this is inconsistent with the rest of the paper and should be corrected.
  3. [Introduction, first paragraph] METEOR is cited as [4], but reference [4] is Sentence-BERT; METEOR is correctly cited as [18] in Section 2.1.
  4. [Section 4.5] The description of the hyperparameter c as 'a small positive rounding number' is vague; please specify how c was chosen and how it interacts with the reward scale.
  5. [Table 3] Reporting base rates for the Rad-100 sub-criteria would help interpret the accuracies, since values above or below chance cannot be assessed without knowing the frequency of each error type.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: human-correlation results are external, and total-as-sum-of-sub-scores is an explicit definition, not a hidden reuse of the target.

full rationale

The central claim—that ReFINE correlates with human radiologist judgments (Kendall's Tau 0.751 on ReXVal, 0.230 on Rad-100)—is tested against external human labels, not against the GPT-4 scores used for training. ReXVal is a public dataset of radiologist error annotations; Rad-100, while constructed by the authors, uses an experienced radiologist's judgments under the MRScore rubric. The training data are GPT-4-generated synthetic reports with injected error tiers, and the model is trained to reproduce GPT-4 margins; this makes the training objective a proxy-learning step, but the evaluation is genuinely external. The total ReFINE score is explicitly defined as the sum of the sub-scores (Section 3.2: 'These rewards are then summed to generate the final ReFINE'; Section 3.3, L_MRE with L_tot = ReLU(-(sum_j r_w - sum_j r_l) + m_i)). This is an explicit architectural definition rather than a hidden circular derivation; the paper does not claim the total is an independently fitted quantity. The only self-citation is [12] (MRScore) in Section 3.1, used to support GPT-4's human-alignment under the MRScore rubric. This is corroborated by external work [11] and by the paper's own 50-sample radiologist accuracy check (Section 3.1), so it is not load-bearing. The acknowledged limitations—low sub-score correlations for categories such as 'Incorrect location or position of a finding' (Tau 0.246) and the absence of paragraph explanations—are validity or explainability concerns, not circularity. No enumerated circularity pattern applies to the derivation chain.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The framework introduces no new scientific entities. It depends on lambda and c as tuned hyperparameters, on hand-chosen synthetic error tiers, and on the assumption that GPT-4 labels and synthetic reports faithfully represent human judgment and real model error patterns.

free parameters (3)
  • lambda (loss balance) = 1.0
    Selected by sweeping values from 0.5 to 3.0; 1.0 gives the best ReXVal correlation. It balances Lind and Ltot.
  • c (equal-subscore tolerance) = 1e-2
    Preset small positive value to judge when two rewards should be considered equal.
  • Error-tier boundaries = 0-2, 3-4, 5-6 errors (RadCliQ); 0-40, 40-70, 70-100 (MRScore)
    Hand-chosen to create three quality tiers in GPT-4 training data; these distributions shape the model's learned scoring.
assumptions (3)
  • domain assumption GPT-4 scores are a reliable proxy for human evaluation of radiology reports.
    The entire training data pipeline relies on GPT-4's scoring outcomes; this is justified by citing [11] and the authors' own [12].
  • domain assumption GPT-4-generated reports with injected error counts are representative of real R2Gen outputs.
    No comparison between the synthetic training distribution and real model outputs is provided.
  • domain assumption Pairwise margin training transfers to accurate pointwise scores.
    Standard preference-learning assumption; not verified with calibration analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReFINE: A Reward-Based Framework for Interpretable and Nuanced Evaluation of Radiology Report Generation." pith.science (2026). https://pith.science/paper/43UZM7IM

@misc{pith2026241117301,
  author       = {Pith},
  title        = {Pith review of: ReFINE: A Reward-Based Framework for Interpretable and Nuanced Evaluation of Radiology Report Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/43UZM7IM}},
  note         = {Machine review of arXiv:2411.17301}
}
read the original abstract

Automated radiology report generation (R2Gen) has advanced significantly, introducing challenges in accurate evaluation due to its complexity. Traditional metrics often fall short by relying on rigid word-matching or focusing only on pathological entities, leading to inconsistencies with human assessments. To bridge this gap, we introduce ReFINE, an automatic evaluation metric designed specifically for R2Gen. Our metric utilizes a reward model, guided by our margin-based reward enforcement loss, along with a tailored training data design that enables customization of evaluation criteria to suit user-defined needs. It not only scores reports according to user-specified criteria but also provides detailed sub-scores, enhancing interpretability and allowing users to adjust the criteria between different aspects of reports. Leveraging GPT-4, we designed an easy-to-use data generation pipeline, enabling us to produce extensive training data based on two distinct scoring systems, each containing reports of varying quality along with corresponding scores. These GPT-generated reports are then paired as accepted and rejected samples through our pairing rule to train an LLM towards our fine-grained reward model, which assigns higher rewards to the report with high quality. Our reward-control loss enables this model to simultaneously output multiple individual rewards corresponding to the number of evaluation criteria, with their summation as our final ReFINE. Our experiments demonstrate ReFINE's heightened correlation with human judgments and superior performance in model selection compared to traditional metrics. Notably, our model provides both an overall score and individual scores for each evaluation item, enhancing interpretability. We also demonstrate its flexible training across various evaluation systems.

Figures

Figures reproduced from arXiv: 2411.17301 by the authors.

Figure 1
Figure 1. Overview of ReFINE. The upper portion illustrates the training data generation process, while the lower [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. An illustration of report pairing rule, taking a scoring system with 4 criteria as an example. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. An visual example of ReFINE from ReXVal Dataset. The highlighted sentences in reports and their [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 13 canonical work pages

  1. [1]

    R2gengpt: Radiology report generation with frozen llms

    Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. R2gengpt: Radiology report generation with frozen llms. arXiv preprint arXiv:2309.09812, 2023

    arXiv 2023

  2. [2]

    Kargen: Knowledge-enhanced automated radiology report generation using large language models

    Yingshu Li, Zhanyu Wang, Yunyi Liu, Lei Wang, Lingqiao Liu, and Luping Zhou. Kargen: Knowledge-enhanced automated radiology report generation using large language models. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 382–392. Springer, 2024

    work page 2024

  3. [3]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  4. [4]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, C...

    work page 2019

  5. [5]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020

    work page 2020

  6. [6]

    Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert

    Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y Ng, and Matthew P Lungren. Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert. arXiv preprint arXiv:2004.09167, 2020

    arXiv 2004

  7. [7]

    Radgraph: Extracting clinical entities and relations from radiology reports

    Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven QH Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P Lungren, Andrew Y Ng, et al. Radgraph: Extracting clinical entities and relations from radiology reports. arXiv preprint arXiv:2106.14463, 2021

    arXiv 2021

  8. [8]

    A systematic evaluation of gpt-4v’s multimodal capability for chest x-ray image analysis

    Yunyi Liu, Yingshu Li, Zhanyu Wang, Xinyu Liang, Lingqiao Liu, Lei Wang, Leyang Cui, Zhaopeng Tu, Longyue Wang, and Luping Zhou. A systematic evaluation of gpt-4v’s multimodal capability for chest x-ray image analysis. Meta-Radiology, page 100099, 2024

    work page 2024

Show all 29 references
  1. [9]

    Evaluating progress in automatic chest x-ray radiology report generation

    Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, Eduardo Kaiser Uru- rahy Nunes Fonseca, Henrique Min Ho Lee, Zahra Shakeri Hossein Abad, Andrew Y Ng, et al. Evaluating progress in automatic chest x-ray radiology report generation. Patterns, 4(9), 2023

    2023

  2. [10]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023

    2023 arXiv

  3. [11]

    Can large language models be an alternative to human evaluations? arXiv preprint arXiv:2305.01937, 2023

    Cheng-Han Chiang and Hung-yi Lee. Can large language models be an alternative to human evaluations? arXiv preprint arXiv:2305.01937, 2023

    2023 arXiv

  4. [12]

    Mrscore: Evaluating radiology report generation with llm-based reward system

    Yunyi Liu, Zhanyu Wang, Yingshu Li, Xinyu Liang, Lingqiao Liu, Lei Wang, and Luping Zhou. Mrscore: Evaluating radiology report generation with llm-based reward system. arXiv preprint arXiv:2404.17778, 2024

    2024 arXiv

  5. [13]

    Introducing meta llama 3: The most capable openly available llm to date

    Meta. Introducing meta llama 3: The most capable openly available llm to date. https://ai.meta.com/blog/ meta-llama-3/, 2024. Accessed: 2024-05-20

    2024

  6. [14]

    Training small multimodal models to bridge biomedical competency gap: A case study in radiology imaging

    Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu, Hanwen Xu, Naoto Usuyama, Sheng Zhang, Fei Wang, Yujia Xie, Mahmoud Khademi, Ziyi Yang, et al. Training small multimodal models to bridge biomedical competency gap: A case study in radiology imaging. CoRR, 2024

    2024

  7. [15]

    Fineradscore: A ra- diology report line-by-line evaluation technique generating corrections with severity scores

    Alyssa Huang, Oishi Banerjee, Kay Wu, Eduardo Pontes Reis, and Pranav Rajpurkar. Fineradscore: A ra- diology report line-by-line evaluation technique generating corrections with severity scores. arXiv preprint arXiv:2405.20613, 2024

    2024 arXiv

  8. [16]

    Green: Generative radiology report evaluation and error notation

    Sophie Ostmeier, Justin Xu, Zhihong Chen, Maya Varma, Louis Blankemeier, Christian Bluethgen, Arne Edward Michalson, Michael Moseley, Curtis Langlotz, Akshay S Chaudhari, et al. Green: Generative radiology report evaluation and error notation. arXiv preprint arXiv:2405.03595, 2024

    2024 arXiv

  9. [17]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004

    2004

  10. [18]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005

    2005

  11. [19]

    Bertscore: Evaluating text generation with bert

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. 11 A PREPRINT - F EBRUARY 14, 2025

    1904 arXiv

  12. [20]

    Ball, Katie S

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Christopher Chute, Henrik Marklund, Behzad Haghgoo, Robyn L. Ball, Katie S. Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Halabi, Jesse K. Sandberg, Ricky Jones, David B. Larson, Curtis P. Langl...

    2019

  13. [21]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  14. [22]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Represen- tations, 2022

    2022

  15. [23]

    Radiology report expert evaluation (rexval) dataset, 2023

    Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, EKU Fonseca, Henrique Lee, Zahra Shakeri, Andrew Ng, et al. Radiology report expert evaluation (rexval) dataset, 2023

    2023

  16. [24]

    Cider: Consensus-based image description evaluation

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015

    2015

  17. [25]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, march 2023

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, march 2023. URL https://lmsys. org/blog/2023-03-30-vicuna,...

    2023

  18. [26]

    Meditron-70b: Scaling medical pretraining for large language models

    Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al. Meditron-70b: Scaling medical pretraining for large language models. arXiv preprint arXiv:2311.16079, 2023

    2023 arXiv

  19. [27]

    Thomas Mesnard Gemma Team, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, and et al. Gemma. 2024

    2024

  20. [28]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Si...

    2023 arXiv

  21. [29]

    Textbooks are all you need ii: phi-1.5 technical report, 2023

    Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report, 2023. 12

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.