REVIEW 21 cited by
MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As large language models (LLMs) like OpenAI's GPT series continue to make strides, we witness the emergence of artificial intelligence applications in an ever-expanding range of fields. In medicine, these LLMs hold considerable promise for improving medical workflows, diagnostics, patient care, and education. Yet, there is an urgent need for open-source models that can be deployed on-premises to safeguard patient privacy. In our work, we present an innovative dataset consisting of over 160,000 entries, specifically crafted to fine-tune LLMs for effective medical applications. We investigate the impact of fine-tuning these datasets on publicly accessible pre-trained LLMs, and subsequently, we juxtapose the performance of pre-trained-only models against the fine-tuned models concerning the examinations that future medical doctors must pass to achieve certification.
Forward citations
Cited by 21 Pith papers
-
Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints
A black-box audit detects unauthorized fine-tuning by measuring a joint semantic-lexical distributional fingerprint in model outputs, robust to paraphrasing and distillation.
-
SciHorizon-GENE: Benchmarking LLM for Life Sciences Inference from Gene Knowledge to Functional Understanding
SciHorizon-GENE is a large-scale benchmark evaluating LLMs on gene-to-function inference across four perspectives, revealing heterogeneity and challenges in faithful, complete, literature-grounded outputs.
-
Dr.Copilot: A Multi-Agent Prompt Optimized Assistant for Improving Patient-Doctor Communication in Romanian
A multi-agent LLM assistant with DSPy-optimized prompts improved Romanian doctors' written communication quality and patient satisfaction in a live telemedicine deployment, but the evaluation is confounded by self-sel...
-
DocCHA: Towards LLM-Augmented Interactive Online diagnosis System
DocCHA, a confidence-scored three-module LLM pipeline, reports improved diagnostic accuracy and information recall over direct-prompting LLMs on two Chinese consultation datasets, but evaluation gaps weaken the claim.
-
DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains
DivScore detects AI-written medical and legal text by dividing a domain-tuned model's entropy by its disagreement with a general model, beating baselines on a new benchmark.
-
Resolving Knowledge Conflicts in Domain-specific Data Selection: A Case Study on Medical Instruction-tuning
KDS selects instruction-tuning data by measuring knowledge alignment (NLI entailment vs reference) and knowledge consistency (cluster entropy of sampled responses), and reports gains on medical QA benchmarks.
-
Diagnosing our datasets: How does my language model learn clinical information?
The frequency of clinical jargon in pretraining corpora predicts how well open-source LLMs interpret that jargon, but hospital notes use abbreviations that appear only rarely online.
-
Gaokerena: A Small Persian Medical Language Model Family
Fine-tuned Persian medical language models reach 49-53% on translated medical MMLU, with datasets released, but the reasoning variant's gain depends on extra test-time compute and a verifier.
-
Go-UT-Bench: A Fine-Tuning Dataset for LLM-Based Unit Test Generation in Go
A new reproducible Go code-and-test dataset lets fine-tuned LLMs beat base models on 76–82% of unit-test generation judgments, though the judgments are made by another LLM and no tests are run.
-
Gene-R1: Reasoning with Data-Augmented Lightweight LLMs for Gene Set Analysis
Gene-R1 combines domain pre-training, GPT-o1 distilled reasoning examples, and GRPO reinforcement learning so lightweight Llama models match commercial LLMs on gene set analysis.
-
The Non-Determinism of Small LLMs: Evidence of Low Answer Consistency in Repetition Trials of Standard Multiple-Choice Benchmarks
Small open-source LLMs give inconsistent answers on 20% to 50% of multiple-choice questions even at low temperature, while 50B-80B models are consistent on over 90% of questions.
-
TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
Trident-Bench provides 2,652 professionally validated harmful prompts across finance, law, and medicine, and shows that domain-specialized LLMs often comply with unethical requests more than generalist models.
-
DoPI: Doctor-like Proactive Interrogation LLM for Traditional Chinese Medicine
DoPI pairs a knowledge-graph-guided questioning model with a TCM expert model and claims 84.68% diagnostic accuracy, but the benchmark is built from the same symptom-disease rules that drive the system.
-
DiaLLMs: EHR Enhanced Clinical Conversational System for Clinical Test Recommendation and Diagnosis Prediction
DiaLLM is an EHR-grounded conversational system that translates clinical codes and test results into text and uses PPO with rejection sampling to recommend lab tests and predict diagnoses, reporting large gains over b...
-
Collaboration among Multiple Large Language Models for Medical Question Answering
An iterative collaboration framework where three LLMs exchange summarized reasoning on disagreed questions raises USMLE-style answer accuracy by 5.2 to 6.6 percentage points per model and raises consensus from 51% to 83%.
-
Mitigating hallucinations in healthcare LLMs with granular fact-checking and domain-specific adaptation
A deterministic, proposition-level fact-checker that compares clinical summaries against electronic health records via (entity, attribute, value, time) claims and hard-coded logical checks reports 0.8904 precision and...
-
Med-GRIM: Enhanced Zero-Shot Medical VQA using prompt-embedded Multimodal Graph RAG
Med-GRIM, combining the BIND encoder with graph retrieval and small language models, reports 83.33% accuracy on a new 30-question DermaGraph dermatology benchmark, outperforming several larger medical VLMs in a zero-s...
-
Preserving Privacy, Increasing Accessibility, and Reducing Cost: An On-Device Artificial Intelligence Model for Medical Transcription and Note Generation
Fine-tuning a 1B Llama model on synthetic endocrinology data improves structured medical note generation and substantially reduces LLM-judged hallucinations and omissions in a browser-based, on-device deployment.
-
CLUES: Collaborative High-Quality Data Selection for LLMs via Training Dynamics
A collaborative data-selection method that scores each private sample's influence on a public anchor set and filters by a global threshold before federated learning or model merging.
-
Balancing Knowledge Delivery and Emotional Comfort in Healthcare Conversational Systems
Fine-tuning a 1B medical chatbot on LLM-rewritten emotional dialogues improves its emotion scores with only small changes in n-gram overlap with the original medical responses.
-
Enabling On-Device Medical AI Assistants via Input-Driven Saliency Adaptation
A saliency-based pruning plus 4-bit quantization pipeline runs Gemma 7B and LLaMA 8B on edge hardware, but medical QA accuracy drops by up to 27 points, contradicting the 'minimal accuracy loss' claim.
Discussion (0). Sign in to comment.