Pith. sign in

REVIEW 5 cited by

Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.02233 v4 pith:D6QC4BVX submitted 2023-09-05 cs.CL cs.AI

Yubo Wang , Xueguang Ma , Wenhu Chen This is my paper · ORCID
classification cs.CLcs.AI
keywords medicalllmsknowledgetextbooksllm-amtauthoritativecorpusmodel
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large-scale language models (LLMs) like ChatGPT have demonstrated impressive abilities in generating responses based on human instructions. However, their use in the medical field can be challenging due to their lack of specific, in-depth knowledge. In this study, we present a system called LLMs Augmented with Medical Textbooks (LLM-AMT) designed to enhance the proficiency of LLMs in specialized domains. LLM-AMT integrates authoritative medical textbooks into the LLMs' framework using plug-and-play modules. These modules include a Query Augmenter, a Hybrid Textbook Retriever, and a Knowledge Self-Refiner. Together, they incorporate authoritative medical knowledge. Additionally, an LLM Reader aids in contextual understanding. Our experimental results on three medical QA tasks demonstrate that LLMAMT significantly improves response quality, with accuracy gains ranging from 11.6% to 16.6%. Notably, with GPT-4-Turbo as the base model, LLM-AMT outperforms the specialized Med-PaLM 2 model pre-trained on a massive amount of medical corpus by 2-3%. We found that despite being 100x smaller in size, medical textbooks as a retrieval corpus is proven to be a more effective knowledge database than Wikipedia in the medical domain, boosting performance by 7.8%-13.7%.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. K-COMP: Retrieval-Augmented Medical Domain Question Answering With Knowledge-Injected Compressor

    cs.CL 2025-01 conditional novelty 6.0 of 10

    K-COMP generates entity definitions and a compressed summary from retrieved medical passages, improving retrieval-augmented QA over baseline compressors on MedQuAD, MASH-QA, and BioASQ.

  2. GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models

    cs.LG 2025-01 conditional novelty 6.0 of 10

    GANQ minimizes layer-wise output error for lookup-table based non-uniform weight quantization, improving LLM perplexity at 3-4 bits and enabling up to 2.57x inference speedup.

  3. COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation

    cs.LG 2024-12 conditional novelty 6.0 of 10

    A diversity-aware combinatorial mutual information retrieval objective (COBRA) outperforms nearest-neighbor retrieval for few-shot CLIP adaptation.

  4. RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Models

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Adding retrieval-augmented actions and a retrieval-based factuality scorer to the rStar tree-search framework improves accuracy on six medical and commonsense QA benchmarks, reportedly matching or beating GPT-4o on se...

  5. Efficient Learning Content Retrieval with Knowledge Injection

    cs.CL 2024-11 conditional novelty 4.0 of 10

    Fine-tuning Phi models with QLoRA and combining them with RAG on 920 GPT-4-generated Q&A pairs produces a limited-resource chatbot that scores best with Phi-2 plus RAG on automatic metrics.

Pith tools