Experiments on four models and three datasets show SFT increases sensitivity to easy contexts while later stages (DPO, RLVR) can reinforce or reverse those preferences depending on the dataset.
Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLM s
4 Pith papers cite this work, alongside 79 external citations. Polarity classification is still indexing.
4
Pith papers citing it
79
external citations · OpenAlex
fields
cs.CL 4years
2026 4representative citing papers
Large-scale evaluation shows retrieval-augmented generation yields only marginal and inconsistent gains (1-2 points) over no-retrieval baselines in biomedical QA, with model choice dominating retriever or corpus effects.
MixSD uses dynamic mixing of the model's expert and naive conditionals to create distribution-aligned supervision that improves the memorization-retention tradeoff over standard SFT.
MeMo encodes new knowledge into a separate memory model that integrates with frozen LLMs, showing strong performance on QA benchmarks while avoiding catastrophic forgetting and working without access to model weights.
citing papers explorer
-
Emergence of Context Characteristics Sensitivity in Large Language Models
Experiments on four models and three datasets show SFT increases sensitivity to easy contexts while later stages (DPO, RLVR) can reinforce or reverse those preferences depending on the dataset.
-
When Retrieval Doesn't Help: A Large-Scale Study of Biomedical RAG
Large-scale evaluation shows retrieval-augmented generation yields only marginal and inconsistent gains (1-2 points) over no-retrieval baselines in biomedical QA, with model choice dominating retriever or corpus effects.
-
MixSD: Mixed Contextual Self-Distillation for Knowledge Injection
MixSD uses dynamic mixing of the model's expert and naive conditionals to create distribution-aligned supervision that improves the memorization-retention tradeoff over standard SFT.
-
MeMo: Memory as a Model
MeMo encodes new knowledge into a separate memory model that integrates with frozen LLMs, showing strong performance on QA benchmarks while avoiding catastrophic forgetting and working without access to model weights.