A small model prompted with evidence and knowledge graphs generated by GPT-4o scores much higher on QA benchmarks, but the result is not true distillation and may be contaminated by teacher answer leakage.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
A small model prompted with evidence and knowledge graphs generated by GPT-4o scores much higher on QA benchmarks, but the result is not true distillation and may be contaminated by teacher answer leakage.