A flipped distillation method lets a decoder-only LLM learn text-matching similarity from a smaller encoder teacher through LoRA and a margin-aware contrastive loss, improving matching accuracy and online FAQ retrieval.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Flipping Knowledge Distillation: Leveraging Small Models' Expertise to Enhance LLMs in Text Matching
A flipped distillation method lets a decoder-only LLM learn text-matching similarity from a smaller encoder teacher through LoRA and a margin-aware contrastive loss, improving matching accuracy and online FAQ retrieval.