A multi-level optimal transport loss combining sequence-level ranking, top-k truncation, and Sinkhorn sequence distance outperforms earlier cross-tokenizer distillation losses on QA and summarization.
G.; Bradley, H.; O’Brien, K.; Hallahan, E.; Khan, M
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models
A multi-level optimal transport loss combining sequence-level ranking, top-k truncation, and Sinkhorn sequence distance outperforms earlier cross-tokenizer distillation losses on QA and summarization.