A single MoE block placed after the final Transformer layer consistently improves TinyBERT dense retrieval across four benchmarks, with marginal or negative effects on larger models.
In: Natural Language Process- ing and Chinese Computing (2022), https://api.semanticscholar.org/CorpusID: 248218762
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.IR 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Investigating Mixture of Experts in Dense Retrieval
A single MoE block placed after the final Transformer layer consistently improves TinyBERT dense retrieval across four benchmarks, with marginal or negative effects on larger models.