The 'knowledge comes from pre-training' pattern seen in DPR/BERT does not generalize to Contriever (mean pooling) or RepLlama (decoder), where fine-tuning instead reduces neuron activation breadth.
Natural questions: A benchmark for question answering
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.IR 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition
The 'knowledge comes from pre-training' pattern seen in DPR/BERT does not generalize to Contriever (mean pooling) or RepLlama (decoder), where fine-tuning instead reduces neuron activation breadth.