Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition

Guido Zuccon; Shuai Wang; Zheng Yao

REVIEW 4 major objections 5 minor 14 references

Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pre-training, not fine-tuning, supplies DPR retrieval knowledge — but not for every retriever.

desk verdict A genuinely useful reproducibility extension that confirms and complicates Reichman and Heck, but the neuron-attribution analysis at the center of the non-generalization claim is too underspecified to carry it. read the letter →

arxiv 2505.07166 v1 pith:HHSF3FPM submitted 2025-05-12 cs.IR cs.CL

Zheng Yao , Shuai Wang , Guido Zuccon This is my paper · ORCID

classification cs.IRcs.CL

keywords denseretrievalpre-trainingfine-tuningknowledgeacquisitiondecentralizationlinearprobingneuronattributionreproducibilitystudy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish whether the claim that dense-retrieval fine-tuning does not add knowledge—only reallocates it across neurons—holds beyond the single setup where it was first made. The authors reproduce the original experiments on BERT-based DPR, extend them to MS MARCO, and then test mean-pooled (Contriever) and decoder-only (RepLlama) retrievers. They find the original pattern reproduces for DPR query encoders: pre-trained BERT already separates relevant from irrelevant passages, and fine-tuning decentralizes neuron activations in intermediate layers. The pattern reverses for Contriever and RepLlama, where fine-tuning narrows the set of active neurons. The upshot is that the pre-training-versus-fine-tuning question has no architecture-independent answer.

What carries the argument

Two analysis tools carry the argument. Linear probing trains a simple classifier on frozen layer-wise embeddings to test whether fine-tuning adds discriminative knowledge beyond pre-training. Integrated-gradient neuron attribution computes each feed-forward neuron's contribution to the final embedding (integrating weights from zero to their learned values, normalizing per example, and thresholding at 0.1 times the maximum, with 0.01 used for Llama models) to measure whether knowledge is centralized or decentralized before and after fine-tuning.

What would settle it

Run the same neuron-activation comparison with an alternative attribution method, such as causally ablating individual intermediate-layer neurons and measuring retrieval effectiveness; if DPR-query does not show greater sensitivity to intermediate layers than the backbone, the decentralization pattern is an artifact of the integrated-gradient estimator. A simpler check: vary the activation threshold from 0.01 to 0.5 and see whether the reported 32–41% intermediate-layer increase persists.

Watch

Extended reading notes

Core claim

The paper's central claim is that DPR fine-tuning does not introduce new retrieval knowledge into a BERT backbone; instead it changes how existing knowledge is accessed, primarily by increasing activations in intermediate feed-forward layers (32–41% more neurons above threshold) while suppressing early output-layer activations. This reproduces the earlier study's conclusion on both Natural Questions and MS MARCO for the DPR query encoder. But the same analysis on a mean-pooled retriever (Contriever) and a decoder-based retriever (RepLlama) shows the opposite: fine-tuning reduces the breadth of activated neurons. The paper therefore concludes that knowledge decentralization is architecture- and pooling-dependent, not a general property of dense retrieval, and that the passage encoder does not follow the query encoder's trend.

Load-bearing premise

The integrated-gradient neuron-attribution procedure, including the choice of activation thresholds, is a valid way to locate and compare knowledge distribution in dense retrievers; the authors themselves note this method may not be reliable for this setting.

Editorial extensions

If this is right

If DPR's query encoder relies on pre-trained knowledge, then for BERT-based retrievers, improving retrieval effectiveness beyond the backbone's ceiling requires pre-training changes, not just more contrastive fine-tuning.
If the decentralization effect reverses for mean-pooled and decoder-only retrievers, then research conclusions drawn from DPR alone cannot be used to guide training of Contriever-style or LLM-based retrievers.
Because the DPR passage encoder behaves differently from the query encoder, the original claim that fine-tuning affects both encoders the same way is weakened, and passage-side knowledge flow needs its own analysis.
Architecture and pooling strategy can matter more than fine-tuning for how discriminative the final embeddings are, as seen in RepLlama's 18–22% deeper-layer accuracy jump over its backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension: if knowledge decentralization is a real mechanism, causal interventions (ablating or amplifying intermediate-layer neurons) should change DPR retrieval effectiveness more than the same interventions on Contriever; the current evidence is correlational.
The authors' own caveat that neuron attribution may actually not be reliable for investigating dense retrieval suggests the principal robust finding may be methodological: linear-probing and attribution results are sensitive to pooling, backbone, and threshold choices, so claims about where knowledge lives need multi-method confirmation.
The RepLlama deeper-layer accuracy jump hints that EOS-token representations accumulate relevance signal late in the network; one could test whether layer-wise pruning of late layers degrades RepLlama more than DPR, which would make the architectural difference practically exploitable.
An implicit practical consequence: the cost of pre-training may be justified for BERT-style encoders, but decoder-based retrievers may benefit more from fine-tuning or representation design, so training budgets should depend on backbone family.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, this is a real reproducibility contribution: it confirms the DPR trend Reichman and Heck reported, finds that Contriever and RepLlama do not show the same decentralization pattern in integrated-gradient attributions, and shows DPR-query and DPR-passage behave differently—an observation absent from the original work. Second, the central non-generalization claim is weaker than the abstract makes it sound, because the attribution method is never pinned down: Equation 4 defines P_x as a 'scalar output' but a dense retriever produces a vector embedding, and the text never says what scalarization was used. The post hoc threshold for Llama (0.01 vs 0.1) and the missing error bars compound the issue. The authors admit in Section 5.7 that neuron attribution 'may actually be not reliable for investigating dense retrieval.'

What it does well: it ships code and data, tries to reconstruct the original setup, acknowledges reproduction difficulties (hard negative sourcing, linear probing details), and extends to a second dataset and two additional architectures. The linear probing results stand on their own: BERT-CLS retaining discriminative power, RepLlama's deeper-layer jump being a real and interesting architecture effect. The DPR reproduction alone makes this worth a serious referee.

Where the soft spots are: the IG analysis, which carries the decentralization reversal. The stress-test concern is on target. Since P_x is undefined and the threshold is tuned to show the desired trend, the comparison between DPR, Contriever, and Llama is not interpretable as evidence about knowledge organization. That does not kill the linear probing contribution, but it means the abstract's first sentence—'pre-trained knowledge underpins retrieval performance'—is a claim the paper never actually tests, because no retrieval effectiveness is measured. Also, the t-tests in figures are between model and backbone, not between conditions, so 'statistically significant' markers are easy to over-read.

Who it is for: people working on dense retrieval interpretation and anyone treating Reichman and Heck as a general result. It deserves peer review, not desk rejection; a good referee would push for a defined scalar target, threshold sensitivity analysis with confidence intervals, and an abstract that matches what was measured. I would send it to review, conditionally. The core DPR finding is likely solid; the architecture-dependence claim needs either a defensible attribution target or softer phrasing.

Referee Report

4 major / 5 minor

Summary. The paper is a reproducibility study of Reichman and Heck's claim that DPR fine-tuning does not add new knowledge but rather decentralizes knowledge that already exists from pre-training. The authors reproduce the original linear-probing and integrated-gradient (IG) neuron-activation analyses on DPR with a BERT backbone, and extend the experiments to the MS MARCO dataset, to mean-pooled Contriever, and to decoder-only RepLlama/Llama with EOS-token pooling. They report that the DPR query encoder reproduces the original trend, while the DPR passage encoder, Contriever, and RepLlama show different activation patterns, and they conclude that the original decentralization finding is architecture- and representation-dependent. The implementation and preprocessed datasets are publicly released.

Significance. If the IG-based conclusions are valid, the paper makes a useful contribution by showing that the decentralization effect is not universal across dense retrieval architectures, and it provides a reproducible extension of a previously single-model study. Concrete strengths include the public code and data, the addition of a second dataset, and the use of statistical significance testing for the linear-probing results. However, the central non-universality claim rests on a neuron-attribution method whose scalar output is left undefined for dense retrievers, the Llama activation threshold is chosen post hoc, the IG results have no quantified uncertainty, and the authors themselves concede in Section 5.7 that the method may be unreliable for dense retrieval. The paper's significance therefore depends on issues that must be resolved before the main claim can be accepted.

major comments (4)

[§5.1, Eq. (4)] The quantity P_x in Eq. (4) is never defined, and for dense retrievers it is not obvious what it should be. The manuscript states in §5.2.1 that questions and passages are processed independently, and the model output is a dense vector embedding rather than a scalar relevance score. If P_x is an arbitrary scalarization (a single coordinate, a norm, or a projection), the resulting attributions and active-neuron percentages cannot be interpreted as knowledge organization without justification. Because Figures 5–8 are the sole evidence for the central claim that decentralization is architecture-dependent, this needs to be fixed by specifying P_x, providing a rationale for the scalarization, and ideally validating the IG attributions against an independent measure.
[§5.6 and footnote 10] The activation threshold is 0.1×max for all models except Llama-based models, where 0.01 is used because it "provided the clearest trend." This is post hoc selection of the very quantity that defines the reported outcome, so the reversal for RepLlama in Figure 8 may be an artifact of threshold choice. The authors should report results for a common threshold across all models, quantify sensitivity over a range of thresholds, and provide uncertainty estimates, such as variability across examples or repeated runs.
[§5.7 and Discussion] The authors explicitly acknowledge in Section 5.7 that neuron attribution, the method on which their main generalization claim rests, "may actually be not reliable for investigating dense retrieval." This admission applies equally to the DPR decentralization result and to the Contriever/Llama reversals. The linear probing results alone do not measure knowledge reorganization; they measure discriminative capacity. As written, the abstract's conclusion that fine-tuning "primarily adjusts neuron activation rather than reorganizing knowledge" and that this pattern is architecture-dependent outruns the evidence. The authors should either validate the method for dense retrieval or explicitly reframe the conclusions as conditional on the attribution method.
[§7 Key Findings] The Discussion states that IG analysis shows DPR fine-tuning increases intermediate-layer activations by 32–41%, but no quantitative table or confidence interval supports this number in the main text, and the figures show aggregate percentages without error bars. This quantitative claim should be either backed by a table with variance estimates or removed.

minor comments (5)

[Throughout] There are numerous typos and naming inconsistencies: 'Contriver' appears after RQ3, 'ReplLlama' and 'RepLlama' are used interchangeably, 'LLAMA' appears in Figure 8, and 'bert-based-unacsed' appears in the captions of Figures 2 and 4.
[References] The reference list contains obvious placeholder citations ('Dai and Other 2022', 'Hao and Coauthors 2021', 'Smith and Doe 2021', 'Lee and Kumar 2021', 'Sun et al. 2020') and inconsistent entries; these must be corrected before publication.
[§4.2.3] The description of the epoch/validation procedure ('within a epoch of 50') is unclear; please specify how the best linear-probing layer is selected and applied.
[Figure captions] Figures 2, 3, and 4 state that a t-test 'between each DPR model with bert-based-unacsed backbone model' is marked with *; for the Contriever and Llama comparisons there is no DPR model, so the caption wording should be corrected.
[§4.2.2] RepLlama is described as 'castorini/repllama-v1-7b-lora-passage'; please clarify whether the comparison uses the LoRA adapter or the merged model and how the Llama backbone is run.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study's claims are derived from external model checkpoints, datasets, and independent analysis methods, not from fitted inputs or self-citation chains.

full rationale

The paper's derivation chain is not circular. Its conclusions are drawn from public pre-trained and fine-tuned checkpoints (BERT, DPR, Contriever, Llama, RepLlama) and public datasets (NQ, MS MARCO), and no parameter is fit to the target conclusion. The linear probing and integrated-gradient attribution pipelines are external methodological tools (Sundararajan et al., 2017; Dai et al., 2022), and Reichman and Heck (2024) is a separate prior work rather than a self-citation that carries the argument. The reproduction of the DPR pattern and the observed non-universality in Contriever and RepLlama are empirical comparisons across the same measurement pipeline, so they are not equivalent to the inputs by construction. Two caveats are validity concerns rather than circularity: Eq. (4) does not specify what the scalar model output P_x is for models whose output is an embedding, and the Llama activation threshold was chosen post hoc because it 'provided the clearest trend' (Section 5.6, footnote 10). The paper also concedes in Section 5.7 that neuron attribution 'may actually be not reliable for investigating dense retrieval.' These caveats substantially weaken the central claim, but they do not make the derivation reduce to its own inputs, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. Its claims depend on a handful of free parameters, most importantly the neuron-activation threshold (with the Llama threshold chosen to produce a trend), and on the domain assumption that linear probing and integrated-gradient attribution validly measure knowledge in retrieval models. The authors explicitly flag the latter as unvalidated for dense retrieval.

free parameters (4)

IG activation threshold (0.1 for BERT/DPR, 0.01 for Llama) = 0.1 / 0.01
Threshold for defining 'active' neurons; the Llama threshold was chosen post hoc to produce the clearest trend (Section 5.6, footnote 10). The central decentralization comparison depends on this threshold.
Number of integration steps nsteps = 20
Approximation of integrated gradients; standard but arbitrary, and may affect attribution quality.
Linear probing hyperparameters (learning rate, batch size, max epochs, train/val ratio, seed) = 1e-4, 32768, 30, 0.99, 42
Training setup for probes; not tuned per model, may affect accuracy levels and the observed gaps between models.
Hard negative sampling procedure = random sample 4, oversample if fewer, remove if none
Data construction choice; differs from the original paper and may explain the 10-15% lower accuracy reported.

assumptions (4)

domain assumption Integrated gradients attributed to individual neuron weights measure the contribution of that neuron to the model's output
Central to the knowledge decentralization analysis; the authors themselves doubt its reliability for dense retrieval (Section 5.7).
domain assumption Linear probing accuracy on the relevant-vs-hard-negative classification task reflects the model's retrieval knowledge or discriminative capacity
Used to support the claim about pre-trained knowledge underpinning retrieval performance; the proxy task is not the actual retrieval task.
domain assumption The BERT-CLS / BERT-Mean / Llama base models are appropriate untrained baselines that isolate the effect of dense retrieval fine-tuning
The comparison assumes the only difference is the fine-tuning recipe; but model variants differ in architecture weights and input length handling.
standard math Standard mathematical tools (integration, Riemann sum approximation, linear classifiers) are used correctly
Background math assumed in Sections 4 and 5.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition." pith.science (2026). https://pith.science/paper/HHSF3FPM

@misc{pith2026250507166,
  author       = {Pith},
  title        = {Pith review of: Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HHSF3FPM}},
  note         = {Machine review of arXiv:2505.07166}
}

read the original abstract

Dense retrievers utilize pre-trained backbone language models (e.g., BERT, LLaMA) that are fine-tuned via contrastive learning to perform the task of encoding text into sense representations that can be then compared via a shallow similarity operation, e.g. inner product. Recent research has questioned the role of fine-tuning vs. that of pre-training within dense retrievers, specifically arguing that retrieval knowledge is primarily gained during pre-training, meaning knowledge not acquired during pre-training cannot be sub-sequentially acquired via fine-tuning. We revisit this idea here as the claim was only studied in the context of a BERT-based encoder using DPR as representative dense retriever. We extend the previous analysis by testing other representation approaches (comparing the use of CLS tokens with that of mean pooling), backbone architectures (encoder-only BERT vs. decoder-only LLaMA), and additional datasets (MSMARCO in addition to Natural Questions). Our study confirms that in DPR tuning, pre-trained knowledge underpins retrieval performance, with fine-tuning primarily adjusting neuron activation rather than reorganizing knowledge. However, this pattern does not hold universally, such as in mean-pooled (Contriever) and decoder-based (LLaMA) models. We ensure full reproducibility and make our implementation publicly available at https://github.com/ielab/DenseRetriever-Knowledge-Acquisition.

Figures

Figures reproduced from arXiv: 2505.07166 by the authors.

**Figure 2.** Generalization of linear probing experiments to MS MARCO. The results are based on four different [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Generalization of Linear Probing experiment when different mean polling strategy is used (Con [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Generalization of Linear Probing Accuracy when decoder model backbone is used (Llama), EOS [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Results for knowledge neuron activation comparisons obtained on the NQ dataset. The plots [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Results for knowledge neuron activation comparisons obtained on the MS MARCO dataset. The [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 7.** Figure 7: Results for knowledge neuron activation comparisons obtained on the NQ dataset. The plots [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

**Figure 8.** Figure 8: Results for knowledge neuron activation comparisons obtained on the NQ dataset. The plots [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 11 canonical work pages

[4]

Pruning-based methods in deep neural networks: A review

Jane Hao and Coauthors. Pruning-based methods in deep neural networks: A review. In Proceedings of the 2021 AAAI Conference on Artificial Intelligence,

work page 2021
[5]

Contriever: A fully unsupervised dense retriever

Gautier Izacard and Edouard Grave. Contriever: A fully unsupervised dense retriever. In Proceedings of the 2021 Conference on Neural Information Processing Systems (NeurIPS),

work page 2021
[6]

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),

work page 2020
[7]

Natural questions: A benchmark for question answering

Tom Kwiatkowski, Jenna Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, David Epstein, Yury Filatov, Daniel Khashabi, Ashish Sabharwal, et al. Natural questions: A benchmark for question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP),

work page 2019
[10]

Sentence-bert: Sentence embeddings using siamese bert-networks

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing , pages 3980–3990,

work page 2019
[11]

Replama: A decoder-based dense retriever for open-domain question answering

John Smith and Jane Doe. Replama: A decoder-based dense retriever for open-domain question answering. In Proceedings of the 2021 Conference on Information Retrieval (SIGIR),

work page 2021
[13]

2d matryoshka training for information retrieval

Shuai Wang, Shengyao Zhuang, Bevan Koopman, and Guido Zuccon. 2d matryoshka training for information retrieval. arXiv preprint arXiv:2411.17299,

arXiv
[14]

Prompt-based representations for enhanced dense retrieval

Qiang Zhuang et al. Prompt-based representations for enhanced dense retrieval. In Proceedings of EMNLP arXiv:2404.18424, 2024a. Shengyao Zhuang, Shuai Wang, Bevan Koopman, and Guido Zuccon. Starbucks: Improved training for 2d matryoshka embeddings. arXiv preprint arXiv:2410.13230, 2024b. 22

arXiv

Show all 14 references

[2017]

Lecture notes on neural information retrieval

Nicola Tonellotto. Lecture notes on neural information retrieval. arXiv preprint arXiv:2207.13443,

arXiv
[2019]

Promptreps: Enhancing dense retrieval with prompt-based representations

Alex Lee and Rahul Kumar. Promptreps: Enhancing dense retrieval with prompt-based representations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP),

2021
[2020]

Knowledge neurons in pre-trained transformers

Zhiyong Dai and Coauthors Other. Knowledge neurons in pre-trained transformers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP),

2022
[2021]

Transformer feed-forward layers are key-value memories

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5484–5495,

2021
[2022]

Simcse: Simple contrastive learning of sentence embeddings

19 Tianyu Gao, Xing Yao, and Dan Chen. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,

2021
[2024]

Pre-training for ad-hoc retrieval: hyperlink is also you need

Zhengyi Ma, Zhicheng Dou, Wei Xu, Xinyu Zhang, Hao Jiang, Zhao Cao, and Ji-Rong Wen. Pre-training for ad-hoc retrieval: hyperlink is also you need. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 1212–1221, 2021b. Thang Nguy...

2016

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [4]

Pruning-based methods in deep neural networks: A review

Jane Hao and Coauthors. Pruning-based methods in deep neural networks: A review. In Proceedings of the 2021 AAAI Conference on Artificial Intelligence,

work page 2021

[2] [5]

Contriever: A fully unsupervised dense retriever

Gautier Izacard and Edouard Grave. Contriever: A fully unsupervised dense retriever. In Proceedings of the 2021 Conference on Neural Information Processing Systems (NeurIPS),

work page 2021

[3] [6]

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),

work page 2020

[4] [7]

Natural questions: A benchmark for question answering

Tom Kwiatkowski, Jenna Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, David Epstein, Yury Filatov, Daniel Khashabi, Ashish Sabharwal, et al. Natural questions: A benchmark for question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP),

work page 2019

[5] [10]

Sentence-bert: Sentence embeddings using siamese bert-networks

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing , pages 3980–3990,

work page 2019

[6] [11]

Replama: A decoder-based dense retriever for open-domain question answering

John Smith and Jane Doe. Replama: A decoder-based dense retriever for open-domain question answering. In Proceedings of the 2021 Conference on Information Retrieval (SIGIR),

work page 2021

[7] [13]

2d matryoshka training for information retrieval

Shuai Wang, Shengyao Zhuang, Bevan Koopman, and Guido Zuccon. 2d matryoshka training for information retrieval. arXiv preprint arXiv:2411.17299,

arXiv

[8] [14]

Prompt-based representations for enhanced dense retrieval

Qiang Zhuang et al. Prompt-based representations for enhanced dense retrieval. In Proceedings of EMNLP arXiv:2404.18424, 2024a. Shengyao Zhuang, Shuai Wang, Bevan Koopman, and Guido Zuccon. Starbucks: Improved training for 2d matryoshka embeddings. arXiv preprint arXiv:2410.13230, 2024b. 22

arXiv