Pith. sign in

REVIEW 2 major objections 5 minor 33 references

Merging an ad-hoc retriever with its conversational fine-tune restores both skills in one model without any further training.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 05:42 UTC pith:O7ZRWGG6

load-bearing objection Clean empirical demo that weight-space merging restores ad-hoc skill in conversational dense retrievers without retraining; solid subfield result, not a paradigm shift. the 2 major comments →

arxiv 2607.08540 v1 pith:O7ZRWGG6 submitted 2026-07-09 cs.IR cs.CL

Improving Ad-hoc Search Effectiveness for Conversational Information Retrieval via Model Merging

Ahmed Rayane Kebir , Jose G. Moreno , Lynda Tamine This is my paper
classification cs.IR cs.CL
keywords Conversational SearchInformation RetrievalModel MergingDense RetrievalCatastrophic ForgettingModel SoupSlerp
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Conversational retrieval systems usually start from a strong ad-hoc search model and then fine-tune it on multi-turn conversations. That specialization often erases the original single-query skill and is expensive to reverse with multi-task retraining. This paper shows that simply interpolating the weights of the original ad-hoc model with the conversational model produces a single retriever that recovers most of the lost ad-hoc performance while keeping conversational ability. The merge is training-free, works with both linear averaging and spherical interpolation, and yields models that also generalize better to unseen conversational and ad-hoc collections. The practical payoff is a dual-purpose dense retriever that does not require re-training on large ad-hoc data.

Core claim

Parameter-wise merging of a base ad-hoc dense retriever (ANCE) with its conversational fine-tuned counterpart (QRACDR) produces a single model whose effectiveness on both ad-hoc and conversational tasks lies between, and often exceeds, the two source models, eliminating catastrophic forgetting without any additional gradient updates.

What carries the argument

Depth-wise model merging (Model Soup linear interpolation or Slerp spherical interpolation) controlled by a short coefficient vector λ that blends the two weight sets layer by layer.

Load-bearing premise

The independently trained ad-hoc and conversational weight vectors already sit in a region of parameter space that can be usefully blended by a single depth-wise coefficient chosen only on in-domain data.

What would settle it

If depth-wise interpolation of the released ANCE and QRACDR checkpoints, using the same λ selection protocol restricted to MS MARCO and QReCC/TopiOCQA, fails to recover MS MARCO NDCG@3 or to improve CAsT session and rewrite scores relative to the pure conversational model, the central claim collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A single dense index can serve both classic keyword search and multi-turn conversational sessions without maintaining two models.
  • Catastrophic forgetting after conversational fine-tuning can be reversed post-hoc without access to the original ad-hoc training set.
  • Multi-task joint training becomes optional rather than mandatory when complementary skills already exist in separate checkpoints.
  • Out-of-domain generalization on both ad-hoc and conversational benchmarks can improve simply by restoring the base ad-hoc capacity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same merge recipe could be applied to any pair of dense retrievers specialized on different query styles (e.g., short vs long, keyword vs natural language) to obtain a more universal encoder.
  • If the optimal λ pattern proves stable across architectures, practitioners could publish a small set of recommended merge coefficients rather than full multi-task models.
  • Model merging may offer a cheap way to keep an evolving conversational system continuously compatible with pure ad-hoc traffic without periodic full retraining.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes model merging as a training-free alternative to multi-task fine-tuning for conversational information retrieval (CIR). Starting from a backbone ad-hoc dense retriever (ANCE) and its conversational fine-tuned variants (QRACDR-Q/T), the authors form a single model via depth-wise linear (Model Soup) or spherical (Slerp) interpolation of parameters (Eq. 1). The resulting merged models are evaluated on in-domain (MS MARCO, QReCC, TopiOCQA) and out-of-domain (CAsT-19/20, NQ, HotpotQA) sets. Results show that merging recovers most of the ad-hoc performance lost to catastrophic forgetting, remains competitive on conversational tasks, matches or exceeds multi-task learning and early-stopping baselines (Fig. 3), and yields up to +10.11% NDCG@3 gains on CAsT session retrieval under zero-shot conditions (Tables 1–2).

Significance. If the empirical findings hold, the work supplies a practical, low-cost remedy for the well-documented trade-off between conversational specialization and ad-hoc robustness. The approach requires no gradient updates, no access to the original large-scale ad-hoc training data, and produces a single dual-purpose retriever. Strengths include a clean experimental design (in-domain λ selection frozen for OOD evaluation), public MergeKit configurations, paired significance tests, and direct comparison against multi-task learning and early stopping. The contribution is incremental rather than foundational, yet it is the first systematic exploration of model merging for CIR and is immediately usable by practitioners.

major comments (2)
  1. §4.1 Merging Optimization and Tables 1–2: only a single pair of source models (ANCE + QRACDR) is examined. While the results are convincing for this family, the central claim that model merging is a general training-free strategy for CIR would be substantially stronger if at least one additional backbone (e.g., a different bi-encoder or a late-interaction model) were shown to exhibit the same interpolability. Without that, the generality of the interpolable-region assumption remains untested.
  2. Fig. 2 and §4.2.1: the paper reports that a large subset of QRACDR-Q merges improve QReCC itself, yet the opposite trend appears for TopiOCQA. The discussion attributes this to topic-shift frequency, but no quantitative analysis (e.g., average history length, coreference density, or layer-wise cosine similarity between θ_adh and θ_cir) is provided. A short diagnostic would clarify when positive transfer can be expected and would strengthen the interpretation of the λ★ vectors.
minor comments (5)
  1. Abstract and §1: the “up to 15% higher NDCG@3” claim is not directly traceable to a single table entry; the largest reported session gain is +10.11% (Table 2). Please align the abstract figure with the concrete numbers or clarify the comparison baseline.
  2. Eq. (1) and the Slerp formula: the notation for the depth-wise vector λ is introduced, yet the concrete λ★ vectors are given only later in prose. Placing the selected vectors next to Eq. (1) would improve readability.
  3. Table 1 header: the symbols δ_i and δ_f are defined in the caption but not in the table itself; a short legend row would help.
  4. §2 Related Work: a brief pointer to concurrent IR model-merging papers (e.g., domain adaptation via task arithmetic) would better situate the novelty claim.
  5. Fig. 3 caption: the vertical arrows are useful, but the exact early-stopping steps (100 / 3000) should also be marked on the x-axis for easier visual comparison.

Circularity Check

0 steps flagged

No significant circularity; purely empirical parameter interpolation evaluated on held-out data.

full rationale

The paper presents an empirical application of known model-merging techniques (Model Soup linear interpolation and Slerp spherical interpolation) to restore ad-hoc retrieval performance in conversational dense retrievers. Equation 1 simply defines the merged parameters as a depth-wise combination of two independently obtained checkpoints (ANCE and QRACDR) controlled by free coefficients λ; those coefficients are selected exclusively on in-domain sets (MS MARCO / QReCC / TopiOCQA) and then evaluated on strictly held-out OOD sets (CAsT, NQ, HotpotQA). No quantity is claimed to be derived or predicted from a fit of itself; the reported NDCG@3 gains (including the abstract’s “up to 15 %”) are measured experimental outcomes against independent baselines (ANCE, QRACDR, multi-task learning, early stopping). Self-citations are limited to the source QRACDR checkpoint (reproduced from public code of non-overlapping authors) and standard merging literature; none supply a uniqueness theorem or load-bearing premise. The derivation chain therefore contains no self-definitional step, no fitted-input-called-prediction, and no circular self-citation.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The central claim rests on the empirical compatibility of two independently fine-tuned checkpoints under linear/spherical interpolation, plus a small set of free interpolation coefficients chosen on in-domain data. No new physical or mathematical entities are postulated; background IR assumptions are standard.

free parameters (2)
  • depth-wise interpolation vector λ (Model Soup) = (1.0, 0.73, 0.47, 0.2)
    Selected by evaluating 40 configurations on in-domain MS MARCO / QReCC / TopiOCQA; final reported vector λ★_MS = (1.0, 0.73, 0.47, 0.2).
  • depth-wise interpolation vector λ (Slerp) = (0.5, 0.6, 0.6, 0.5)
    Selected analogously among 40 Slerp configurations; final λ★_SL = (0.5, 0.6, 0.6, 0.5).
axioms (3)
  • domain assumption Fine-tuned models for related tasks occupy compatible regions of parameter space that can be usefully interpolated (Matena & Raffel, Wortsman et al.).
    Invoked in §2 and §3 to justify applying Model Soup and Slerp to ANCE + QRACDR.
  • domain assumption Dense bi-encoder retrieval with ANCE-style contrastive training is a valid base for both ad-hoc and conversational tasks.
    Background of the entire experimental setup (§4.1 Models).
  • ad hoc to paper In-domain performance on MS MARCO / QReCC / TopiOCQA is a reliable proxy for selecting λ that will generalize to OOD CAsT / NQ / HotpotQA.
    Stated in Merging Optimization paragraph of §4.1; no theoretical guarantee is offered.

pith-pipeline@v1.1.0-grok45 · 16711 in / 2405 out tokens · 26252 ms · 2026-07-10T05:42:24.885312+00:00 · methodology

0 comments
read the original abstract

Conversational information retrieval is challenging since it requires the consideration of the conversation history which potentially gives rise to topic shifts and coreference resolution across previous turns. To address these challenges, previous work mainly rely on traditional fine-tuning of ad-hoc retrievers on conversational datasets or extrapolates their generalizability through multi-tasking. However, this mainstream approach is costly - since it requires model re-training - and exhibits catastrophic forgetting, where the model loses its foundational ad-hoc retrieval performance. In this paper, we fill this gap by introducing model merging as a training-free strategy enabling the design of a single retrieval model that operates across both ad-hoc and conversational settings with no additional fine-tuning. We conduct experiments using linear and non-linear parameter-wise merging strategies - namely Model Soup and Slerp - on standard ad-hoc search and conversational retrieval datasets. Our results demonstrate that model merging significantly enhances the ad-hoc search capabilities of conversational retrievers while improving generalizability across task-specific datasets, achieving up to 15% higher NDCG@3 under zero-shot conditions.

Figures

Figures reproduced from arXiv: 2607.08540 by Ahmed Rayane Kebir, Jose G. Moreno, Lynda Tamine.

Figure 1
Figure 1. Figure 1: Model merging of a base ad-hoc retriever and a conversa [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance of a total of 80 merged models (40 for MS [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: NDCG@3 performance across fine-tuning steps for Top [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages · 3 internal anchors

  1. [1]

    Vaibhav Adlakha, Shehzaad Dhuliawala, Kaheer Suleman, Harm de Vries, and Siva Reddy. 2022. Topiocqa: Open-domain conversational question answering with topic switching.Transactions of the Association for Computational Linguistics 10 (2022), 468–483

    work page 2022

  2. [2]

    Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, and Srinivas Chappidi. 2021. Open-domain question answering goes conversational via question rewriting. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 520–534

    work page 2021

  3. [3]

    Marco Braga, Pranav Kasela, Alessandro Raganato, and Gabriella Pasi. 2025. Investigating task arithmetic for zero-shot information retrieval. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2738–2743

    work page 2025

  4. [4]

    Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. 2020. CAsT 2019: The Conver- sational Assistance Track Overview. (2020)

    work page 2020

  5. [5]

    Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. 2021. Cast 2020: The conversa- tional assistance track overview

    work page 2021

  6. [6]

    2023.Neural approaches to conversational information retrieval

    Jianfeng Gao, Chenyan Xiong, Paul Bennett, and Nick Craswell. 2023.Neural approaches to conversational information retrieval. Vol. 44. Springer

    work page 2023

  7. [7]

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee’s mergekit: A toolkit for merging large language models.arXiv preprint arXiv:2403.13257(2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Han- naneh Hajishirzi, and Ali Farhadi. [n. d.]. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations

    work page

  9. [9]

    Young Kyun Jang, Dat Huynh, Ashish Shah, Wen-Kai Chen, and Ser-Nam Lim

    work page

  10. [10]

    InEuropean Conference on Computer Vision

    Spherical linear interpolation and text-anchoring for zero-shot composed image retrieval. InEuropean Conference on Computer Vision. Springer, 239–254

    work page

  11. [11]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics7 (2019), 453–466

    work page 2019

  12. [12]

    Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2021. Contextualized Query Embeddings for Conversational Search. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, Online and Punta Cana, Domin...

    work page 2021

  13. [13]

    doi:10.18653/v1/2021.emnlp-main.77

    work page doi:10.18653/v1/2021.emnlp-main.77 2021

  14. [14]

    Kelong Mao, Chenlong Deng, Haonan Chen, Fengran Mo, Zheng Liu, Tetsuya Sakai, and Zhicheng Dou. 2024. ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 1227–1240

    work page 2024

  15. [15]

    Kelong Mao, Zhicheng Dou, Fengran Mo, Jiewen Hou, Haonan Chen, and Hongjin Qian. 2023. Large language models know your contextual search intent: A prompting framework for conversational search.arXiv preprint arXiv:2303.06573 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  16. [16]

    Kelong Mao, Hongjin Qian, Fengran Mo, Zhicheng Dou, Bang Liu, Xiaohua Cheng, and Zhao Cao. 2023. Learning denoised and interpretable session repre- sentation for conversational search. InProceedings of the ACM Web Conference SIGIR ’26, July 20–24, 2026, Melbourne, VIC, Australia Ahmed Rayane Kebir, Jose G. Moreno, and Lynda Tamine

    work page 2023

  17. [17]

    Michael S Matena and Colin A Raffel. 2022. Merging models with fisher-weighted averaging.Advances in Neural Information Processing Systems35 (2022), 17703– 17716

    work page 2022

  18. [18]

    Fengran Mo, Abbas Ghaddar, Kelong Mao, Mehdi Rezagholizadeh, Boxing Chen, Qun Liu, and Jian-Yun Nie. 2024. CHIQ: Contextual history enhancement for im- proving query rewriting in conversational search.arXiv preprint arXiv:2406.05013 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Fengran Mo, Kelong Mao, Ziliang Zhao, Hongjin Qian, Haonan Chen, Yiruo Cheng, Xiaoxi Li, Yutao Zhu, Zhicheng Dou, and Jian-Yun Nie. 2025. A survey of conversational search.ACM Transactions on Information Systems43, 6 (2025), 1–50

    work page 2025

  20. [20]

    Fengran Mo, Chen Qu, Kelong Mao, Yihong Wu, Zhan Su, Kaiyu Huang, and Jian-Yun Nie. 2024. Aligning query representation with rewritten query and relevance judgments in conversational search. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management. 1700–1710

    work page 2024

  21. [21]

    Fengran Mo, Chen Qu, Kelong Mao, Tianyu Zhu, Zhan Su, Kaiyu Huang, and Jian-Yun Nie. 2024. History-Aware Conversational Dense Retrieval. InFindings of the Association for Computational Linguistics: ACL 2024. 13366–13378

    work page 2024

  22. [22]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human-generated machine reading comprehension dataset. (2016)

    work page 2016

  23. [23]

    Filip Radlinski and Nick Craswell. 2017. A Theoretical Framework for Conversa- tional Search. InProceedings of the 2017 Conference on Conference Human Infor- mation Interaction and Retrieval(Oslo, Norway)(CHIIR ’17). Association for Com- puting Machinery, New York, NY, USA, 117–126. doi:10.1145/3020165.3020183

    work page doi:10.1145/3020165.3020183 2017

  24. [24]

    Taiga Sasaki, Takehiro Yamamoto, Hiroaki Ohshima, and Sumio Fujita. 2025. Effect of Model Merging in Domain-Specific Ad-hoc Retrieval. InProceedings of the 34th ACM International Conference on Information and Knowledge Management. 5208–5212

    work page 2025

  25. [25]

    Seungjong Sun, Seo Yeon Baek, and Jang Hyun Kim. 2025. Personality Vec- tor: Modulating Personality of Large Language Models by Model Merging. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Lingui...

    work page doi:10.18653/v1/2025.emnlp-main.1253 2025

  26. [26]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. [n. d.]. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. ([n. d.])

    work page

  27. [27]

    Christophe Van Gysel and Maarten de Rijke. 2018. Pytrec_eval: An extremely fast python interface to trec_eval. InThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 873–876

    work page 2018

  28. [28]

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Si- mon Kornblith, et al. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. InInternational conference on machine learning. PMLR, 23965–23998

    work page 2022

  29. [29]

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, and Arnold Overwijk. [n. d.]. Approximate Nearest Neighbor Neg- ative Contrastive Learning for Dense Text Retrieval. InInternational Conference on Learning Representations

    work page

  30. [30]

    Seunghan Yang, Juntae Lee, Jihwan Bang, Kyuhong Shim, Minsoo Kim, and Simyung Chang. 2025. Learning Contextual Retrieval for Robust Conversational Search. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Car- olyn Rose, and Violet Peng (Eds.). Association for Computa...

    work page doi:10.18653/v1/2025.emnlp-main.602 2025

  31. [31]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. InProceedings of the 2018 conference on empirical methods in natural language processing. 2369–2380

    work page 2018

  32. [32]

    Shi Yu, Zhenghao Liu, Chenyan Xiong, Tao Feng, and Zhiyuan Liu. 2021. Few- shot conversational dense retrieval. InProceedings of the 44th International ACM SIGIR Conference on research and development in information retrieval. 829–838

    work page 2021

  33. [33]

    Yuhang Zhou, Giannis Karamanolakis, Victor Soto, Anna Rumshisky, Mayank Kulkarni, Furong Huang, Wei Ai, and Jianhua Lu. 2025. MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Vo...

    work page doi:10.18653/v1/2025.naacl- 2025