Pith. sign in

REVIEW 3 major objections 6 minor 74 references

Latent reasoning steps supervised to reconstruct explicit reasoning beat prior retrievers on hard queries.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 07:45 UTC pith:JO6CXU7B

load-bearing objection A plausible latent-reasoning retriever with real in-domain gains, but the 'consistently outperforms' claim is not supported by the OOD numbers or the absence of error bars. the 3 major comments →

arxiv 2607.21333 v1 pith:JO6CXU7B submitted 2026-07-23 cs.IR

SHIFT: Self-reconstruction Harnesses Implicit Fine-grained Thinking for Retrieval

Yuxiao Luo , Da Li , Mingjie Zhang , Zhentao He , Shikun Zhang , Wei Ye This is my paper
classification cs.IR
keywords latent reasoningdense retrievalLLM-based retrievernext-token predictionself-reconstructioncontrastive learningimplicit chain-of-thoughtreasoning-intensive retrieval
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the reason implicit-reasoning retrievers underperform is not that latent reasoning is ineffective, but that the latent thinking states are both poorly adapted for retrieval and poorly supervised. It proposes SHIFT, which converts an LLM into a retriever that, for each query, generates three continuous 'thinking' tokens in latent space, transforms them through a residual projection, and pools them with bidirectional attention. The key move is a training objective: instead of aligning latent states to explicit chain-of-thought (which the paper shows collapses them into near-identical vectors), SHIFT expands each latent step into token-level representations and trains the model to predict the next explicit reasoning token. On reasoning-intensive benchmarks, this yields consistent gains over dense retrievers, rewrite-then-retrieve pipelines, and prior latent-reasoning methods (for example, 31.0 versus 29.3 average nDCG@10 on Bright for the 8B model). The claim matters because it suggests a cheap way to inject reasoning into retrieval without paying the inference cost of generating explicit text.

Core claim

SHIFT establishes that the bottleneck in latent-reasoning retrievers is a supervision and representation mismatch, not the idea of reasoning in latent space itself. The paper identifies that causal LLM hidden states are anisotropic and generation-oriented, so directly using them or aligning them to teacher reasoning states makes the supposed reasoning steps collapse into redundant embeddings. SHIFT instead applies a residual projection to map each latent step into a retrieval-oriented space, uses bidirectional multi-head attention to dynamically aggregate the K steps (the best step varies across tasks), and trains the model with a fine-grained next-token-prediction reconstruction loss: each

What carries the argument

The central mechanism is fine-grained self-reconstruction: the model's K latent thinking states are expanded to token-level representations (each state repeated l_k times to match the length of its explicit reasoning segment) and supervised by a next-token-prediction loss over the explicit reasoning trace. This is paired with a residual projection that strips generation-oriented anisotropy from hidden states and a bidirectional attention pooling that dynamically weights the K steps rather than assuming the last step is best. Together they resolve the two mismatches the paper names: representation mismatch (causal hidden states are not retrieval-shaped) and supervision mismatch (contrastive l

Load-bearing premise

The method assumes that next-token prediction over an external generator's explicit reasoning traces is a faithful supervision signal for what latent retrieval reasoning should encode.

What would settle it

Train SHIFT with the exact same setup but replace the explicit reasoning traces with shuffled or meaningless token sequences while keeping the rest of the pipeline unchanged. If retrieval performance on Bright does not drop materially, the reconstruction loss is not doing the claimed shaping work.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Implicit reasoning can replace explicit query rewriting or rationales at inference time, cutting latency while matching or beating explicit pipelines.
  • Shaping latent states through token-level reconstruction is a viable alternative to direct alignment with a teacher, avoiding the anisotropy collapse observed in prior methods.
  • Adaptive bidirectional pooling is necessary because the most informative reasoning step varies across tasks; a fixed monotonic weighting loses information.
  • Document-side latent reasoning is unnecessary; restricting reasoning to the query side reduces indexing cost without harming effectiveness.
  • With only three latent steps, the added compute over a standard LLM retriever is modest, making reasoning-intensive retrieval practical.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reconstruction loss effectively distills the reasoning ability of the external generator into the retriever's latent space; a testable implication is that degrading the quality of the explicit reasoning traces should degrade retrieval proportionally, pinpointing the teacher as the main lever.
  • Because the method uses a fixed K=3, adaptive latent-step allocation or query-dependent early stopping could further improve efficiency and effectiveness; the paper itself flags this as future work.
  • The bidirectional attention pooling can be seen as learned routing across reasoning modes; probing attention weights might reveal interpretable query types that favor different latent steps.
  • The NTP loss couples retrieval quality to the backbone's language-model head and embedding matrix, suggesting these components matter more than in standard contrastive retrievers and could be tuned separately.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes SHIFT, an LLM-based dense retriever that performs implicit latent reasoning before producing query embeddings. The method generates K=3 continuous latent 'thinking' tokens via a soft vocabulary distribution, transforms the resulting hidden states with a residual projection and bidirectional multi-head attention pooling, and trains with a contrastive InfoNCE loss combined with a fine-grained next-token-prediction self-reconstruction loss. The reconstruction loss expands step-level latent states to reconstruct explicit reasoning traces generated by GPT-4o-mini. Experiments on Bright, FollowIR, and BrowseComp-Plus compare SHIFT with dense retrievers, rewrite-then-retrieve pipelines, and latent-reasoning baselines GIRCSE and LaSER, and include ablations of the proposed components. The authors claim that SHIFT consistently outperforms other retrievers.

Significance. If the claimed gains are robust, SHIFT is a useful step in latent-reasoning retrieval: it directly attacks the representation-supervision mismatch with a concrete, well-specified architecture, and its analysis showing that LaSER's latent steps collapse to near-identical vectors (pairwise cosine similarity ≈ 1.0) is a valuable diagnostic. The method is clearly specified with explicit formulas, and the ablation table covers the main components. However, the evidence for the headline claim is limited by single-run results, mixed out-of-domain numbers, and some overinterpretation of component ablations. The contribution is promising rather than fully established; with additional statistical rigor and a rewording of the central claim, it could become a solid paper for the retrieval community.

major comments (3)
  1. [§5, Table 4, Abstract] The abstract and conclusion claim that SHIFT 'consistently outperforms' other retrievers, but the paper's own out-of-distribution results contradict this. For Qwen3-8B on BrowseComp-Plus, SHIFT achieves R@1000=66.8, below GIRCSE's 68.1 and LaSER's 66.9; for LLaMA3.1-8B, SHIFT's R@1000=52.7 is below LaSER's 52.9. On FollowIR, which contains only 104 queries, the Qwen3-8B advantage over LaSER is p-MRR 2.5 vs 2.4, within plausible noise. No error bars, multiple-seed runs, or significance tests are reported anywhere, so even the Bright improvement (31.0 vs 29.3) cannot be confidently distinguished from variance. The central claim of the paper therefore needs to be reworded or supported with interval estimates and a variance-aware evaluation.
  2. [§6.1, Table 3] Several component-level claims are stronger than the ablation data. §6.4 states that mean pooling 'introduces redundancy' and is suboptimal, but mean pooling achieves FollowIR p-MRR 4.5 vs SHIFT's 1.6 and comparable BrowseComp-Plus R@5 (6.8 vs 6.9). §6.6 states that document-side reasoning is 'a waste,' but the 'w Document Reasoning' row improves FollowIR Score (11.7 vs 11.3) and p-MRR (2.0 vs 1.6) and matches R@1000 (56.2 vs 56.1). The overall Bright result favors SHIFT, but these claims should be qualified to specific metrics and benchmarks, or the experiments extended to resolve the inconsistency.
  3. [§3.6, Eq. (13), Limitations] The reconstruction loss is the key novelty, yet all experiments train on a single reasoning dataset (ReasonEmbed) with GPT-4o-mini generated traces. The Limitations section acknowledges that the quality, style, and domain coverage of these traces can affect the learned latent space, but no experiment varies the reasoning source, length, or style, nor examines the effect of noisy or low-quality CoT on retrieval performance. As a result, the claimed general superiority cannot be separated from properties of this particular training signal. A sensitivity analysis with alternative CoT generators, controlled noise, or truncated reasoning traces would materially strengthen the central claim.
minor comments (6)
  1. [§1, Contributions] The second contribution bullet promises 'theoretical analysis,' but no theorem, proof, or formal analysis appears in the manuscript. Either provide the analysis or remove the claim.
  2. [§3.6, Eq. (13)] The conditioning variable I is never defined. Clarify whether it denotes the instruction prompt shown in Appendix A or something else.
  3. [Table 3] The row 'Casual MHA Pooling' should read 'Causal.' Table 2 also contains formatting/spacing issues (e.g., the Qwen3-Embedding rows).
  4. [Figure 2] The caption 'Relevant nDCG@10 ratio of SHIFT between LaSER' is grammatically unclear; it should be reformulated, e.g., 'ratio of SHIFT to LaSER.'
  5. [Table 4] Several p-MRR values are negative. The paper should state explicitly what the pairwise p-MRR metric is and whether negative values are expected.
  6. [§6.3, Figure 4] The text says the performance similarity 'is caused by' the indistinguishability of latent states, but the evidence is correlational. The causal wording should be softened unless a controlled experiment is added.

Circularity Check

0 steps flagged

No significant circularity: SHIFT's training objectives and benchmark evaluations are externally grounded.

full rationale

I walked SHIFT's claimed derivation chain. The method trains latent reasoning states with a contrastive InfoNCE loss over retrieval pairs (Eq. 9) and a fine-grained next-token-prediction reconstruction loss whose supervision targets are explicit reasoning traces from ReasonEmbed, generated by GPT-4o-mini (Eq. 13). These are training objectives, not fitted predictions of the reported evaluation numbers. The retrieval benchmarks are external test sets (Bright, FollowIR, BrowseComp-Plus) and no benchmark labels are used as training inputs. The residual projection and bidirectional attention pooling are architectural choices supported by ablation experiments (Table 3) rather than by definition or by citing prior work. A few cited references (CREM, Compressing then Matching, Chen et al. 2026a) share authors with this paper, but they appear only as contextual related work and are not load-bearing: the central claim rests on empirical comparisons in Tables 2 and 4 and on ablations, not on a self-citation chain. The Limitations section's admission that training depends on the quality/style of GPT-4o-mini reasoning traces is a data-quality assumption, not a circular definition. The abstract's 'consistently outperforms' is weakened by the paper's own mixed OOD numbers, but that is an evidence-strength / correctness concern, not circularity. No step in the derivation reduces by construction to its own input.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The central method relies on learned components (residual projection, bi-attention, 2D position MLP) with standard hyperparameters. No new physical or mathematical entities are introduced. The main external dependence is the ReasonEmbed dataset's GPT-4o-mini-generated reasoning traces, whose quality is acknowledged as a limitation.

free parameters (4)
  • K (number of latent thinking steps) = 3
    Fixed to 3 for training/inference (Section 4.3); chosen by hand, affects quality/efficiency; no sweep reported.
  • temperature tau for InfoNCE = 0.02
    Set in Section 4.3; standard value, not tuned here.
  • reconstruction loss weight = 1.0 (implicit)
    Combined loss L = L_cl + L_ntp (Eq. 14) with no weighting coefficient; assumes both losses contribute equally.
  • LoRA rank/alpha = r=64, alpha=32
    Section 4.3; chosen without reported ablation.
axioms (3)
  • domain assumption Soft latent token t_j = softmax(W_lm h_{j-1})^T E is a differentiable surrogate for discrete reasoning that preserves semantics.
    Section 3.3, Eq. 3; assumes the embedding expectation adequately represents a reasoning step and that causal LM hidden states can be recycled for retrieval.
  • domain assumption Explicit reasoning traces in ReasonEmbed (generated by GPT-4o-mini) provide useful supervision for latent reasoning.
    Section 3.6, Eq. 13; if the traces are noisy or style-shifted, the NTP reconstruction may shape the latent space incorrectly.
  • domain assumption Bidirectional attention pooling is appropriate for latent reasoning states without a temporal-ordering prior.
    Section 3.5; the paper assumes later reasoning steps are not always more useful and that bidirectional attention is preferable to causal or fixed weighting.

pith-pipeline@v1.3.0-alltime-deepseek · 15701 in / 14342 out tokens · 134469 ms · 2026-08-01T07:45:41.815595+00:00 · methodology

0 comments
read the original abstract

LLM-based retrievers have become a fundamental component of modern information retrieval systems. The paradigm of "rewrite-then-retriev" introduces explicit reasoning before retrieval. In addition, implicit-reasoning retrievers such as GIRCSE and LaSER improve efficiency by replacing explicit reasoning with soft tokens. Although these methods demonstrated competitive performance on reasoning-intensive retrieval benchmarks, they struggle to address the mismatch between the objectives of retrieval and generation. In this work, we propose SHIFT ($\textbf{S}$elf-reconstruction $\textbf{H}$arnesses $\textbf{I}$mplicit $\textbf{F}$ine-grained $\textbf{T}$hinking for Retrieval), a retrieval training framework based on LLMs. Firstly, we transfer LLMs into reasoning-efficient retrievers with residual projection and task-oriented bidirectional attention aggregation in the latent space. Secondly, we alleviate the mismatch between contrastive learning and implicit reasoning using fine-grained next-token-prediction-based reconstruction. Extensive experiments on reasoning-intensive retrieval benchmarks show that SHIFT consistently outperforms other widely used retrievers. We also carried out a detailed analysis to illustrate how our method works.

Figures

Figures reproduced from arXiv: 2607.21333 by Da Li, Mingjie Zhang, Shikun Zhang, Wei Ye, Yuxiao Luo, Zhentao He.

Figure 1
Figure 1. Figure 1: Main architecture of SHIFT. During training, we supervise the latent representations by both [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Similarity between latent representations. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The average similarity of the last hidden states [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 2
Figure 2. Figure 2: Relevant nDCG@10 ratio of SHIFT between LaSER. The numbers 1, 2, 3 above the bars are the token index, and p means pooled. We then discovered that the performance simi￾larity is caused by the indistinguishability of latent states: their pairwise cosine similarities are close to 1, as shown in [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 5
Figure 5. Figure 5: The similarity between the final pooled rep [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The similarity between the final pooled rep [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

74 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    2011 , publisher=

    Thinking, Fast and Slow , author=. 2011 , publisher=

    2011

  2. [2]

    2026 , eprint=

    Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding , author=. 2026 , eprint=

    2026

  3. [3]

    2026 , eprint=

    Reconstructing Content via Collaborative Attention to Improve Multimodal Embedding Quality , author=. 2026 , eprint=

    2026

  4. [4]

    Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning , publisher =

    Chen, Xinghao and Zhao, Anhao and Xia, Heming and Lu, Xuan and Wang, Hanlin and Chen, Yanjun and Zhang, Wei and Wang, Jian and Li, Wenjie and Shen, Xiaoyu , keywords =. Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning , publisher =. 2025 , copyright =. doi:10.48550/ARXIV.2505.16782 , url =

    doi:10.48550/arxiv.2505.16782 2025

  5. [5]

    2025 , eprint=

    Training Large Language Models to Reason in a Continuous Latent Space , author=. 2025 , eprint=

    2025

  6. [6]

    arXiv preprint arXiv:2409.08561 , year =

    Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding , author =. arXiv preprint arXiv:2409.08561 , year =. doi:10.48550/arXiv.2409.08561 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.08561

  7. [7]

    CODI : Compressing Chain-of-Thought into Continuous Space via Self-Distillation

    Shen, Zhenyi and Yan, Hanqi and Zhang, Linhai and Hu, Zhanghao and Du, Yali and He, Yulan. CODI : Compressing Chain-of-Thought into Continuous Space via Self-Distillation. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.36

    doi:10.18653/v1/2025.emnlp-main.36 2025

  8. [8]

    arXiv preprint arXiv:2502.17416 , year =

    Reasoning with Latent Thoughts: On the Power of Looped Transformers , author =. arXiv preprint arXiv:2502.17416 , year =. doi:10.48550/arXiv.2502.17416 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.17416

  9. [9]

    Soft Thinking: Unlocking the Reasoning Potential of

    Zhang, Zhen and He, Xuehai and Yan, Weixiang and Shen, Ao and Zhao, Chenyang and Wang, Shuohang and Shen, Yelong and Wang, Xin Eric , journal =. Soft Thinking: Unlocking the Reasoning Potential of. 2025 , doi =

    2025

  10. [10]

    Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains , url =

    Tan, Wenhui and Li, Jiaze and Ju, Jianzhong and Luo, Zhenbo and Song, Ruihua and Luan, Jian , booktitle =. Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains , url =

  11. [11]

    arXiv preprint arXiv:2602.08220 , year =

    Pretraining with Token-Level Adaptive Latent Chain-of-Thought , author =. arXiv preprint arXiv:2602.08220 , year =. doi:10.48550/arXiv.2602.08220 , url =

    doi:10.48550/arxiv.2602.08220

  12. [12]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =

    Reinforced Query Reasoners for Reasoning-intensive Retrieval Tasks , author =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =. doi:10.18653/v1/2025.emnlp-main.1078 , pages =

    doi:10.18653/v1/2025.emnlp-main.1078 2025

  13. [13]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =

    Improving Context Fidelity via Native Retrieval-Augmented Reasoning , author =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =. doi:10.18653/v1/2025.emnlp-main.1075 , pages =

    doi:10.18653/v1/2025.emnlp-main.1075 2025

  14. [14]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =

    Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking , author =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , month = nov, year =. doi:10.18653/v1/2025.emnlp-main.1214 , pages =

    doi:10.18653/v1/2025.emnlp-main.1214 2025

  15. [15]

    2026 , eprint=

    LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrieval , author=. 2026 , eprint=

    2026

  16. [16]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , url =

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and ichter, brian and Xia, Fei and Chi, Ed and Le, Quoc V and Zhou, Denny , booktitle =. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , url =

  17. [17]

    Think before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation , year=

    Tang, Jiakai and Dai, Sunhao and Shi, Teng and Xu, Jun and Chen, Xu and Chen, Wen and Wu, Jian and Jiang, Yuning , journal=. Think before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation , year=

  18. [18]

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics , year=

    Anisotropy Is Inherent to Self-Attention in Transformers , author=. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics , year=

  19. [19]

    Is Anisotropy Really the Cause of

    Fuster-Baggetto, Alejandro and Fresno, Victor , booktitle=. Is Anisotropy Really the Cause of. 2022 , publisher=

    2022

  20. [20]

    R etro MAE : Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder

    Xiao, Shitao and Liu, Zheng and Shao, Yingxia and Cao, Zhao. R etro MAE : Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.35

    doi:10.18653/v1/2022.emnlp-main.35 2022

  21. [21]

    Prefix-Tuning: Optimizing Continuous Prompts for Generation

    Li, Xiang Lisa and Liang, Percy. Prefix-Tuning: Optimizing Continuous Prompts for Generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021. doi:10.18653/v1/2021.acl-long.353

    doi:10.18653/v1/2021.acl-long.353 2021

  22. [22]

    Sentence-

    Reimers, Nils and Gurevych, Iryna , booktitle=. Sentence-. 2019 , publisher=

    2019

  23. [23]

    2023 , eprint=

    Implicit Chain of Thought Reasoning via Knowledge Distillation , author=. 2023 , eprint=

    2023

  24. [24]

    Proceedings of the 38th International Conference on Machine Learning , pages=

    Barlow Twins: Self-Supervised Learning via Redundancy Reduction , author=. Proceedings of the 38th International Conference on Machine Learning , pages=. 2021 , organization=

    2021

  25. [25]

    Bardes, Adrien and Ponce, Jean and LeCun, Yann , booktitle=

  26. [26]

    How Contextual are Contextualized Word Representations? C omparing the Geometry of BERT , ELM o, and GPT -2 Embeddings

    Ethayarajh, Kawin. How Contextual are Contextualized Word Representations? C omparing the Geometry of BERT , ELM o, and GPT -2 Embeddings. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019. doi:10.18653/v1/D19-1006

    doi:10.18653/v1/d19-1006 2019

  27. [27]

    International Conference on Learning Representations , year=

    Representation Degeneration Problem in Training Natural Language Generation Models , author=. International Conference on Learning Representations , year=

  28. [28]

    International Conference on Learning Representations , year=

    Improving Neural Language Generation with Spectrum Control , author=. International Conference on Learning Representations , year=

  29. [29]

    Information Retrieval meets Large Language Models: A strategic report from Chinese IR community , volume=

    Ai, Qingyao and Bai, Ting and Cao, Zhao and Chang, Yi and Chen, Jiawei and Chen, Zhumin and Cheng, Zhiyong and Dong, Shoubin and Dou, Zhicheng and Feng, Fuli and Gao, Shen and Guo, Jiafeng and He, Xiangnan and Lan, Yanyan and Li, Chenliang and Liu, Yiqun and Lyu, Ziyu and Ma, Weizhi and Ma, Jun and Ren, Zhaochun and Ren, Pengjie and Wang, Zhiqiang and Wan...

    2023

  30. [30]

    Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval , publisher =

    Chen, Hung-Ting and Liu, Xiang and Ravfogel, Shauli and Choi, Eunsol , keywords =. Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval , publisher =. 2025 , copyright =. doi:10.48550/ARXIV.2511.02770 , url =

    doi:10.48550/arxiv.2511.02770 2025

  31. [31]

    Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =

    Xiao, Shitao and Liu, Zheng and Zhang, Peitian and Muennighoff, Niklas and Lian, Defu and Nie, Jian-Yun , title =. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =. 2024 , isbn =. doi:10.1145/3626772.3657878 , abstract =

    arXiv 2024

  32. [32]

    2025 , eprint=

    M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation , author=. 2025 , eprint=

    2025

  33. [33]

    2026 , eprint=

    ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrieval , author=. 2026 , eprint=

    2026

  34. [34]

    2025 , eprint=

    BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent , author=. 2025 , eprint=

    2025

  35. [35]

    2026 , eprint=

    Think Then Embed: Generative Context Improves Multimodal Embedding , author=. 2026 , eprint=

    2026

  36. [36]

    RaDeR: Reasoning-aware Dense Retrieval Models , publisher =

    Das, Debrup and Nuallain, Sam O' and Rahimi, Razieh , keywords =. RaDeR: Reasoning-aware Dense Retrieval Models , publisher =. 2025 , copyright =. doi:10.48550/ARXIV.2505.18405 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.18405 2025

  37. [37]

    Precise Zero-Shot Dense Retrieval without Relevance Labels

    Gao, Luyu and Ma, Xueguang and Lin, Jimmy and Callan, Jamie. Precise Zero-Shot Dense Retrieval without Relevance Labels. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023. doi:10.18653/v1/2023.acl-long.99

    doi:10.18653/v1/2023.acl-long.99 2023

  38. [38]

    2026 , eprint=

    Search-R3: Unifying Reasoning and Embedding in Large Language Models , author=. 2026 , eprint=

    2026

  39. [39]

    arXiv preprint arXiv:2410.21276 , year =

    GPT-4o System Card , author =. arXiv preprint arXiv:2410.21276 , year =. doi:10.48550/arXiv.2410.21276 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.21276

  40. [40]

    2022 , note =

    Unsupervised dense information retrieval with contrastive learning , author =. 2022 , note =

    2022

  41. [41]

    Dense Passage Retrieval for Open-Domain Question Answering

    Karpukhin, Vladimir and Oguz, Barlas and Min, Sewon and Lewis, Patrick and Wu, Ledell and Edunov, Sergey and Chen, Danqi and Yih, Wen-tau. Dense Passage Retrieval for Open-Domain Question Answering. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020. doi:10.18653/v1/2020.emnlp-main.550

    doi:10.18653/v1/2020.emnlp-main.550 2020

  42. [42]

    BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019. doi:10.18653/v...

    doi:10.18653/v1/n19-1423 2019

  43. [43]

    2025 , note =

    NV-embed: improved techniques for training llms as generalist embedding models , author =. 2025 , note =

    2025

  44. [44]

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , url =

    Lewis, Patrick and Perez, Ethan and Piktus, Aleksandra and Petroni, Fabio and Karpukhin, Vladimir and Goyal, Naman and K\". Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , url =. Advances in Neural Information Processing Systems , editor =

  45. [45]

    Towards General Text Embeddings with Multi-stage Contrastive Learning , publisher =

    Li, Zehan and Zhang, Xin and Zhang, Yanzhao and Long, Dingkun and Xie, Pengjun and Zhang, Meishan , keywords =. Towards General Text Embeddings with Multi-stage Contrastive Learning , publisher =. 2023 , copyright =. doi:10.48550/ARXIV.2308.03281 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.03281 2023

  46. [46]

    Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation Learning

    Liang, Zhengyang and Liang, Meiyu and Huang, Wei and Li, Yawen and Xue, Zhe , keywords =. Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation Learning , publisher =. 2024 , copyright =. doi:10.48550/ARXIV.2404.10838 , url =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2404.10838 2024

  47. [47]

    ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability , publisher =

    Liu, Wenhan and Ma, Xinyu and Sun, Weiwei and Zhu, Yutao and Li, Yuchen and Yin, Dawei and Dou, Zhicheng , keywords =. ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability , publisher =. 2025 , copyright =. doi:10.48550/ARXIV.2508.07050 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.07050 2025

  48. [48]

    2025 , note =

    A comprehensive survey on knowledge distillation , author =. 2025 , note =

    2025

  49. [49]

    Generative Representational Instruction Tuning , publisher =

    Muennighoff, Niklas and Su, Hongjin and Wang, Liang and Yang, Nan and Wei, Furu and Yu, Tao and Singh, Amanpreet and Kiela, Douwe , keywords =. Generative Representational Instruction Tuning , publisher =. 2024 , copyright =. doi:10.48550/ARXIV.2402.09906 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.09906 2024

  50. [50]

    Answer is All You Need: Instruction-following Text Embedding via Answering the Question

    Peng, Letian and Zhang, Yuwei and Wang, Zilong and Srinivasa, Jayanth and Liu, Gaowen and Wang, Zihan and Shang, Jingbo. Answer is All You Need: Instruction-following Text Embedding via Answering the Question. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.27

    doi:10.18653/v1/2024.acl-long.27 2024

  51. [51]

    2025 , eprint=

    TongSearch-QR: Reinforced Query Reasoning for Retrieval , author=. 2025 , eprint=

    2025

  52. [52]

    ReasonIR: Training Retrievers for Reasoning Tasks , publisher =

    Shao, Rulin and Qiao, Rui and Kishore, Varsha and Muennighoff, Niklas and Lin, Xi Victoria and Rus, Daniela and Low, Bryan Kian Hsiang and Min, Sewon and Yih, Wen-tau and Koh, Pang Wei and Zettlemoyer, Luke , keywords =. ReasonIR: Training Retrievers for Reasoning Tasks , publisher =. 2025 , copyright =. doi:10.48550/ARXIV.2504.20595 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.20595 2025

  53. [53]

    BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval , url =

    SU, Hongjin and Yen, Howard and Xia, Mengzhou and Shi, Weijia and Muennighoff, Niklas and Wang, Han-yu and Haisu, Liu and Shi, Quan and Siegel, Zachary and Tang, Michael and Sun, Ruoxi and Yoon, Jinsung and Arik, Sercan and Chen, Danqi and Yu, Tao , booktitle =. BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval , url =

  54. [54]

    2025 , eprint=

    GRACE: Generative Representation Learning via Contrastive Policy Optimization , author=. 2025 , eprint=

    2025

  55. [55]

    2026 , eprint=

    Let LLMs Speak Embedding Languages: Generative Text Embeddings via Iterative Contrastive Refinement , author=. 2026 , eprint=

    2026

  56. [56]

    2024 , eprint=

    Text Embeddings by Weakly-Supervised Contrastive Pre-training , author=. 2024 , eprint=

    2024

  57. [57]

    Improving Text Embeddings with Large Language Models

    Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu. Improving Text Embeddings with Large Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.642

    doi:10.18653/v1/2024.acl-long.642 2024

  58. [58]

    Query2doc: Query Expansion with Large Language Models

    Wang, Liang and Yang, Nan and Wei, Furu. Query2doc: Query Expansion with Large Language Models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023. doi:10.18653/v1/2023.emnlp-main.585

    doi:10.18653/v1/2023.emnlp-main.585 2023

  59. [59]

    F ollow IR : Evaluating and Teaching Information Retrieval Models to Follow Instructions

    Weller, Orion and Chang, Benjamin and MacAvaney, Sean and Lo, Kyle and Cohan, Arman and Van Durme, Benjamin and Lawrie, Dawn and Soldaini, Luca. F ollow IR : Evaluating and Teaching Information Retrieval Models to Follow Instructions. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics...

    doi:10.18653/v1/2025.naacl-long.597 2025

  60. [60]

    E xpand R : Teaching Dense Retrievers Beyond Queries with LLM Guidance

    Yao, Sijia and Huang, Pengcheng and Liu, Zhenghao and Gu, Yu and Yan, Yukun and Yu, Shi and Yu, Ge. E xpand R : Teaching Dense Retrievers Beyond Queries with LLM Guidance. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.963

    work page doi:10.18653/v1/2025.emnlp-main.963 2025

  61. [61]

    2024 , eprint=

    Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding , author=. 2024 , eprint=

    2024

  62. [62]

    Jasper and Stella: distillation of SOTA embedding models , publisher =

    Zhang, Dun and Li, Jiacheng and Zeng, Ziyang and Wang, Fulong , keywords =. Jasper and Stella: distillation of SOTA embedding models , publisher =. 2024 , copyright =. doi:10.48550/ARXIV.2412.19048 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.19048 2024

  63. [63]

    2025 , eprint=

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models , author=. 2025 , eprint=

    2025

  64. [64]

    2025 , eprint=

    Your Dense Retriever is Secretly an Expeditious Reasoner , author=. 2025 , eprint=

    2025

  65. [65]

    A Survey of Large Language Models , publisher =

    Zhao, Wayne Xin and Zhou, Kun and Li, Junyi and Tang, Tianyi and Wang, Xiaolei and Hou, Yupeng and Min, Yingqian and Zhang, Beichen and Zhang, Junjie and Dong, Zican and Du, Yifan and Yang, Chen and Chen, Yushuo and Chen, Zhipeng and Jiang, Jinhao and Ren, Ruiyang and Li, Yifan and Tang, Xinyu and Liu, Zikang and Liu, Peiyu and Nie, Jian-Yun and Wen, Ji-R...

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.18223 2023

  66. [66]

    Large Language Models for Information Retrieval: A Survey , publisher =

    Zhu, Yutao and Yuan, Huaying and Wang, Shuting and Liu, Jiongnan and Liu, Wenhan and Deng, Chenlong and Chen, Haonan and Liu, Zheng and Dou, Zhicheng and Wen, Ji-Rong , keywords =. Large Language Models for Information Retrieval: A Survey , publisher =. 2023 , copyright =. doi:10.48550/ARXIV.2308.07107 , url =

    doi:10.48550/arxiv.2308.07107 2023

  67. [67]

    Learning Phrase Representations using RNN Encoder -- Decoder for Statistical Machine Translation

    Cho, Kyunghyun and van Merri. Learning Phrase Representations using RNN Encoder -- Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ). 2014. doi:10.3115/v1/D14-1179

    doi:10.3115/v1/d14-1179 2014

  68. [68]

    Attention is All you Need , url =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , url =

  69. [69]

    Proceedings of the 37th International Conference on Machine Learning , pages =

    Retrieval Augmented Language Model Pre-Training , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , editor =

    2020

  70. [70]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    Condenser: A Pre-training Architecture for Dense Retrieval , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=. 2021 , publisher=

    2021

  71. [71]

    Liu, Yinhan and Ott, Myle and Goyal, Naman and Du, Jingfei and Joshi, Mandar and Chen, Danqi and Levy, Omer and Lewis, Mike and Zettlemoyer, Luke and Stoyanov, Veselin , journal=

  72. [72]

    arXiv preprint arXiv:2309.12871 , year=

    AnglE-Optimized Text Embeddings , author=. arXiv preprint arXiv:2309.12871 , year=

    Pith/arXiv arXiv

  73. [73]

    S im CSE : Simple Contrastive Learning of Sentence Embeddings

    Gao, Tianyu and Yao, Xingcheng and Chen, Danqi. S im CSE : Simple Contrastive Learning of Sentence Embeddings. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021. doi:10.18653/v1/2021.emnlp-main.552

    doi:10.18653/v1/2021.emnlp-main.552 2021

  74. [74]

    2026 , eprint=

    CREM: Compression-Driven Representation Enhancement for Multimodal Retrieval and Comprehension , author=. 2026 , eprint=

    2026