pith. machine review for the scientific record. sign in

arxiv: 2604.23336 · v2 · submitted 2026-04-25 · 💻 cs.IR · cs.CL· cs.LG

Recognition: 2 theorem links

· Lean Theorem

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA

Teng Chen , Sheng Xu , Feixiang Guo , Xiaoyu Wang , Qingqing Gu , Hongyan Li , Luo Ji
Authors on Pith no claims yet

Pith reviewed 2026-05-14 21:06 UTC · model grok-4.3

classification 💻 cs.IR cs.CLcs.LG
keywords rationale-based retrievalon-policy distillationgenerative rerankerJEPAefficient retrievalembedding alignmentLLM distillationcross-encoder replacement
0
0 comments X

The pith

Rabtriever distills a generative reranker into an independent encoder that matches cross-encoding comprehension at linear complexity.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that a student retriever can independently encode queries and documents yet reconstruct the contextual query embeddings produced by a full cross-encoding generative reranker. It achieves this through on-policy distillation guided by a Joint-Embedding Predictive Architecture that aligns the student's projected embeddings with the teacher's without performing joint attention. A sympathetic reader would care because rationale-based retrieval, which requires grasping explanatory context rather than simple fact matching, currently demands expensive quadratic-cost cross-encoding for every query-document pair. If the claim holds, the approach delivers comparable accuracy on tasks such as empathetic dialogue and robotic manipulation while scaling linearly with document length and generalizing to standard benchmarks.

Core claim

Rabtriever is trained by first creating an LLM-based generative reranker teacher that places the document before the query and scores relevance via log-probabilities; the student is initialized from the teacher with frozen parameters and then uses a lightweight JEPA predictor inserted between layers to project the query embedding toward the teacher's contextual representation, treating the document embedding as the latent target, while an auxiliary reverse-KL loss on logits improves sampling efficiency; this converts the teacher's quadratic complexity in document length to linear, as shown both theoretically and empirically, and yields performance that exceeds retriever baselines on diverse

What carries the argument

Joint-Embedding Predictive Architecture (JEPA) predictor: a lightweight trainable module placed between LLM layers and heads that projects the independently encoded query embedding into the teacher's hidden space using the document embedding as the latent vector and minimizes their distribution difference.

If this is right

  • Retrieval cost scales linearly rather than quadratically with document length.
  • Rabtriever outperforms standard retriever baselines on rationale-heavy tasks including empathetic conversations and robotic manipulations.
  • Accuracy loss relative to the full reranker teacher stays minor across the tested domains.
  • The same model reaches performance comparable to the best retriever baselines on conventional benchmarks such as MS MARCO and BEIR.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same JEPA alignment technique could be applied to other joint-encoding tasks that currently rely on cross-attention, such as long-context summarization or multi-turn reasoning.
  • If the reconstruction holds under distribution shift, the method offers a route to scale retrieval to documents far longer than those used in current cross-encoder training.
  • The auxiliary logit loss suggests that combining embedding-level and token-level distillation may become a standard pattern for preserving generative quality in student models.

Load-bearing premise

A lightweight predictor can reconstruct the teacher's contextual-aware query embedding from independent query and document encodings well enough to deliver comparable cross-query-document comprehension.

What would settle it

If a held-out rationale-based retrieval test shows Rabtriever accuracy dropping substantially below both the generative reranker teacher and the strongest independent retriever baselines, the claim of effective reconstruction would be refuted.

Figures

Figures reproduced from arXiv: 2604.23336 by Feixiang Guo, Hongyan Li, Luo Ji, Qingqing Gu, Sheng Xu, Teng Chen, Xiaoyu Wang.

Figure 1
Figure 1. Figure 1: Top: By effective distillation, Rabtriever achieves a view at source ↗
Figure 3
Figure 3. Figure 3: Pipeline to develop Rabtriever. I: A teacher cross-encodes query ( view at source ↗
Figure 4
Figure 4. Figure 4: Averaged Online RTs corresponding to different view at source ↗
Figure 5
Figure 5. Figure 5: Method scalability on SayCan view at source ↗
Figure 6
Figure 6. Figure 6: MRR comparison of models on MS MARCO passage retrieval. view at source ↗
Figure 7
Figure 7. Figure 7: Comparison between Rabtriever and more textual view at source ↗
Figure 8
Figure 8. Figure 8: Recall curves of SayCan on varied loss weights view at source ↗
read the original abstract

Unlike traditional fact-based retrieval, rationale-based retrieval typically necessitates cross-encoding of query-document pairs using large language models, incurring substantial computational costs. To address this limitation, we propose Rabtriever, which independently encodes queries and documents, while providing comparable cross query-document comprehension capabilities to rerankers. We start from training a LLM-based generative reranker, which puts the document prior to the query and prompts the LLM to generate the relevance score by log probabilities. We then employ it as the teacher of an on-policy distillation framework, with Rabtriever as the student to reconstruct the teacher's contextual-aware query embedding. To achieve this effect, Rabtriever is first initialized from the teacher, with parameters frozen. The Joint-Embedding Predictive Architecture (JEPA) paradigm is then adopted, which integrates a lightweight, trainable predictor between LLM layers and heads, projecting the query embedding into a new hidden space, with the document embedding as the latent vector. JEPA then minimizes the distribution difference between this projected embedding and the teacher embedding. To strengthen the sampling efficiency of on-policy distillation, we also add an auxiliary loss on the reverse KL of LLM logits, to reshape the student's logit distribution. Rabtriever optimizes the teacher's quadratic complexity on the document length to linear, verified both theoretically and empirically. Experiments show that Rabtriever outperforms different retriever baselines across diverse rationale-based tasks, including empathetic conversations and robotic manipulations, with minor accuracy degradation from the reranker. Rabtriever also generalizes well on traditional retrieval benchmarks such as MS MARCO and BEIR, with comparable performance to the best retriever baseline.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes Rabtriever, a student model for rationale-based retrieval distilled from an LLM-based generative reranker teacher via on-policy distillation using the JEPA paradigm. The teacher generates relevance scores by placing the document before the query and using log probabilities; the student is initialized from the teacher (with parameters frozen), independently encodes queries and documents, and uses a lightweight trainable predictor inserted between LLM layers/heads to reconstruct the teacher's contextual query embedding from the document embedding as latent vector, minimizing distribution differences. An auxiliary reverse-KL loss on logits is added for sampling efficiency. The central claims are that this reduces the teacher's quadratic complexity in document length to linear (verified theoretically and empirically), delivers performance close to the reranker while outperforming retriever baselines on rationale-based tasks (empathetic conversations, robotic manipulations) and generalizes to MS MARCO/BEIR.

Significance. If the complexity reduction and performance claims hold with rigorous verification, the work would offer a practical advance for efficient rationale-based retrieval by avoiding full cross-encoding while retaining much of the teacher's comprehension. The on-policy JEPA distillation and auxiliary loss are interesting technical choices that could generalize. However, the absence of equations, experimental details, data splits, or error analysis in the provided manuscript makes it difficult to assess whether the data support the claims.

major comments (2)
  1. [Abstract] Abstract: the central claim that 'Rabtriever optimizes the teacher's quadratic complexity on the document length to linear, verified both theoretically and empirically' is not supported by the described architecture. Independent encodings of queries and documents using the (frozen) LLM still require standard transformer self-attention, which is quadratic in document token length L; the JEPA predictor and auxiliary loss add only lower-order terms. This directly undermines the efficiency claim that is load-bearing for the paper's contribution.
  2. [Abstract] Abstract and method description: no equations, complexity analysis, or pseudocode are supplied to substantiate the theoretical verification of linear scaling or the precise form of the JEPA loss and reverse-KL auxiliary. Without these, it is impossible to evaluate whether the student truly approximates the teacher's cross-query-document comprehension or merely matches embeddings in a way that preserves performance.
minor comments (2)
  1. The manuscript should include explicit experimental details (data splits, model sizes, training hyperparameters, error bars, and statistical significance tests) to allow reproduction and assessment of the reported outperformance over baselines.
  2. Clarify the exact initialization and freezing schedule: the description states Rabtriever is 'initialized from the teacher, with parameters frozen' yet also uses the LLM for independent encodings; it is unclear whether any components remain trainable beyond the JEPA predictor.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. We agree that the current manuscript lacks sufficient formalization and that the efficiency claim requires clarification and supporting analysis. We will submit a revised version that includes equations, pseudocode, a dedicated complexity section, and empirical runtime measurements. Point-by-point responses to the major comments follow.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that 'Rabtriever optimizes the teacher's quadratic complexity on the document length to linear, verified both theoretically and empirically' is not supported by the described architecture. Independent encodings of queries and documents using the (frozen) LLM still require standard transformer self-attention, which is quadratic in document token length L; the JEPA predictor and auxiliary loss add only lower-order terms. This directly undermines the efficiency claim that is load-bearing for the paper's contribution.

    Authors: We acknowledge that self-attention within each independent encoder remains quadratic in its input length. However, the intended efficiency claim concerns online inference complexity with respect to document length: document embeddings are precomputed once offline (O(L^2) per document, amortized over all queries), after which scoring a query against any number of documents reduces to a single query encoding (O(Q^2)) plus linear dot-product scoring (O(N)). In contrast, the teacher requires a full cross-encoding of each query-document pair at query time, incurring O(N · (L + Q)^2). We will revise the abstract to state this distinction explicitly and add a new section with asymptotic analysis, wall-clock timings on documents of varying lengths, and a table comparing online versus offline costs. The JEPA predictor and auxiliary loss do not alter the quadratic term but are lower-order additions that do not affect the overall reduction in online dependence on L. revision: partial

  2. Referee: [Abstract] Abstract and method description: no equations, complexity analysis, or pseudocode are supplied to substantiate the theoretical verification of linear scaling or the precise form of the JEPA loss and reverse-KL auxiliary. Without these, it is impossible to evaluate whether the student truly approximates the teacher's cross-query-document comprehension or merely matches embeddings in a way that preserves performance.

    Authors: We agree that the absence of equations and pseudocode hinders evaluation. In the revision we will add: (1) the exact JEPA objective (L2 or cosine distance between the predictor output and the teacher's contextual query embedding, with the document embedding serving as the latent conditioning vector); (2) the auxiliary reverse-KL term on the student's logit distribution; (3) the combined training loss; (4) a formal complexity derivation showing the online reduction from quadratic to effectively constant in L; and (5) pseudocode for both the distillation procedure and inference. These additions will make clear how the student reconstructs the teacher's cross-comprehension signal while retaining independent encoding. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation chain is self-contained

full rationale

The paper describes a standard on-policy distillation setup: initialize student from frozen teacher LLM, insert lightweight JEPA predictor to match contextual query embeddings, and add auxiliary reverse-KL loss. The claimed complexity reduction from quadratic to linear is presented as a direct consequence of switching to independent encodings (with theoretical and empirical verification stated but not derived via self-reference). No equation reduces to its input by construction, no load-bearing self-citation, and no fitted parameter renamed as a prediction. Experiments on rationale-based tasks and standard benchmarks provide external grounding independent of the training targets.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no explicit free parameters, axioms, or invented entities; the core addition is the Rabtriever model and JEPA application itself.

pith-pipeline@v0.9.0 · 5613 in / 1141 out tokens · 42827 ms · 2026-05-14T21:06:12.842808+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

33 extracted references · 22 canonical work pages · 5 internal anchors

  1. [1]

    Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, and Wen-tau Yih. [n. d.]. Task-aware Retrieval with Instructions. InFindings of the Association for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). 3650–3675

    2023

  2. [2]

    Anna Dawid and Yann LeCun. 2024. Introduction to latent variable energy-based models: a path toward autonomous machine intelligence.Journal of Statistical Mechanics: Theory and Experiment2024, 10 (31 Oct. 2024). doi:10.1088/1742- 5468/ad292b

    work page doi:10.1088/1742- 2024

  3. [3]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(Barcelona, Spain) (KDD ’24). Association for Computing Machinery, New York, NY, U...

    work page doi:10.1145/3637528.3671470 2024

  4. [4]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997 [cs.CL] https://arxiv.org/abs/2312.10997

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  5. [5]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge Distillation of Large Language Models. InThe Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=5h0qf7IBZZ

    2024

  6. [6]

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, Sergey Levine, Karol Hausman, and Brian Ichter. 2022. Inner Monologue: Embodied Reasoning through Planning with Language Models. InarXiv preprint arXiv:2207.05608

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  7. [7]

    ichter et al. 2023. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. InProceedings of The 6th Conference on Robot Learning, Vol. 205. PMLR, 287–318. https://proceedings.mlr.press/v205/ichter23a.html

    2023

  8. [8]

    Luo Ji, Feixiang Guo, Teng Chen, Qingqing Gu, Xiaoyu Wang, Ningyuan Xi, Yihong Wang, Peng Yu, Yue Zhao, Hongyang Lei, Zhonglin Jiang, and Yong Chen. 2025. Large Language Model Can Be a Foundation for Hidden Rationale- Based Retrieval. InAdvances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, April 6–1...

    2025

  9. [9]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Mem- ory Management for Large Language Model Serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

  10. [10]

    Chaofan Li, Zheng Liu, Shitao Xiao, Yingxia Shao, and Defu Lian. 2024. Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Re- trieval. InProceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Li...

    work page doi:10.18653/v1/2024.acl-long.191 2024

  11. [11]

    Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2021. In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval. InProceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP- 2021), Anna Rogers, Iacer Calixto, Ivan Vulić, Naomi Saphra, Nora Kassner, Oana-Maria Camburu, Trapit Bansal, and Vered Shwar...

    work page doi:10.18653/v1/2021.repl4nlp-1.17 2021

  12. [12]

    Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards Emotional Support Dialog Systems. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro- cessing (Volume 1: Long Papers), Chengqing Zong...

    work page doi:10.18653/v1/2021.acl-long.269 2021

  13. [13]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine- Tuning LLaMA for Multi-Stage Text Retrieval. InProceedings of the 47th Interna- tional ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA)(SIGIR ’24). Association for Computing Machinery, New York, NY, USA, 2421–2425. doi:10.1145/3626772.3657951

    work page doi:10.1145/3626772.3657951 2024

  14. [14]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback.ArXiv abs/2303.17651 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  15. [15]

    Amir M. Mansourian, Rozhan Ahmadi, Masoud Ghafouri, Amir Mohammad Babaei, Elaheh Badali Golezani, Zeynab yasamani ghamchi, Vida Ramezanian, Alireza Taherian, Kimia Dinashi, Amirali Miri, and Shohreh Kasaei. 2025. A Comprehensive Survey on Knowledge Distillation.Transactions on Machine Learning Research(2025)

    2025

  16. [16]

    Niklas Muennighoff. 2022. SGPT: GPT Sentence Embeddings for Semantic Search. arXiv preprint arXiv:2202.08904(2022)

    work page arXiv 2022

  17. [17]

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Aman- preet Singh, and Douwe Kiela. 2024. Generative Representational Instruction Tuning. arXiv:2402.09906 [cs.CL]

    work page arXiv 2024

  18. [18]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Gen- erated MAchine Reading COmprehension Dataset. (November 2016). https://www.microsoft.com/en-us/research/publication/ms-marco-human- generated-machine-reading-comprehension-dataset/

    2016

  19. [19]

    2024.QWEN2 TECHNICAL REPORT

    Alibaba Group Qwen Team. 2024.QWEN2 TECHNICAL REPORT. Technical Report. Alibaba Group

    2024

  20. [20]

    Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu, Dorsa Sadigh, Andy Zeng, and Anirudha Majumdar. 2023. Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners. InProceedings of the Conference on Robot Learning (CoRL)

    2023

  21. [21]

    Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021. RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau ...

    work page doi:10.18653/v1/2021.emnlp-main.224 2021

  22. [22]

    Smith, Luke Zettlemoyer, and Tao Yu

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. One Embedder, Any Task: Instruction-Finetuned Text Embeddings. InFindings of the Associa- tion for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Comp...

    work page doi:10.18653/v1/2023.findings-acl.71 2023

  23. [23]

    Hao Sun, Zhenru Lin, Chujie Zheng, Siyang Liu, and Minlie Huang. 2021. PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support. InFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, Online, 1489–...

    work page doi:10.18653/v1/2021.findings- 2021

  24. [24]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? In- vestigating Large Language Models as Re-Ranking Agents. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for ...

    work page doi:10.18653/v1/2023.emnlp-main.923 2023

  25. [25]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models.arXiv preprint arXiv:2104.08663(2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  26. [26]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Improving Text Embeddings with Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Ba...

    work page doi:10.18653/v1/2024.acl-long.642 2024

  27. [27]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems35 (2022), 24824–24837

    2022

  28. [28]

    Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. 2024. Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models. arXiv:2409.11136 [cs.IR] https://arxiv.org/ abs/2409.11136

    work page arXiv 2024

  29. [29]

    Hang Zhang, Yeyun Gong, Yelong Shen, Jiancheng Lv, Nan Duan, and Weizhu Chen. 2022. Adversarial Retriever-Ranker for Dense Text Retrieval. InInterna- tional Conference on Learning Representations. https://openreview.net/forum?id= MR7XubKUFB

    2022

  30. [30]

    Jintian Zhang, Cheng Peng, Mengshu Sun, Xiang Chen, Lei Liang, Zhiqiang Zhang, Jun Zhou, Huajun Chen, and Ningyu Zhang. 2024. OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs. arXiv:2409.05152 [cs.CL] https://arxiv.org/abs/2409.05152

    work page arXiv 2024

  31. [31]

    Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, and Bin Cui. 2024. Retrieval-Augmented Generation for AI-Generated Content: A Survey. arXiv:2402.19473 [cs.CV] https://arxiv.org/abs/2402.19473

    work page arXiv 2024

  32. [32]

    Qiu, and Lili Qiu

    Siyun Zhao, Yuqing Yang, Zilong Wang, Zhiyuan He, Luna K. Qiu, and Lili Qiu. 2024. Retrieval Augmented Generation (RAG) and Beyond: A Compre- hensive Survey on How to Make your LLMs use External Data More Wisely. arXiv:2409.14924 [cs.CL] https://arxiv.org/abs/2409.14924

    work page arXiv 2024

  33. [33]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, and Yongqiang Ma. 2024. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Lan- guage Models.arXiv preprint arXiv:2403.13372(2024). http://arxiv.org/abs/2403. 13372 ICMR ’26, June 16–19, 2026, Amsterdam, Netherlands Chen et al. A More Implementation Details A.1 Detailed Prompt Here we pres...

    work page internal anchor Pith review Pith/arXiv arXiv 2024