Pith. sign in

REVIEW 5 major objections 5 minor 61 references

Token-Level Credit Assignment Optimization for Generative Document Retrieval

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Token-level credit assignment for each DocID decoding step improves generative retrieval by giving every token its own relevance reward.

desk verdict A solid, internally consistent empirical study showing that position-wise cosine rewards against a frozen SFT reference's gold hidden states improve generative retrieval over sequence-level RL baselines—but the paper sells this as measured token-level relevance when it is really an imitation signal, and a few under-specified equations need fixing before it is fully reproducible. read the letter →

arxiv 2608.12049 v1 pith:3FLI5ODO submitted 2026-08-12 cs.IR

Xinpeng Zhao , Yang Liu , Ran Chen , Xinyu Ma , Daiting Shi , Pengjie Ren , Zhumin Chen , Zhaochun Ren
show 1 more author
Xin Xin
This is my paper · ORCID
classification cs.IR
keywords GenerativeretrievalToken-levelcreditassignmentReinforcementlearningDocIDgenerationRewarddesignGRPOPPODocumentranking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generative retrieval models produce documents by generating an identifier (DocID) token by token, yet they are judged only after the whole sequence is complete. This paper argues that a single document-level reward sent back to every token is too coarse: an early token can route decoding into a bad region of the DocID space, while later tokens may be locally plausible but lead to an irrelevant document. To fix this, it proposes a token-level credit assignment framework that rewards each decoding step by how close the generated hidden state stays to the hidden-state trajectory of the gold DocID from a frozen supervised reference model. Instantiating this with GRPO and PPO improves R@1 and MRR@10 over sequence-level reward baselines on MS MARCO and Natural Questions. The central claim is that fine-grained, position-wise rewards align token generation with document-level relevance better than whole-sequence rewards.

What carries the argument

The load-bearing object is the token-level trajectory reward of Eq. (10), which compares the last-layer decoder hidden state before each generated token with the corresponding hidden state of the gold DocID, cached once under teacher forcing from a frozen SFT reference model, via cosine similarity. It converts a document-level target into a dense per-position signal, and is coupled with a length penalty and a global exact-match bonus. TCA-GRPO then normalizes these rewards position-wise across a group of candidates sampled for the same query to compute advantages, while TCA-PPO uses a value head on the prefix hidden state with generalized advantage estimation. This shared reward design is what lets the paper compare reward granularity independently of the optimizer.

What would settle it

On the same MS MARCO-TU and NQ-PQ setups, replace the gold hidden states in Eq. (10) with hidden states from a different document at the same token position, keeping the exact-match bonus and all hyperparameters fixed. If R@1 and MRR@10 still improve over the sequence-level baseline, the gains do not come from trajectory-specific credit assignment; if they disappear or reverse, the cosine-to-gold-trajectory reward is the operative mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that the credit-assignment problem in generative retrieval is token-level, not sequence-level: each decoded token is a routing decision over the constrained DocID space, and the same final relevance score cannot distinguish which tokens helped or hurt. TCA therefore assigns a token-level reward $r_{i,t} = \mathbb{1}[t\le T]\,\cos(h_t^i, h_t^*)$ comparing the decoder hidden state of a generated token with the hidden state of the gold DocID at the same position, plus a length penalty for over-generation and an exact-match bonus for hitting the gold DocID. These rewards are turned into advantages either by position-wise normalization over a group of candidates (GRPO) or by a learned value function with generalized advantage estimation (PPO). On MS MARCO with title/URL DocIDs, TCA-GRPO raises R@1 from 38.24 to 39.60 and MRR@10 from 50.33 to 51.10 over the DDRO baseline, and on Natural Questions with PQ DocIDs it raises R@1 from 48.10 to 50.02 and MRR@10 from 54.32 to 56.10. The paper also reports that the effect is separable from RL alone: a sequence-level version of the same optimizer gains less than the token-level version. The authors close by acknowledging that the pipeline is more complex and adds rollout overhead.

Load-bearing premise

The reward in Eq. (10) assumes that, at each decoding position, how close the model's internal state before predicting a token is to the internal state of the gold DocID at the same position measures how much that token decision contributes to document-level relevance; the paper does not validate this against any per-token relevance signal.

Editorial extensions

If this is right

  • If token-level credit assignment is responsible for the gains, then retrieval quality is not limited by what sequence-level RL can achieve, and future generative retrievers can be aligned by position-wise relevance signals.
  • The same trajectory-reward design should transfer to other DocID types beyond title/URL and PQ codes, since it only requires hidden states of the gold trajectory.
  • The improvement is concentrated in R@1 and MRR@10, so token-level rewards mainly sharpen early precision; recall-oriented or listwise objectives remain complementary.
  • Both GRPO and PPO instantiations improve over SFT, so the token-level reward is optimizer-agnostic; however, group-relative normalization is more reliable than learned value estimation on discrete prefixed DocID states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication left implicit is that the method's ceiling is tied to the quality of the frozen SFT reference: if the reference model's hidden states are poor at representing relevance, the cosine reward may reward imitation rather than retrieval; updating or distilling the reference during training is a natural extension the paper does not test.
  • A testable consequence is that the position-wise group normalization matters most at early decoding positions, where the candidate space is large; ablating advantages by position should show larger gains early.
  • The same token-level reward design could be plugged into preference-based objectives such as token-wise DPO, not just GRPO and PPO, because it produces a per-token scalar signal.
  • The paper's stated cost—a supervised reference plus rollout overhead—invites a lighter single-stage variant, and whether it retains the gains is untested.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes TCA, a two-stage reinforcement-learning framework for generative document retrieval that assigns token-level rewards during DocID generation. After supervised fine-tuning, a frozen SFT reference model is used to cache the gold DocID hidden-state trajectory, and the policy's decoder hidden states at each generated position are scored by cosine similarity to the corresponding gold hidden state, augmented by an overflow length penalty and an exact-match bonus. These token rewards are then optimized with either GRPO or PPO. Experiments on MS MARCO and Natural Questions with TU and PQ DocIDs report consistent improvements in R@1 and MRR@10 over the DDRO baseline and over a sequence-level GRPO variant, together with ablations on constrained decoding, group size, KL coefficient, and a case study of gold-token probability shifts.

Significance. If the central claim holds, the paper makes a useful contribution: it provides a dense, reward-model-free training signal for generative retrieval and shows that the resulting gains are not tied to a single policy optimizer. The final evaluation is external retrieval quality, so the headline result is not circular, and the controlled comparison against sequence-level RL under the same optimizer is the right experimental design. The paper also gives credit for careful reporting of multiple benchmarks, two DocID types, statistical significance tests, and extensive ablations. However, the validity of the token-level reward itself is the load-bearing assumption: the reward is a closeness-to-gold-trajectory measure, and the manuscript does not yet establish that this measure is a faithful per-token relevance signal. The reported gains are consistent with hidden-state distillation, which would weaken the claimed connection between token-level credit assignment and retrieval quality.

major comments (5)
  1. [Section 4.4, Eqs. (10)-(12)] The token-level reward is defined as the cosine similarity between the generated decoder hidden state and the gold-DocID hidden state at the same absolute position from the frozen SFT model. The abstract and Section 1 promise a reward that measures how each token decision changes expected retrieval quality, but Eq. (10) measures agreement with one fixed gold trajectory. Because the comparison is position-wise against the gold prefix, any generated token that deviates at an early position will tend to receive low cosine scores at all later positions even if the deviation leads to another relevant document; the reward cannot distinguish helpful alternative routing decisions from harmful ones. Since the reference is the SFT model and Eq. (19) adds KL regularization toward that same reference, the RL stage may largely be a self-distillation objective. This is load-bearing: the claim that the gains come from token-level credit assignment for retrieval relevance requires either an independent validation of the cosine measure against per-token relevance, or a substantially revised interpretation of what the reward measures.
  2. [Section 4.4, Eq. (14)] The mask m_{i,t} is defined as 1 if position t is less than the length of the target DocID and 0 otherwise, which conflicts with Eq. (10), where r_traj is defined for t <= T, and with Eq. (11), where the overflow penalty r_len is applied for t > T. Under Eq. (14), the overflow positions and also the final gold position t=T are masked out from the reward and from the normalization denominators in Eq. (15), so the length penalty cannot influence training. The authors should clarify the intended definition of a valid non-padding position and ensure that Eqs. (10), (11), and (14) are mutually consistent; as written, the implementation is ambiguous.
  3. [Section 4.4, Eq. (12)] The exact-match bonus R_hit is introduced but never given a numeric value in Section 4.4 or in the implementation details of Section 5.4. Because this bonus is applied to every valid position of an exactly matched DocID, it is a strong sequence-level signal that could dominate the token-level trajectory reward, and the paper does not report the contribution of R_hit relative to r_traj. The authors should report the value of R_hit and provide an ablation that separates r_traj, r_len, and the exact-match bonus, otherwise it is possible that the headline gains are driven by the bonus rather than by the proposed token-level credit assignment.
  4. [Section 6.4, Table 4] The sequence-level baseline TCA-GRPOseq is not specified anywhere in the paper. To support the Q3 claim that token-level credit assignment adds gains over sequence-level rewards while keeping the optimizer fixed to GRPO, the authors must define exactly which reward is used by TCA-GRPOseq (for example, document-level relevance, exact-match, or a terminal reward only), and state whether the group size, KL coefficient, and rollout procedure are identical to TCA-GRPO. Without this information, the comparison in Table 4 is not reproducible and the central ablation is incomplete.
  5. [Section 4.6, Eq. (23)] The text states that the value head is trained with an MSE loss against GAE return targets R_{i,t}, but Eq. (23) writes the target as the raw token reward r_{i,t}. If the implementation follows the equation, the value baseline is trained on immediate rewards rather than returns, which changes the advantage estimates and the behavior of TCA-PPO. Please correct the equation or the surrounding text and report the actual objective used in the experiments.
minor comments (5)
  1. [Section 5.2] The paired t-test is mentioned but the number of test queries, the variance across runs, and whether significance is computed on the full dev set are not reported; adding these details would strengthen the statistical claims.
  2. [Table 4] The sentence "All variants are initialized from baseline-sft model" is repeated in the table caption; the duplication should be removed.
  3. [Section 4.5, Eq. (16)] The GRPO advantage uses a position-wise normalization across candidates, but the paper does not explain how positions with only one valid candidate are handled; with G=8 the variance estimate at such a position is degenerate and the epsilon term is the only stabilizer.
  4. [Section 4.6, Eq. (21)] The KL reward shaping term uses log pi_theta_old - log pi_ref, while Eq. (19) uses a KL divergence; the relationship between the two KL formulations should be clarified.
  5. [Section 6.5, Figure 3] The caption and text describe a 10th-90th percentile range, but the figure appears to show only one shaded band; please confirm that the band corresponds to the stated percentile range.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the retrieval gains are measured on external benchmarks, and the hidden-state reward, while unvalidated as a per-token relevance signal, is not equivalent by construction to the reported prediction.

full rationale

The paper's central empirical claim—that TCA improves R@1 and MRR@10 over DDRO and other baselines—is tested on held-out retrieval benchmarks (Tables 1 and 2) with externally defined metrics, so the headline result is not a consequence of the reward definition by construction. The token-level reward in Eq. (10) is a cosine similarity between the policy's hidden state and the frozen SFT reference's gold-token hidden state; this is a modeling choice that is not validated as a per-token relevance signal, and Eq. (12) copies the exact-match bonus to every position, meaning the ablation in Table 4 may conflate the trajectory reward with that bonus. These are construct-validity and confound concerns, not circular reductions: the reward could in principle lower or raise the external metrics, and the evaluation does not reuse any fitted parameter as its own prediction. No load-bearing step is defined in terms of the target result, and no self-citation chain or uniqueness theorem is used to force the outcome. Therefore no significant circularity is found.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central method rests on no new physical or conceptual entity. The load-bearing choices are the hand-set reward coefficients, the validation-tuned group size and KL coefficient, and the assumption that a frozen SFT hidden-state trajectory encodes token-level retrieval quality. The unreported R_hit is the most serious gap in the parameter ledger.

free parameters (6)
  • R_hit (exact-match bonus) = not reported in paper
    Appears in Eq. (12) as a per-position bonus when the generated DocID exactly matches the gold DocID. The magnitude is never specified, so the scale of the sequence-level component of the token reward is unknown.
  • lambda_len (overflow length penalty) = -0.5
    Set by hand in Eq. (11) to penalize generated DocIDs longer than the gold DocID. It may be ineffective because Eq. (14) masks positions beyond the target length.
  • KL coefficient beta = 0.01 for TU, 0 for PQ
    Selected per DocID type based on Figure 5. Controls the strength of the KL regularizer toward the frozen SFT reference policy in Eq. (19).
  • GRPO group size G = 8
    Selected from Figure 4 as the best or near-best value. Used in Eq. (15) and Eq. (16) for position-wise group-relative normalization.
  • PPO value loss coefficient c_v = 0.1
    Hand-chosen weight on the value loss in Eq. (24) for the TCA-PPO variant.
  • GAE discount gamma and trace-decay lambda = gamma=1.0, trace-decay not reported
    Used in TCA-PPO advantage estimation in Section 4.6. The trace-decay value is not stated, which prevents exact replication.
assumptions (4)
  • domain assumption A single gold DocID is available for every training query and is treated as the unique ground-truth retrieval target.
    Section 4.2, Eq. (7). The method does not handle multiple relevant documents or alternative valid DocIDs for the same query.
  • ad hoc to paper Cosine similarity between generated hidden states and gold-DocID hidden states from a frozen SFT model is a valid token-level relevance signal.
    Section 4.4, Eq. (10). This is the paper's core modeling premise, and it is not validated against any per-token relevance measurement.
  • domain assumption An exact DocID match is a sufficient document-level relevance signal.
    Eq. (12). The exact-match bonus assumes the gold DocID is the only route to a relevant document, ignoring semantically equivalent identifiers.
  • domain assumption Trie-constrained decoding over the legal DocID space produces rollouts that are meaningful for policy learning.
    Section 3.2, Eq. (6), and Table 5. Removing constraints hurts performance, but the assumption that legal prefixes preserve relevance structure is not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Token-Level Credit Assignment Optimization for Generative Document Retrieval." pith.science (2026). https://pith.science/paper/3FLI5ODO

@misc{pith2026260812049,
  author       = {Pith},
  title        = {Pith review of: Token-Level Credit Assignment Optimization for Generative Document Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3FLI5ODO}},
  note         = {Machine review of arXiv:2608.12049}
}
read the original abstract

Generative retrieval models perform document retrieval by autoregressively generating document identifiers (DocIDs). This process naturally forms a sequential decision problem, where each decoding step selects a DocID token and the complete token sequence determines the retrieved document. However, retrieval effectiveness is typically evaluated only after the full DocID is generated, creating a mismatch between token-level generation and document-level relevance supervision. As a result, existing reinforcement learning methods for generative retrieval mostly rely on sequence-level rewards, where the same document-level feedback is propagated to all decoding steps. Such coarse-grained feedback makes it difficult to identify which token decisions are responsible for successful or failed retrieval. In this work, we propose a fine-grained reinforcement learning framework for generative retrieval with token-level relevance rewards. Instead of assigning a single reward to the entire generated DocID, we estimate step-wise rewards by measuring how each token decision changes the expected retrieval quality of the corresponding generation trajectory. This enables more precise credit assignment and encourages the policy to favor token decisions that contribute more directly to document-level relevance. We further develop practical reward estimation strategies tailored to the DocID generation process and incorporate them into a policy optimization framework. Experiments on retrieval benchmarks show that our method consistently outperforms sequence-level reward baselines, demonstrating the effectiveness of fine-grained supervision for aligning autoregressive DocID generation with retrieval objectives.

Figures

Figures reproduced from arXiv: 2608.12049 by the authors.

Figure 1
Figure 1. Comparison of sequence-level and token-level re [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed Token-Level Credit Assignment for Generative Retrieval with GRPO (TCA-GRPO). [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. illustrates the evolution of reward values during the train￾ing process using TCA-GRPO. We plot reward trajectories for two representative settings, MS MARCO-TU and NQ-PQ. Because per￾step rewards fluctuate substantially during policy optimization, we group training steps into equal-width progress bins and report both the mean reward and the 10th–90th percentile range within each bin. The mean reward increases consi… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Effect of group size in TCA-GRPO on MS MARCO [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 6
Figure 6. Figure 6: Case study on MS MARCO for the query “how much [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 5
Figure 5. Figure 5: Effect of the KL regularization coefficient [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 24 canonical work pages

  1. [1]

    Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Scott Yih, Sebastian Riedel, and Fabio Petroni. 2022. Autoregressive search engines: Generating substrings as document identifiers.Advances in Neural Information Processing Systems35 (2022), 31668–31683

    2022

  2. [2]

    Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. Au- toregressive Entity Retrieval. InICLR

    work page 2021

  3. [3]

    Alex J Chan, Hao Sun, Samuel Holt, and Mihaela Van Der Schaar. 2024. Dense reward for free in reinforcement learning from human feedback.arXiv preprint arXiv:2402.00782(2024)

    arXiv 2024

  4. [4]

    Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Yixing Fan, and Xueqi Cheng. 2023. Continual learning for generative retrieval over dynamic corpora. InProceedings of the 32nd ACM international conference on information and knowledge management. 306–315

    2023

  5. [5]

    Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Yiqun Liu, Yixing Fan, and Xueqi Cheng. 2022. Corpusbrain: Pre-train a generative retrieval model for knowledge- intensive language tasks. InProceedings of the 31st ACM International Conference on Information & Knowledge Management. 191–200

    work page 2022

  6. [6]

    Jiehan Cheng, Zhicheng Dou, Yutao Zhu, and Xiaoxi Li. 2025. Descriptive and Discriminative Document Identifiers for Generative Retrieval. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 11518–11526

    2025

  7. [7]

    Anna Harutyunyan, Will Dabney, Thomas Mesnard, Mohammad Ghesh- laghi Azar, Bilal Piot, Nicolas Heess, Hado P van Hasselt, Gregory Wayne, Satinder Singh, Doina Precup, et al. 2019. Hindsight credit assignment.Advances in neural information processing systems32 (2019)

    work page 2019

  8. [8]

    Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021. Efficiently teaching an effective dense retriever with balanced topic aware sampling. InProceedings of the 44th international ACM SIGIR confer- ence on research and development in information retrieval. 113–122

    2021

Show all 61 references
  1. [9]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense in- formation retrieval with contrastive learning.arXiv preprint arXiv:2112.09118 (2021)

    2021 arXiv

  2. [10]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)...

    2020

  3. [11]

    Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. InProceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 39–48

    2020

  4. [12]

    Saar Kuzi, Mingyang Zhang, Cheng Li, Michael Bendersky, and Marc Najork. 2020. Leveraging semantic and lexical matching to improve the recall of document retrieval systems: A hybrid approach.arXiv preprint arXiv:2010.01195(2020)

    2020 arXiv

  5. [13]

    Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob De- vlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob De- vlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc V. Le, ...

    2019

  6. [14]

    Alan Li, Daniel Cheng, Phillip Keung, Jungo Kasai, and Noah A Smith. 2024. Summarization-Based Document IDs for Generative Retrieval with Language Models. InProceedings of the First Workshop on Advancing Natural Language Processing for Wikipedia. 126–135

    2024

  7. [15]

    Jiahui Li, Lin Li, Tai-Wei Chang, Kun Kuang, Long Chen, Jun Zhou, and Cheng Yang. 2025. RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 4993–5022

    2025

  8. [16]

    Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. 2025. From matching to generation: A survey on generative information retrieval.ACM Transactions on Information Systems43, 3 (2025), 1–62

    2025

  9. [17]

    Yongqi Li, Nan Yang, Liang Wang, Furu Wei, and Wenjie Li. 2023. Multiview identifiers enhanced generative retrieval. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 6636– 6648

    2023

  10. [18]

    Yongqi Li, Nan Yang, Liang Wang, Furu Wei, and Wenjie Li. 2024. Learning to rank in generative retrieval. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 8716–8723. CIKM ’26, November 7–11, 2026, Rome, Italy Zhao et al

    2024

  11. [19]

    Yongqi Li, Zhen Zhang, Wenjie Wang, Liqiang Nie, Wenjie Li, and Tat-Seng Chua

  12. [20]

    Penghao Lu, Xin Dong, Yuansheng Zhou, Lei Cheng, Chuan Yuan, and Linjian Mo

  13. [21]

    Sanket Vaibhav Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Q Tran, Jinfeng Rao, Marc Najork, Emma Strubell, and Donald Metzler. 2023. DSI++: Updating transformer memory with new documents. InProceedings of the 2023 conference on empirical methods in natural language proce...

    2023

  14. [22]

    Kidist Amde Mekonnen, Yubao Tang, and Maarten de Rijke. 2025. Lightweight and Direct Document Relevance Optimization for Generative Information Retrieval. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1327–1338

    2025

  15. [23]

    Donald Metzler, Yi Tay, Dara Bahri, and Marc Najork. 2021. Rethinking search: making domain experts out of dilettantes. InAcm sigir forum, Vol. 55. ACM New York, NY, USA, 1–27

    2021

  16. [24]

    Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. Sentence-t5: Scalable sentence encoders from pre- trained text-to-text models. InFindings of the association for computational lin- guistics: ACL 2022. 1864–1874

    2022

  17. [25]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (...

    2022

  18. [26]

    Ruiyang Ren, Wayne Xin Zhao, Jing Liu, Hua Wu, Ji-Rong Wen, and Haifeng Wang. 2023. TOME: A two-stage approach for model-based retrieval. InProceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 6102–6114

    2023

  19. [27]

    2009.The probabilistic relevance frame- work: BM25 and beyond

    Stephen Robertson and Hugo Zaragoza. 2009.The probabilistic relevance frame- work: BM25 and beyond. Vol. 4. Now Publishers Inc

    2009

  20. [28]

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel

  21. [29]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  22. [30]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. Deepseekmath: Push- ing the limits of mathematical reasoning in open language models, 2024.URL https://arxiv. org/abs/2402.033002, 3 (2024), 5

    2024 arXiv

  23. [31]

    Weiwei Sun, Keyi Kong, Xinyu Ma, Shuaiqiang Wang, Dawei Yin, Maarten de Rijke, Zhaochun Ren, and Yiming Yang. 2026. ZeroGR: A Generalizable and Scalable Framework for Zero-Shot Generative Retrieval. InThe Fourteenth Inter- national Conference on Learning Representations. https...

    2026

  24. [32]

    Yubao Tang, Ruqing Zhang, Jiafeng Guo, Jiangui Chen, Zuowei Zhu, Shuaiqiang Wang, Dawei Yin, and Xueqi Cheng. 2023. Semantic-enhanced differentiable search index inspired by learning strategies. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mi...

    2023

  25. [33]

    Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. 2022. Transformer memory as a differentiable search index.Advances in neural information processing systems35, 21831–21843

    2022

  26. [34]

    Ravisri Valluri, Akash Kumar Mohankumar, Kushal Dave, Amit Singh, Jian Jiao, Manik Varma, and Gaurav Sinha. 2024. Scaling the vocabulary of non-autoregressive models for efficient generative retrieval.arXiv preprint arXiv:2406.06739(2024)

    2024 arXiv

  27. [35]

    Yujing Wang, Yingyan Hou, Haonan Wang, Ziming Miao, Shibin Wu, Qi Chen, Yuqing Xia, Chengmin Chi, Guoshuai Zhao, Zheng Liu, et al . 2022. A neural corpus indexer for document retrieval.Advances in Neural Information Processing Systems35, 25600–25614

    2022

  28. [36]

    Yidan Wang, Zhaochun Ren, Weiwei Sun, Jiyuan Yang, Zhixiang Liang, Xin Chen, Ruobing Xie, Su Yan, Xu Zhang, Pengjie Ren, et al. 2024. Content-based collaborative generation for recommender systems. InProceedings of the 33rd ACM International Conference on Information and Knowl...

    2024

  29. [37]

    Zihan Wang, Yujia Zhou, Yiteng Tu, and Zhicheng Dou. 2023. NOVO: learnable and interpretable document identifiers for model-based IR. InProceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2656–2665

    2023

  30. [38]

    Haoyang Wen, Jiang Guo, Yi Zhang, Jiarong Jiang, and Zhiguo Wang. 2025. On synthetic data strategies for domain-specific generative retrieval. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 7961–7976

    2025

  31. [39]

    Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Am- manabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. 2023. Fine-grained human feedback gives better rewards for language model training. Advances in Neural Information Processing Systems36 (20...

    2023

  32. [40]

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. InICLR

    2021

  33. [41]

    Kailai Yang, Zhiwei Liu, Qianqian Xie, Jimin Huang, Erxue Min, and Sophia Ana- niadou. 2025. Selective preference optimization via token-level reward function estimation. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 7043–7067

    2025

  34. [42]

    Eunseop Yoon, Hee Suk Yoon, SooHwan Eom, Gunsoo Han, Daniel Nam, Daejin Jo, Kyoung-Woon On, Mark Hasegawa-Johnson, Sungwoong Kim, and Chang Yoo. 2024. Tlcr: Token-level continuous reward for fine-grained reinforcement learning from human feedback. InFindings of the Association...

    2024

  35. [43]

    Hansi Zeng, Chen Luo, Bowen Jin, Sheikh Muhammad Sarwar, Tianxin Wei, and Hamed Zamani. 2024. Scalable and effective generative information retrieval. In Proceedings of the ACM Web Conference 2024. 1441–1452

    2024

  36. [44]

    Hansi Zeng, Chen Luo, and Hamed Zamani. 2024. Planning ahead in generative retrieval: Guiding autoregressive generation through simultaneous decoding. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 469–480

    2024

  37. [45]

    Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang, Haifeng Zhang, and Jun Wang. 2024. Token-level direct preference optimization.arXiv preprint arXiv:2404.11999(2024)

    2024 arXiv

  38. [46]

    Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma

  39. [47]

    Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2020. Rep- bert: Contextualized text embeddings for first-stage retrieval.arXiv preprint arXiv:2006.15498(2020)

    2020 arXiv

  40. [48]

    Zhen Zhang, Xinyu Ma, Weiwei Sun, Pengjie Ren, Zhumin Chen, Shuaiqiang Wang, Dawei Yin, Maarten de Rijke, and Zhaochun Ren. 2025. Replication and Exploration of Generative Retrieval over Dynamic Corpora. InProceedings of the 48th International ACM SIGIR Conference on Research ...

    2025

  41. [49]

    Zhen Zhang, Zihan Wang, Xinyu Ma, Shuaiqiang Wang, Dawei Yin, Xin Xin, Pengjie Ren, Maarten de Rijke, and Zhaochun Ren. 2026. Model Editing for New Document Integration in Generative Information Retrieval. InProceedings of the ACM Web Conference 2026. 1993–2003

    2026

  42. [50]

    Xinpeng Zhao, Zhaochun Ren, Yukun Zhao, Zhenyang Li, Mengqi Zhang, Jun Feng, Ran Chen, Ying Zhou, Zhumin Chen, Shuaiqiang Wang, et al. 2025. DiffuGR: Generative Document Retrieval with Diffusion Language Models.arXiv preprint arXiv:2511.08150(2025)

    2025

  43. [51]

    Han Zhong, Zikang Shan, Guhao Feng, Wei Xiong, Xinle Cheng, Li Zhao, Di He, Jiang Bian, and Liwei Wang. 2024. Dpo meets ppo: Reinforced token optimization for rlhf.arXiv preprint arXiv:2404.18922(2024)

    2024 arXiv

  44. [52]

    Yujia Zhou, Zhicheng Dou, and Ji-Rong Wen. 2023. Enhancing generative retrieval with reinforcement learning from relevance feedback. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 12481–12490

    2023

  45. [53]

    Yujia Zhou, Jing Yao, Zhicheng Dou, Yiteng Tu, Ledell Wu, Tat-Seng Chua, and Ji-Rong Wen. 2024. ROGER: Ranking-Oriented Generative Retrieval.ACM Trans. Inf. Syst.42, 6, Article 155 (Oct. 2024), 25 pages. doi:10.1145/3603167

    2024 doi

  46. [54]

    Yujia Zhou, Jing Yao, Zhicheng Dou, Ledell Yu Wu, Peitian Zhang, and Ji rong Wen. 2022. Ultron: An Ultimate Retriever on Corpus with a Model-based Indexer. ArXivabs/2208.09257 (2022)

    2022 arXiv

  47. [55]

    Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. 2025. Large language models for information retrieval: A survey.ACM Transactions on Information Systems44, 1 (2025), 1–54

    2025

  48. [56]

    Zuccon, and Daxin Jiang

    Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, G. Zuccon, and Daxin Jiang. 2022. Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation.ArXivabs/2206.10128 (2022)

    2022 arXiv

  49. [2015]

    High-dimensional continuous control using generalized advantage estima- tion.arXiv preprint arXiv:1506.02438(2015)

    2015 arXiv

  50. [2017]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  51. [2021]

    InProceed- ings of the 44th international ACM SIGIR conference on research and development in information retrieval

    Optimizing dense retrieval model training with hard negatives. InProceed- ings of the 44th international ACM SIGIR conference on research and development in information retrieval. 1503–1512

  52. [2024]

    InFindings of the Association for Computational Linguistics: ACL 2024

    Distillation enhanced generative retrieval. InFindings of the Association for Computational Linguistics: ACL 2024. 11119–11129

    2024

  53. [2025]

    InProceedings of the AAAI Conference on Artificial Intelligence, Vol

    DOGR: Leveraging Document-Oriented Contrastive Learning in Generative Retrieval. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 24732–24740

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.