REVIEW 4 major objections 5 minor 50 references
The paper claims that a discriminative two-tower retriever beats generative LLM recommenders at a fraction of the cost.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 01:25 UTC pith:UT7S4JRC
load-bearing objection Well-ablated two-tower recipe, but the headline SOTA claim is undercut by an unspecified evaluation candidate set. the 4 major comments →
The Case Against Generation for Retrieval: Discriminative Language Models as Effective Retrievers
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that LLMs should be repurposed as semantic encoders rather than generators for candidate retrieval. The paper builds an LLM-native two-tower model with a shared user/item encoder, EOS-token pooling, cross-dataset transfer learning, candidate-set score-distribution distillation from a cross-encoder teacher, and one latent reasoning step confined to the user tower. It reports that this student improves Recall@10 over the generative OneRec-Think baseline by 4.3%, 31.6%, and 35.3% on Beauty, Sports and Outdoors, and Toys and Games respectively, while using only a 0.6B backbone instead of 8B. The cross-encoder teacher also sets state-of-the-art results on most dataset–metric
What carries the argument
The load-bearing mechanism is the knowledge-distillation transfer from a cross-encoder teacher to a factorized two-tower student. The teacher uses a yes/no verbalized relevance head plus user-conditioned next-token prediction on item text; the student shares a single LLM encoder with EOS pooling, and its candidate-set score distribution is aligned to the teacher's via KL divergence. Because the user tower alone performs one extra cached latent-reasoning step, item embeddings stay precomputable and retrieval remains a dot-product search over an offline index. The paper shows CE2TT distillation is the most consequential optional component: removing it drops Recall@10 by 8.0% to 23.1% across da
Load-bearing premise
The results assume the baseline numbers and the item pool used to compute Recall@10 are directly comparable across models; if the cross-encoder or student was evaluated over a narrower candidate set than the generative baselines, the reported gains could reflect evaluation scope rather than retrieval quality.
What would settle it
Re-run the proposed cross-encoder and two-tower models and the OneRec-Think baseline on identical preprocessing and an identical candidate pool (ideally full-corpus recall), and also compute metrics when the student retrieves over the same candidate set used for the teacher's distillation. If the gap on Recall@10 disappears or reverses under a matched candidate set, the 'case against generation' loses its empirical foundation.
If this is right
- If validated, web-scale retrieval can avoid autoregressive decoding entirely: users are encoded once, items are precomputed, and candidate generation becomes an ANN lookup.
- Distillation from a stronger cross-encoder teacher is sufficient to carry most of the teacher's ranking quality into an efficient student, so quality and latency can be decoupled.
- LLM-native two-tower retrieval is strikingly data-efficient, reaching production quality at 0.5% of the training data used by a classic DLRM retriever.
- Grounded token vocabulary makes the model resilient to staleness and gives favorable data-scaling behavior, unlike item-ID-based models that degrade quickly when item distributions shift.
- The architecture puts the expensive cross-encoder offline in the teacher role, making the online path a single vector search rather than a sequence generation.
Where Pith is reading between the lines
- The headline comparison may be sensitive to evaluation scope: the cross-encoder appears to be used with a candidate set of 50 during distillation, and the paper does not state whether the reported Recall@10 comes from a full-corpus evaluation or a reduced candidate pool. A matched-candidate re-evaluation is needed before the 'beats an 8B generative model' claim is taken at face value.
- The success of cross-dataset transfer suggests a single shared retrieval encoder could serve many catalogs or domains with minimal per-domain fine-tuning; this is testable on non-Amazon data with heterogeneous item text.
- Confining expensive latent reasoning to the user tower is a generally useful trick beyond recommendation: any query-side refinement that can be cached offline preserves the precomputation advantage in matching tasks such as search or ad retrieval.
- The case against generation is empirical rather than principled: a generative model given a matched inference budget, or trained with the same teacher signals, could conceivably close the gap on the same benchmarks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that generative LLM recommenders are impractical at scale and that a discriminative LLM-native two-tower retriever, augmented with a cross-encoder teacher, can match or beat state-of-the-art generative and classic baselines. Using a Qwen3-0.6B backbone, it combines a shared encoder with EOS pooling, cross-dataset transfer learning, candidate-set distillation from a cross-encoder with a yes/no head and auxiliary next-token prediction, and one latent reasoning step in the user tower. On Beauty, Sports and Outdoors, and Toys and Games, the paper reports that the cross-encoder is SOTA on 10 of 12 metric-dataset pairs and that the two-tower student surpasses OneRec-Think's R@10 on all three datasets. Internal production experiments report NE parity with a DLRM baseline at 0.5% of training data, QPS gains from serving optimizations, staleness resilience, and favorable data scaling.
Significance. If the empirical claims hold, the paper is significant: it would show that a 0.6B discriminative two-tower model can outperform an 8B generative retriever on standard benchmarks while preserving industrial serving efficiency, and that cross-encoder distillation plus shared-LLM representation learning closes much of the quality gap. The paper also provides unusually thorough ablations (all component combinations in Appendix E), concrete hyperparameters, and production serving details, which are valuable for reproducibility. However, the central comparison to prior SOTA rests on an underspecified evaluation protocol and copied baseline numbers, so the significance is conditional on closing that validation gap.
major comments (4)
- [§6.2.1, Eq. (1), Appendix C, Table 10] The evaluation candidate set is never specified. Eq. (1) defines retrieval over the full item set I, and Appendix C's Recall/NDCG definitions silently inherit that, but a cross-encoder cannot be scored over the full corpus without an explicit protocol (e.g., pairwise forward passes for ~22k test users × 12k items on Beauty). The only candidate-set number in the paper is Table 10's 'Teacher candidates K=50' for CE2TT distillation. If the reported CE (or TT) metrics were computed over a 50-item pool rather than the full corpus, the headline R@10 gains over ORT are not apples-to-apples. This is load-bearing for both the 'CE SOTA' and 'TT surpasses ORT' claims. Please state the exact candidate pool for every model and, if it is not full-corpus, recompute all baselines under the same pool.
- [§6.2.1, Table 1] Baseline numbers are copied from Table 1 of OneRec-Think without re-running or demonstrating protocol equivalence. The paper says its own data follow 'the same preprocessing protocol used by TIGER,' but it does not establish that ORT's reported numbers used that protocol, the same chronological split, the same metric computation, or the same candidate set. Since the paper's TT R@10 gain over ORT on Beauty is only 4.3% relative, small protocol differences could change the conclusion. The authors should either re-run the baselines under their protocol or provide concrete evidence that the ORT table is directly comparable.
- [§7.2.2, Tables 5–6; §8] Staleness and data-scaling evidence is presented only for the LLM-native cross-encoder, not for the two-tower retriever. Table 5 has no TT row and Table 6 reports only LLM-native CE, yet the abstract and conclusion attribute 'high resilience to model staleness' and 'superior data scaling' to the proposed LLM-native two-tower framework. Similarly, the NE-parity claims ('LLM-native TT matches ... using only 0.5% of training data') are stated in the text without a corresponding table, experimental detail, or error range. Please either restrict the claims to the cross-encoder or provide the missing two-tower evidence.
- [§6.2.3, Table 3; §6.2.1, Table 1] All experiments use a single seed (seed 42) and no error bars or significance tests. Several headline margins are small: TT R@10 gain over ORT on Beauty is 4.3% relative, and in Table 3 the R@5 difference between Full and w/o Latent is 0.0010 on Beauty and 0.0006 on Toys. Given that the paper makes strong 'surpasses SOTA' and component-contribution claims, at least a few repeated seeds or a statement of single-run variance is needed; without it, the finer ablations are not statistically distinguishable.
minor comments (5)
- [Table 1] The ORT row has missing spacing ('0.07910.03980.0471'); also clarify in the caption that parenthesized percentages are relative changes over ORT.
- [Appendix C / Table 10] Use distinct notation for the distillation candidate count (K=50) and the evaluation candidate set to avoid the ambiguity raised in the major comments.
- [§4.2.5] Eq. (19) writes the latent vector c_u inside the token sequence as [x_u; c_u; EOS]. Since c_u is a continuous hidden state, clarify whether it is injected as an embedding or as an extra token embedding; the mechanism is currently ambiguous.
- [Table 5] The 'ds+1' cell for Frozen DLRM is blank. State explicitly whether frozen DLRM is identical to the production baseline on ds+1 or whether that measurement was omitted.
- [§6.1 / Appendix B] Please state whether the validation split is used for early stopping or hyperparameter selection; the current text describes the split but not how it influences the reported test numbers.
Circularity Check
No significant circularity: central claims rest on external benchmarks and held-out evaluations; the single self-citation (MixLM) is a non-load-bearing design pointer.
full rationale
The paper's derivation chain is empirical rather than definitional. The cross-encoder teacher (Eqs. 11-14) is trained with contrastive and NTP losses, then evaluated by leave-one-out next-item retrieval on the held-out last item per user; the two-tower student is trained via CE2TT distillation (Eq. 17) and tested on the same held-out items. Gains for each component are isolated in Tables 2, 3, and 11, so no component is 'predicted' merely by re-using its own fitted values. The SOTA comparison in Table 1 imports baseline numbers from OneRec-Think, but that is an external published baseline, not a self-citation, and the comparison concern about candidate-set consistency is an evaluation-validity issue rather than circularity. The only self-citation is MixLM (Li et al., 2025), used to justify the yes/no scoring head design; that design is independently ablated in Table 2, so the citation is not load-bearing. No equation reduces to its own input, no fitted parameter is renamed as a prediction, and no uniqueness or ansatz is imported from the authors' prior work. The principal risks are protocol comparability and baseline integrity, not circularity.
Axiom & Free-Parameter Ledger
free parameters (6)
- distillation temperature T =
1.0 (Beauty), 0.5 (Sports, Toys)
- contrastive temperature tau =
0.07
- NTP loss weight lambda_ntp =
0.5
- KD loss weight lambda_KD =
0.2
- teacher candidate set size K =
50
- latent reasoning steps =
1
axioms (7)
- domain assumption EOS token pooling gives a better sequence embedding than mean pooling for decoder LLMs.
- domain assumption Distilling a cross-encoder's candidate-set score distribution improves two-tower ranking.
- domain assumption Sharing one encoder for users and items aligns the two sides in a common semantic space better than separate towers.
- domain assumption User-conditioned next-token prediction on item text adds recommendation-specific supervision without harming the contrastive objective.
- standard math In-batch negatives with a contrastive softmax are a valid approximation of full-corpus ranking.
- domain assumption The verbalized user/item features in Appendix A preserve the needed information and do not leak the target item through 'also-bought' or taste summary fields.
- domain assumption Internal NE improvements relative to an undisclosed production DLRM baseline are meaningful evidence of retrieval quality.
read the original abstract
Large Language Models (LLMs) have emerged as powerful assets for recommender systems. However, deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges. In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines. We introduce an LLM-native two-tower framework engineered for high-throughput, large-scale retrieval. Our architecture introduces several key innovations: a shared LLM encoder for joint user-item modeling, End-Of-Sentence (EOS) token pooling for compact sequence embedding, cross-dataset transfer learning, knowledge distillation from powerful cross-encoder teachers, and latent reasoning within the user tower. Extensive evaluation across three public benchmarks demonstrates that cross-encoder architecture outperforms current state-of-the-art (SoTA) models, while the efficient two-tower student achieves SoTA-comparable retrieval performance. Furthermore, experiments on internal large-scale production systems yield substantial topline retrieval improvements along with high resilience to model staleness and superior data scaling. Our findings demonstrate that when augmented with modern representation learning, the traditional two-tower paradigm remains an exceptionally competitive and practical solution for industrial retrieval systems.
Reference graph
Works this paper leans on
-
[1]
World Wide Web , volume =
A Survey on Large Language Models for Recommendation , author =. World Wide Web , volume =. 2024 , publisher =
2024
-
[2]
ACM Transactions on Information Systems , volume =
How Can Recommender Systems Benefit from Large Language Models: A Survey , author =. ACM Transactions on Information Systems , volume =. 2025 , publisher =
2025
-
[3]
Proceedings of the 16th ACM Conference on Recommender Systems , series =
Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) , author =. Proceedings of the 16th ACM Conference on Recommender Systems , series =. 2022 , publisher =
2022
-
[4]
Advances in Neural Information Processing Systems , volume =
Recommender Systems with Generative Retrieval , author =. Advances in Neural Information Processing Systems , volume =
-
[5]
Advances in Information Retrieval , series =
Large Language Models are Zero-Shot Rankers for Recommender Systems , author =. Advances in Information Retrieval , series =. 2024 , publisher =
2024
-
[6]
Proceedings of the 17th ACM Conference on Recommender Systems , series =
TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation , author =. Proceedings of the 17th ACM Conference on Recommender Systems , series =. 2023 , publisher =
2023
-
[7]
Proceedings of the 22nd ACM International Conference on Information & Knowledge Management , series =
Learning Deep Structured Semantic Models for Web Search Using Clickthrough Data , author =. Proceedings of the 22nd ACM International Conference on Information & Knowledge Management , series =. 2013 , publisher =
2013
-
[8]
Proceedings of the 10th ACM Conference on Recommender Systems , series =
Deep Neural Networks for YouTube Recommendations , author =. Proceedings of the 10th ACM Conference on Recommender Systems , series =. 2016 , publisher =
2016
-
[9]
Sentence-
Reimers, Nils and Gurevych, Iryna , booktitle =. Sentence-. 2019 , publisher =
2019
-
[10]
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing , series =
Dense Passage Retrieval for Open-Domain Question Answering , author =. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing , series =. 2020 , publisher =
2020
-
[11]
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , series =
Improving Text Embeddings with Large Language Models , author =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , series =. 2024 , publisher =
2024
-
[12]
2025 , url =
Lee, Chankyu and Roy, Rajarshi and Xu, Mengyao and Raiman, Jonathan and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei , booktitle =. 2025 , url =
2025
-
[13]
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , series =
Towards Universal Sequence Representation Learning for Recommender Systems , author =. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , series =. 2022 , publisher =
2022
-
[14]
Proceedings of the ACM Web Conference 2023 , series =
Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders , author =. Proceedings of the ACM Web Conference 2023 , series =. 2023 , publisher =
2023
-
[15]
2018 IEEE International Conference on Data Mining , series =
Self-Attentive Sequential Recommendation , author =. 2018 IEEE International Conference on Data Mining , series =. 2018 , publisher =
2018
-
[16]
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , series =
Hierarchical Gating Networks for Sequential Recommendation , author =. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , series =. 2019 , publisher =
2019
-
[17]
2019 , publisher =
Sun, Fei and Liu, Jun and Wu, Jian and Pei, Changhua and Lin, Xiao and Ou, Wenwu and Jiang, Peng , booktitle =. 2019 , publisher =
2019
-
[18]
4th International Conference on Learning Representations , series =
Session-based Recommendations with Recurrent Neural Networks , author =. 4th International Conference on Learning Representations , series =. 2016 , url =
2016
-
[19]
Passage Re-ranking with
Nogueira, Rodrigo and Cho, Kyunghyun , journal =. Passage Re-ranking with. 2019 , doi =
2019
-
[20]
2020 , publisher =
Khattab, Omar and Zaharia, Matei , booktitle =. 2020 , publisher =
2020
-
[21]
Proceedings of the 39th International Conference on Machine Learning , series =
In Defense of Dual-Encoders for Neural Ranking , author =. Proceedings of the 39th International Conference on Machine Learning , series =. 2022 , publisher =
2022
-
[22]
2022 , doi =
Cui, Zeyu and Ma, Jianxin and Zhou, Chang and Zhou, Jingren and Yang, Hongxia , journal =. 2022 , doi =
2022
-
[23]
2024 , publisher =
Ji, Jianchao and Li, Zelong and Xu, Shuyuan and Hua, Wenyue and Ge, Yingqiang and Tan, Juntao and Zhang, Yongfeng , booktitle =. 2024 , publisher =
2024
-
[24]
Uncovering
Dai, Sunhao and Shao, Ninglu and Zhao, Haiyuan and Yu, Weijie and Si, Zihua and Xu, Chen and Sun, Zhongxiang and Zhang, Xiao and Xu, Jun , booktitle =. Uncovering. 2023 , publisher =
2023
-
[25]
2024 , publisher =
Lyu, Hanjia and Jiang, Song and Zeng, Hanqing and Xia, Yinglong and Wang, Qifan and Zhang, Si and Chen, Ren and Leung, Christopher and Tang, Jiajie and Luo, Jiebo , booktitle =. 2024 , publisher =
2024
-
[26]
Computer , volume =
Matrix Factorization Techniques for Recommender Systems , author =. Computer , volume =. 2009 , publisher =
2009
-
[27]
2009 , publisher =
Rendle, Steffen and Freudenthaler, Christoph and Gantner, Zeno and Schmidt-Thieme, Lars , booktitle =. 2009 , publisher =
2009
-
[28]
Proceedings of the 26th International Conference on World Wide Web , series =
Neural Collaborative Filtering , author =. Proceedings of the 26th International Conference on World Wide Web , series =. 2017 , publisher =
2017
-
[29]
Proceedings of the 13th ACM Conference on Recommender Systems , series =
Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations , author =. Proceedings of the 13th ACM Conference on Recommender Systems , series =. 2019 , publisher =
2019
-
[30]
Companion Proceedings of the Web Conference 2020 , series =
Mixed Negative Sampling for Learning Two-Tower Neural Networks in Recommendations , author =. Companion Proceedings of the Web Conference 2020 , series =. 2020 , publisher =
2020
-
[31]
arXiv preprint arXiv:2110.15154 , year =
Cross-Batch Negative Sampling for Training Two-Tower Recommenders , author =. arXiv preprint arXiv:2110.15154 , year =
-
[32]
2022 , publisher =
Santhanam, Keshav and Khattab, Omar and Saad-Falcon, Jon and Potts, Christopher and Zaharia, Matei , booktitle =. 2022 , publisher =
2022
-
[33]
NIPS Deep Learning and Representation Learning Workshop , year =
Distilling the Knowledge in a Neural Network , author =. NIPS Deep Learning and Representation Learning Workshop , year =
-
[34]
2025 , doi =
Li, Guoyao and He, Ran and Jing, Shusen and Behdin, Kayhan and Wang, Yubo and Ramachandran, Sundara Raman and Nguyen, Chanh and Sheng, Jian and Ma, Xiaojing and Zhu, Chuanrui and Vasudevan, Sriram and Wu, Muchen and Ghosh, Sayan and Su, Lin and Song, Qingquan and Wang, Xiaoqing and Wang, Zhipeng and Lan, Qing and Chen, Yanning and Wu, Jingwei and Simon, L...
2025
-
[35]
Advances in Neural Information Processing Systems , volume =
Language Models are Few-Shot Learners , author =. Advances in Neural Information Processing Systems , volume =
-
[36]
2019 , publisher =
Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , booktitle =. 2019 , publisher =
2019
-
[37]
2024 , url =
BehnamGhader, Parishad and Adlakha, Vaibhav and Mosbach, Marius and Bahdanau, Dzmitry and Chapados, Nicolas and Reddy, Siva , booktitle =. 2024 , url =
2024
-
[38]
arXiv preprint arXiv:2411.18021 , year =
Can Bidirectional Encoder Become the Ultimate Winner for Downstream Applications of Foundation Models? , author =. arXiv preprint arXiv:2411.18021 , year =
-
[39]
Proceedings of the 25th International Conference on World Wide Web , series =
Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering , author =. Proceedings of the 25th International Conference on World Wide Web , series =. 2016 , publisher =
2016
-
[40]
arXiv preprint arXiv:2505.09388 , year =
Qwen3 Technical Report , author =. arXiv preprint arXiv:2505.09388 , year =
-
[41]
arXiv preprint arXiv:2412.06769 , year =
Training Large Language Models to Reason in a Continuous Latent Space , author =. arXiv preprint arXiv:2412.06769 , year =
-
[42]
Proceedings of the 41st International Conference on Machine Learning , series =
Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations , author =. Proceedings of the 41st International Conference on Machine Learning , series =. 2024 , publisher =
2024
-
[43]
arXiv preprint arXiv:2503.22675 , year =
Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation , author =. arXiv preprint arXiv:2503.22675 , year =
-
[44]
Liu, Zhanyu and Wang, Shiyao and Wang, Xingmei and Zhang, Rongzhou and Deng, Jiaxin and Bao, Honghui and Zhang, Jinghao and Li, Wuchao and Zheng, Pengfei and Wu, Xiangyu and Hu, Yifei and Hu, Qigen and Luo, Xinchen and Ren, Lejian and Zhang, Zixing and Wang, Qianqian and Cai, Kuo and Wu, Yunfan and Cheng, Hongtao and Cheng, Zexuan and Ren, Lu and Wang, Hu...
-
[45]
2025 , doi =
He, Ruining and Heldt, Lukasz and Hong, Lichan and Keshavan, Raghunandan and Mao, Shifan and Mehta, Nikhil and Su, Zhengyang and Tsai, Alicia and Wang, Yueqi and Wang, Shao-Chuan and Yi, Xinyang and Baugher, Lexi and Cakici, Baykal and Chi, Ed and Goodrow, Cristos and Han, Ningren and Ma, He and Rosales, Romer and Van Soest, Abby and Tandon, Devansh and W...
2025
-
[46]
2025 , doi =
Deng, Jiaxin and Wang, Shiyao and Cai, Kuo and Ren, Lejian and Hu, Qigen and Ding, Weifeng and Luo, Qiang and Zhou, Guorui , journal =. 2025 , doi =
2025
-
[47]
2025 , doi =
Zhou, Guorui and Hu, Hengrui and Cheng, Hongtao and Wang, Huanjie and Deng, Jiaxin and Zhang, Jinghao and Cai, Kuo and Ren, Lejian and Ren, Lu and Yu, Liao and Zheng, Pengfei and Luo, Qiang and Wang, Qianqian and Hu, Qigen and Huang, Rui and Tang, Ruiming and Wang, Shiyao and Yang, Shujie and Wu, Tao and Li, Wuchao and Luo, Xinchen and Wang, Xingmei and S...
2025
-
[48]
Finite Scalar Quantization:
Mentzer, Fabian and Minnen, David and Agustsson, Eirikur and Tschannen, Michael , journal =. Finite Scalar Quantization:. 2023 , doi =
2023
-
[49]
Li, Junnan and Li, Dongxu and Savarese, Silvio and Hoi, Steven C. H. , booktitle =. 2023 , publisher =
2023
-
[50]
arXiv preprint arXiv:1906.00091 , year =
Deep Learning Recommendation Model for Personalization and Recommendation Systems , author =. arXiv preprint arXiv:1906.00091 , year =. doi:10.48550/arXiv.1906.00091 , url =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.