Pith. sign in

REVIEW 3 major objections 5 minor 38 references

SnapLGR — an LLM-based generative retriever using PPR-enhanced semantic IDs — beats the incumbent T5-style system in a live A/B test, and the paper's offline analysis attributes most of the gain to the decoder-only architecture and model sc

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 01:21 UTC pith:6YFHACVJ

load-bearing objection A credible production validation of LLM-based generative retrieval, but the unstated temporal cutoff in the PPR graph leaves a leakage question that needs a clear answer before the attribution claims are taken at face value. the 3 major comments →

arxiv 2607.28895 v1 pith:6YFHACVJ submitted 2026-07-30 cs.IR

LLM-Based Generative Retrieval for Snapchat Content Recommendation

Liam Collins , Jiwen Ren , Donald Loveland , Bhuvesh Kumar , Clark Mingxuan Ju , Xuan Guo , Mo Li , Alvin Hou
show 14 more authors
Yi Cui Peng Yang Jian Wang Saud Afzal Shafi Nga Than Ruiming Lu Wenfeng Zhuo Dongheng Li Lili Zhang Mingtao Zhang Jinchao Ye Vincent Xue Chunhui Zhu Neil Shah
This is my paper
classification cs.IR
keywords generative retrievalsemantic IDsLLM recommendationshort-video recommendationPersonalized PageRankcontinued pretrainingproduction deploymentbeam search
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Generative retrieval recasts recommendation as a language-model task: instead of scoring every candidate video, a model directly generates discrete 'semantic IDs' for the next videos a user will engage with. This paper claims that swapping the standard small T5 encoder-decoder backbone for a pretrained decoder-only LLM makes this approach work at industrial scale. The system, SnapLGR, builds the IDs from visual and textual embeddings of videos, sharpened by a Personalized PageRank co-engagement signal, grounds the new ID tokens in the LLM before fine-tuning, and serves wide-beam search efficiently on GPUs. In a seven-day live A/B test it raised View Time by 0.37% and Deep Sessions by 0.18% over the incumbent TIGER-style baseline; offline, switching to a decoder-only architecture was the largest single contributor to the improvement, ahead of scaling and pretraining. A sympathetic reader would take the paper as evidence that the LLM-based approach is genuinely better, not just a promising research demo.

Core claim

The central claim is that an LLM-based generative retriever can outperform the established TIGER-style generative retrieval baseline in production. On the paper's own reported numbers, the launched SnapLGR system achieved +0.37% View Time, +0.09% Time Spent, +0.18% Deep Sessions, and +0.11% Deep Sessions Unique User in a 7-day A/B test (Table 8), with all p-values below 0.05. The authors also decompose the offline gap under a fixed tokenizer and report that the decoder-only Qwen3 backbone outperforms T5 at matched parameter counts (up to ~1.5× at 13M), that scaling helps both architectures but the decoder-only advantage persists, and that CPT plus pretrained initialization add small consiste

What carries the argument

The central object is the semantic identifier (SID) — a multi-level discrete code (e.g., three tokens from codebooks of sizes 1024/512/256) assigned to each video. SnapLGR's SIDs are produced by residual quantization of Qwen3-VL multimodal embeddings, with a Personalized PageRank co-engagement contrastive loss pulling co-watched videos together in the code space; this is what injects collaborative signal and reduces collisions. The generative backbone (Qwen3-0.6B) is then taught to emit these SIDs: first a frozen-LLM continued-pretraining stage grounds the new tokens in natural-language video descriptions, then full fine-tuning on user interaction sequences teaches next-item prediction. The

Load-bearing premise

The load-bearing premise is that the Personalized PageRank co-engagement graph used to build the semantic IDs is temporally clean: the paper does not state a cutoff for the 'eligible engagements' in §2.1/Appendix B.1.1 relative to the SFT training window and evaluation hour, so if future labels enter the SID space, the reported gains could be inflated by leakage.

What would settle it

Rebuild the SIDs from a PPR graph constructed only from engagements strictly before the SFT training window, rerun the offline retrieval table (RQ1) and, if possible, the 7-day A/B test; if the Recall@32/Pass@32 gap over T5 and the online lifts collapse, temporal leakage in the PPR graph was the driver rather than the LLM backbone.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Decoder-only backbones are structurally better suited to generative retrieval than encoder-decoder T5 at matched scale: at 13M parameters Qwen3 gives ~1.5× Pass@32 over T5, and this advantage persists up to 220M.
  • Model scaling helps both architectures, but decoder-only does not plateau as early: T5 gains +4.4% from 60M to 220M while Qwen3 gains +14.3%.
  • Pretrained LLM initialization plus continued pretraining (CPT) add consistent offline gains (~1–2% relative on Recall@10/32), even though CPT's explicit text grounding mostly dissipates after SFT.
  • PPR-supervised co-engagement contrastive learning improves SID codebook utilization from 31.8% to 48.2% and nearly doubles uniqueness, reducing collisions.
  • The system-level optimizations make the approach deployable: 3.63× training throughput and 45.7× inference throughput gains, enabling the launched A/B test.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the architecture effect is real, one would expect decoder-only generative-retrieval backbones to become the default, and a natural next test is an online A/B that holds the tokenizer fixed to confirm the offline attribution.
  • The paper does not report a temporal cutoff for the engagements feeding the PPR graph; a strict re-run with the graph built only from pre-training-window data would settle whether any future-label leakage inflates the reported gains.
  • The finding that CPT's grounding largely washes out after SFT suggests lighter vocabulary-initization schemes might capture most of the benefit with less training cost.
  • The same pipeline—multimodal embeddings, PPR-supervised quantization, and decode-only fine-tuning—is directly portable to other content domains such as long-form video, audio, or text items.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents SnapLGR, a production LLM-based generative retrieval system for short-video recommendation at Snapchat. SIDs are built by residual quantizing multimodal Qwen3-VL embeddings, augmented with a Personalized PageRank-based co-engagement contrastive loss. The added SID tokens are grounded via continued pretraining on video text descriptions, followed by SFT on user SID interaction sequences, and served using TensorRT-LLM beam search and a decentralized worker-loop architecture. Offline experiments report large retrieval gains over a TIGER-style T5 baseline and an attribution study isolating architecture, scaling, pretraining, and tokenizer effects. A 7-day live A/B test reports significant positive lifts in View Time (+0.37%), Time Spent (+0.09%), Deep Sessions (+0.18%), and Deep Sessions Unique User (+0.11%).

Significance. If the results hold, this is a valuable industrial demonstration that LLM-based generative retrieval can outperform a TIGER-style baseline online at scale. The live A/B test is a genuine external benchmark, and the reported p-values and confidence intervals are strengths. The offline attribution is also carefully structured: it fixes the tokenizer when comparing architecture/scale/pretraining, and it provides concrete throughput engineering numbers (3.63× training, 45.7× inference speedups). The main correctness risk is temporal leakage in the PPR co-engagement graph that defines the SID space; if the graph includes evaluation-period engagements, both the offline metrics and the collaborative-signal attribution are not identifiable. Additional weaknesses are the absence of uncertainty quantification in all offline tables and incomplete reporting of training compute in the scaling/architecture experiments.

major comments (3)
  1. [§2.1, Eq. (8), Appendix B.1.1, and §4.1] The PPR graph used for SID construction is described as built from 'eligible engagements' with no temporal cutoff. §4.1 gives explicit temporal splits for SFT (1-day window) and CPT (1-month ending 1 week before SFT), with evaluation in the immediately following hour. If the PPR graph includes engagements from the evaluation hour, targets co-engaged with history items in that period are pulled into the same SID, leaking future labels into the representation and inflating Recall@k/Pass@k; the collaborative-signal contribution in §4.3 is then unidentifiable. The same ambiguity affects the A/B if the production tokenizer is refreshed during the test using outcome-period data. Please state the exact training interval of the engagement graph and rerun the tokenizer ablations with edges truncated before the SFT window; also specify the tokenizer refresh policy during the 7-day A/B.
  2. [Tables 3–5 and 9–14] All offline retrieval results are point estimates from single runs; no confidence intervals, significance tests, seeds, or sample counts are reported. The headline gaps are large (e.g., P@10 21.78 vs 8.685), but the attribution effects in Tables 4 and 5 are small (e.g., P@10 +0.83% for CPT; R@32 10.98 vs 10.58 for Qwen3-VL vs MM tokenizer). Without bootstrap CIs over requests/users or multiple seeds, the claims that these effects are 'consistent' and 'practically relevant' are not supported. Please provide uncertainty estimates at least for the core attribution tables.
  3. [Figure 4, §4.3.2, Appendix C.3] The architecture/scaling attribution reports model sizes and learning rates for the T5 runs only, but no training-step or token budgets (or FLOPs) for the downscaled Qwen3 runs. The conclusion that the decoder-only backbone and model scaling 'contribute most significantly' could be confounded by unequal optimization budgets. Please report the training budget for each curve and, ideally, add iso-compute comparisons.
minor comments (5)
  1. [§4.1, Table 3] The sentence 'To cleanly isolate model architecture and capacity' is imprecise: Table 3 varies both architecture and model size simultaneously. The matched-scale architecture isolation is in Figure 4; please rephrase to avoid implying that Table 3 is capacity-matched.
  2. [§2.1] The P75 threshold for PPR positive pairs is described as 'the P75 score across all pairs' without specifying the population (training set? corpus? time window?). Define the reference distribution exactly.
  3. [§4.5, Table 8] Please report the randomization unit, number of users, whether the four metrics were pre-specified, and whether p-values are one- or two-sided. This will make the online claim easier to interpret.
  4. [§3.2] The Text-Grounding RSA metric compares SID embeddings against text embeddings of descriptions that were used as supervision during CPT. High RSA after CPT is thus partly mechanical; the drop after SFT is informative, but the interpretation of RSA as 'grounding quality' should acknowledge this construction.
  5. [§2.3] Typo: 'historical user SID sequences are daily by a TensorRT-LLM batch inference engine' appears to be missing a verb ('processed daily by').

Circularity Check

0 steps flagged

No significant circularity: A/B and offline attribution are externally grounded; minor self-referential RSA diagnostic and self-citations are not load-bearing.

full rationale

The central load-bearing claims are the 7-day live A/B result (Table 8) and the offline attribution of gains to architecture, scaling, and pretraining (Figure 4; Tables 3–5). The A/B is an external benchmark that is not used to fit SnapLGR, and the offline attribution holds the tokenizer fixed and varies controlled factors, so neither claim reduces to its own inputs by construction. The paper's self-citations to Ju et al. [12,13] for the RQ-VAE design and straight-through estimator are methodological inheritance: the relevant equations are re-derived in Section 2.1, and those citations do not justify the deployment outcome. The Text-Grounding RSA metric in Section 3.2 is partly self-referential, because CPT trains SID embeddings to generate text descriptions and RSA then measures correlation between SID embeddings and text embeddings; however, the paper uses RSA as a diagnostic rather than as evidence for retrieval gains, and the downstream benefit of CPT is separately quantified in Table 4. The main caveat is data-construction: the PPR co-engagement graph is built from 'eligible engagements' (Section 2.1; Eq. 8; Appendix B.1.1) with no temporal cutoff stated relative to the SFT window or evaluation hour. If those engagements included evaluation-hour labels, the SID representation would leak future labels into offline metrics and potentially the A/B. This is a potential leakage/identifiability problem, not a demonstrated circular reduction, because the paper does not specify the cutoff and no equation shows evaluation labels entering the PPR construction. Overall, the derivation is self-contained against an external online benchmark, so no significant circularity is established.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 0 invented entities

The paper's empirical system relies on several free hyperparameters (PPR threshold, alpha, tau, loss weights, codebook sizes, learning rates, materialization weights), none of which are given with sensitivity analyses. The main structural assumptions are about the validity of the embedding/description inputs, the temporal safety of the PPR graph, and the fairness of the T5 baseline. No new physical or conceptual entities are introduced; SID tokens are a learned representation, not an invented explanatory entity.

free parameters (7)
  • PPR positive-pair threshold (P75) = production threshold: P75 of PPR scores across all pairs
    Selected as the minimum threshold for contrastive positives; P50 vs P75 are compared in Table 1, but no held-out validation split is described.
  • PPR restart/transition parameter alpha = not reported
    Eq. 8 defines the PPR stationary solution with transition probability alpha; the value is not given, yet it shapes the co-engagement positive pairs.
  • Contrastive temperature tau = not reported
    Eq. 6 uses tau to scale the co-engagement contrastive loss; no value or tuning range is reported.
  • Tokenizer loss weights lambda_com, lambda_co = not reported
    The full tokenizer objective L = L_rec + lambda_com L_com + lambda_co L_co has unspecified weights that determine the balance between reconstruction and collaborative signal.
  • Codebook sizes and embedding dimension = (1024,512,256) for Qwen3-VL SIDs; (256,256,256) for MM baselines; first 1024 dims of a 4096-dim embedding
    Hand-chosen capacity settings; they directly limit SID expressiveness and collision resistance.
  • Per-size learning rates in scaling experiment = (13M, 1e-3), (60M, 3.4e-4), (220M, 2.6e-4)
    Learning rates are estimated with Step's Law anchored at 13M and then tuned; this affects the fairness of the architecture comparison.
  • Value-weighted materialization weights w_m = not reported
    Eq. 9 ranks videos inside each SID bucket by a weighted multi-metric value; the weights are not disclosed and influence video-level Recall.
axioms (6)
  • domain assumption Qwen3-VL-Embedding-8B embeddings capture sufficient multimodal semantics for short-video recommendation.
    Used as the sole input signal for SID construction; truncation to the first 1024 dimensions relies on Matryoshka representation validity (§B.1).
  • domain assumption Gemini 2.5 Flash text descriptions are faithful to video content.
    Descriptions feed both the multimodal embedding input and the CPT grounding task; systematic caption errors would degrade SID semantics.
  • domain assumption PPR co-engagement positives on the bipartite user-video graph are a valid training signal that does not leak future engagements.
    §2.1 and Appendix B.1.1 do not state the temporal cutoff for the engagement graph relative to the SFT/evaluation split; leakage would inflate results.
  • domain assumption Offline SID-level Pass@k and Recall@k with beam width 32 are valid proxies for online engagement.
    Used as the primary offline metrics; the online A/B is the eventual check, but all component attribution in §4.3 relies on this proxy.
  • domain assumption T5 baselines in the scaling comparison were fairly tuned.
    Learning rates are estimated via Step's Law (§C.3) with no demonstrated compute-equivalence or tuning-budget parity between T5 and Qwen3.
  • domain assumption The full-codebook straight-through estimator from Ju et al. behaves as described.
    Adopted from prior work without re-derivation; if the gradient surrogate is biased, SID quality and retrieval conclusions could shift.

pith-pipeline@v1.3.0-alltime-deepseek · 19981 in / 13044 out tokens · 147593 ms · 2026-08-03T01:21:50.294362+00:00 · methodology

0 comments
read the original abstract

Pretrained large language models (LLMs) are promising retrieval engines because they combine rich semantic priors, strong sequence modeling capabilities, and favorable scaling behavior. However, turning a pretrained LLM into a generative retriever in production deployment raises several challenges: the model must learn an internal item vocabulary that was absent from pretraining, and generate valid item identifiers under strict latency and cost constraints. We address these challenges through the design and launch of SnapLGR, an LLM-based generative retrieval system for short-video recommendation at Snapchat. The system is built around three main designs. First, we construct semantic identifiers (SIDs) from multimodal item embeddings and enhance them with Personalized PageRank (PPR)-based co-engagement contrastive learning, resulting in improved codebook utilization, reduced collisions, and infused collaborative signal. Second, we use continued pretraining (CPT) to ground the introduced SID tokens before supervised fine-tuning (SFT) on user interaction sequences. Third, we make SnapLGR serving practical through TensorRT-LLM CUDA-backed beam search and a decentralized worker-loop architecture. In a live A/B test, the launched system increased View Time by 0.37%, Time Spent by 0.09%, Deep Sessions by 0.18%, and Deep Sessions Unique User by 0.11% relative to the existing TIGER-style generative retrieval baseline. We then decompose this offline gap under a fixed tokenizer and quantify the gains due to model architecture, scaling, and pretraining. Overall, our deployment shows that successful production SnapLGR requires joint design across representation learning, vocabulary grounding, and efficient training and serving.

Figures

Figures reproduced from arXiv: 2607.28895 by Alvin Hou, Bhuvesh Kumar, Chunhui Zhu, Clark Mingxuan Ju, Donald Loveland, Dongheng Li, Jian Wang, Jinchao Ye, Jiwen Ren, Liam Collins, Lili Zhang, Mingtao Zhang, Mo Li, Neil Shah, Nga Than, Peng Yang, Ruiming Lu, Saud Afzal Shafi, Vincent Xue, Wenfeng Zhuo, Xuan Guo, Yi Cui.

Figure 1
Figure 1. Figure 1: SnapLGR Training pipeline. Stage 1: Tokenization (§2.1). We train an RQ-VAE on multimodal Qwen3-VL embeddings with PPR-based co-engagement supervision to produce the video-to-SID map. Stage 2: CPT (§2.2.1). We ground newly added SID embeddings in the LLM’s vocabulary using video text descriptions. Stage 3: SFT (§2.2.2). We fine-tune the full LLM to predict future engaged SIDs from users’ SID interaction hi… view at source ↗
Figure 2
Figure 2. Figure 2: illustrates our end-to-end serving pipeline, which is struc￾turally divided into offline batch generation and online serving stages. In the offline stage, historical user SID sequences are daily by a TensorRT-LLM batch inference engine to generate predicted future SIDs. Because SIDs represent semantic clusters rather than exact items, they pass through a value-weighted materialization step using a precompu… view at source ↗
Figure 3
Figure 3. Figure 3: SID collision concentration. SIDs are ordered by de [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Pass@32 (left) and Recall@32 (right) for Qwen3 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Illustrative CPT training prompt. The video corpus [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example SID-to-video mappings. Each row con [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 12 linked inside Pith

  1. [1]

    Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. Au- toregressive Entity Retrieval. InInternational Conference on Learning Represen- tations (ICLR). https://openreview.net/forum?id=5k8F6UU39V LLM-Based Generative Retrieval for Snapchat Content Recommendation Conference’17, July 2017, Washington, DC, USA

    2021

  2. [2]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261(2025)

    Pith/arXiv arXiv 2025

  3. [3]

    Tri Dao. 2024. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. InThe Twelfth International Conference on Learning Represen- tations (ICLR). https://arxiv.org/abs/2307.08691

    Pith/arXiv arXiv 2024

  4. [4]

    Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. 2025. OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment. arXiv preprint arXiv:2502.18965. arXiv:2502.18965 [cs.IR] https://arxiv.org/abs/2502.18965

    Pith/arXiv arXiv 2025

  5. [5]

    Hamed Firooz, Maziar Sanjabi, Adrian Englhardt, Aman Gupta, Ben Levine, Dre Olgiati, Gungor Polatkan, Iuliia Melnychuk, Karthik Ramgopal, Kirill Talanine, Kutta Srinivasan, Luke Simon, Natesh Sivasubramoniapillai, Necip Fazil Ayan, Qingquan Song, Samira Sriram, Souvik Ghosh, Tao Song, Vignesh Kothapalli, Xi- aoling Zhai, Ya Xu, Yu Wang, and Yun Dai. 2025....

    arXiv 2025

  6. [6]

    Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as Language Processing (RLP): A Unified Pretrain, Personal- ized Prompt & Predict Paradigm (P5). InProceedings of the 16th ACM Conference on Recommender Systems (RecSys). doi:10.1145/3523227.3546767

    arXiv 2022

  7. [7]

    Ruidong Han, Bin Yin, Shangyu Chen, He Jiang, Fei Jiang, Xiang Li, Chi Ma, Min- cong Huang, Xiaoguang Li, Chunzhen Jing, Yueming Han, Menglei Zhou, Lei Yu, Chuan Liu, and Wei Lin. 2025. MTGR: Industrial-Scale Generative Recommen- dation Framework in Meituan. InProceedings of the 34th ACM International Con- ference on Information and Knowledge Management (...

    doi:10.1145/3746252 2025

  8. [8]

    Taher H Haveliwala. 2002. Topic-sensitive pagerank. InProceedings of the 11th international conference on World Wide Web. 517–526

    2002

  9. [9]

    Ruining He, Lukasz Heldt, Lichan Hong, Raghunandan Keshavan, Shifan Mao, Nikhil Mehta, Zhengyang Su, Alicia Tsai, Yueqi Wang, Shao-Chuan Wang, Xinyang Yi, Lexi Baugher, Baykal Cakici, Ed Chi, Cristos Goodrow, Ningren Han, He Ma, Romer Rosales, Abby Van Soest, Devansh Tandon, Su-Lin Wu, Weilong Yang, and Yilin Zheng. 2026. PLUM: Adapting Pre-trained Langua...

    arXiv 2026

  10. [10]

    Rae, Oriol Vinyals, and Laurent Sifre

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Jo- hannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Si- monyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent S...

    2022

  11. [11]

    Yanhua Huang, Yuqi Chen, Xiong Cao, Rui Yang, Mingliang Qi, Yinghao Zhu, Qingchang Han, Yaowei Liu, Zhaoyu Liu, Xuefeng Yao, Yuting Jia, Leilei Ma, Yinqi Zhang, Taoyu Zhu, Liujie Zhang, Lei Chen, Weihang Chen, Min Zhu, Ruiwen Xu, and Lei Zhang. 2025. Towards Large-scale Generative Rank- ing. arXiv preprint arXiv:2505.04180; presented at GenAIRecP @ KDD 20...

    Pith/arXiv arXiv 2025

  12. [12]

    Clark Mingxuan Ju, Liam Collins, Leonardo Neves, Bhuvesh Kumar, Louis Yufeng Wang, Tong Zhao, and Neil Shah. 2025. Generative Recom- mendation with Semantic IDs: A Practitioner’s Handbook. InProceedings of the 34th ACM International Conference on Information and Knowledge Management. 6420–6425

    2025

  13. [13]

    Clark Mingxuan Ju, Tong Zhao, Leonardo Neves, Liam Collins, Bhuvesh Ku- mar, Jiwen Ren, Lili Zhang, Wenfeng Zhuo, Vincent Zhang, Xiao Bai, Jinchao Li, Karthik Iyer, Zihao Fan, Yilun Xu, Yiwen Chen, Peicheng Yu, Manish Malik, and Neil Shah. 2026. Semantic IDs for Recommender Systems at Snapchat: Use Cases, Technical Challenges, and Design Choices.arXiv pre...

    Pith/arXiv arXiv 2026

  14. [14]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling Laws for Neural Language Models. arXiv preprint arXiv:2001.08361. arXiv:2001.08361 [cs.LG]

    Pith/arXiv arXiv 2020

  15. [15]

    Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. 2008. Represen- tational similarity analysis-connecting the branches of systems neuroscience. Frontiers in systems neuroscience2 (2008), 4

    2008

  16. [16]

    Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, et al. 2022. Matryoshka representation learning.Advances in Neural Information Processing Systems35 (2022), 30233–30249

    2022

  17. [17]

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. 2022. Autoregressive Image Generation using Residual Quantization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11523–11532

    2022

  18. [18]

    Houyi Li, Wenzheng Zheng, Jingcheng Hu, Qiufeng Wang, Hanshan Zhang, Zili Wang, Yangshijie Xu, Shuigeng Zhou, Xiangyu Zhang, and Daxin Jiang. 2025. Predictable Scale: Part I – Optimal Hyperparameter Scaling Law in Large Lan- guage Model Pretraining. arXiv:2503.04715 [cs.LG] https://arxiv.org/abs/2503. 04715

    Pith/arXiv arXiv 2025

  19. [19]

    Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2026. Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Frame- work for State-of-the-Art Multimodal Retrieval and Ranking.arXiv preprint arXiv:2601.04720(2026). arXiv:2601.04720 [cs.CV] https://arxiv.o...

    Pith/arXiv arXiv 2026

  20. [20]

    Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. 2024. LLaRA: Large Language-Recommendation Assistant. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 1785–1795

    2024

  21. [21]

    Qingyun Liu, Bo Yan, Yang Liu, Yuji Roh, Ekansh Sharma, Likang Yin, Emma Olowo, Min hsuan Tsai, Yuxuan Li, Diego Uribe, Saksham Aggarwal, Siqi Wu, Yuan Hao, Vikas Kedigehalli, Lukasz Heldt, Lichan Hong, Li Wei, and Xinyang Yi. 2026. TokenMinds: Pretrained User Tokens and Embeddings for User Un- derstanding in Large Recommender Systems. arXiv preprint arXi...

    Pith/arXiv arXiv 2026

  22. [22]

    Zhanyu Liu, Shiyao Wang, Xingmei Wang, Rongzhou Zhang, Jiaxin Deng, Honghui Bao, Jinghao Zhang, Wuchao Li, Pengfei Zheng, Xiangyu Wu, Yifei Hu, Qigen Hu, Xinchen Luo, Lejian Ren, Zixing Zhang, Qianqian Wang, Kuo Cai, Yunfan Wu, Hongtao Cheng, Zexuan Cheng, Lu Ren, Huanjie Wang, Yi Su, Ruiming Tang, Kun Gai, and Guorui Zhou. 2025. OneRec-Think: In-Text Rea...

    arXiv 2025

  23. [23]

    NVIDIA Corporation. 2023. TensorRT-LLM: A TensorRT Toolbox for Opti- mized Large Language Model Inference. GitHub repository, https://github.com/ NVIDIA/TensorRT-LLM

    2023

  24. [24]

    OneRec Team, Biao Yang, Boyang Ding, Chenglong Chu, Dunju Zang, Fei Pan, Han Li, Hao Jiang, Honghui Bao, Huanjie Wang, Jian Liang, Jiangxia Cao, Jiao Ou, Jiaxin Deng, Jinghao Zhang, Kun Gai, Lu Ren, Peiru Du, Pengfei Zheng, Rongzhou Zhang, Ruiming Tang, Shiyao Wang, Siyang Mao, Siyuan Lou, Teng Shi, Wei Yuan, Wenlong Xu, Xingchen Liu, Xingmei Wang, Xinqi ...

    Pith/arXiv arXiv 2026

  25. [25]

    Qwen Team. 2025. Qwen3-0.6B. https://huggingface.co/Qwen/Qwen3-0.6B. Hugging Face model card

    2025

  26. [26]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.Journal of Ma- chine Learning Research21, 140 (2020), 1–67. https://www.jmlr.org/papers/v21/ 20-074.html

    2020

  27. [27]

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, Maciej Kula, Ed Chi, and Maheswaran Sathiamoorthy. 2023. Recommender Systems with Generative Retrieval. InAdvances in Neural Information Processing Systems 36 (NeurIPS)

    2023

  28. [28]

    Chi, and Xinyang Yi

    Anima Singh, Trung Vu, Nikhil Mehta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Devansh Tan- don, Ed H. Chi, and Xinyang Yi. 2024. Better Generalization with Seman- tic IDs: A Case Study in Ranking for Recommendations. InProceedings of the 18th ACM Conference on Recommender Systems (RecSys). ACM, Bari, ...

    arXiv 2024

  29. [29]

    Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W

    Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. 2022. Transformer Memory as a Differentiable Search Index. In Advances in Neural Information Processing Systems 35 (NeurIPS)

    2022

  30. [30]

    Wenjie Wang, Honghui Bao, Xinyu Lin, Jizhi Zhang, Yongqi Li, Fuli Feng, See- Kiong Ng, and Tat-Seng Chua. 2024. Learnable Item Tokenization for Generative Recommendation. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management (CIKM). ACM, Boise, ID, USA, 2400–

    2024

  31. [31]

    Yuhao Yang, Zhi Ji, Zhaopeng Li, Yi Li, Zhonglin Mo, Yue Ding, Kai Chen, Zijian Zhang, Jie Li, Shuanglong Li, and Lin Liu. 2025. Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations. InAdvances in Neural Information Process- ing Systems, Vol. 38. https://papers.nips.cc/paper_files/paper/2025/hash/ 86ba836d4c5dd...

    2025

  32. [32]

    Chao Yi, Dian Chen, Gaoyang Guo, Jiakai Tang, Jian Wu, Jing Yu, Mao Zhang, Sunhao Dai, Wen Chen, Wenjun Yang, Yuning Jiang, Zhujin Gao, Bo Zheng, et al. 2025. RecGPT Technical Report. arXiv preprint arXiv:2507.22879. arXiv:2507.22879 [cs.IR] https://arxiv.org/abs/2507.22879 Conference’17, July 2017, Washington, DC, USA [Authors]

    Pith/arXiv arXiv 2025

  33. [33]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, Yinghai Lu, and Yu Shi. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Gen- erative Recommendations. InProceedings of the 41st International Conference on Machine Learning (ICML) (Proceedings of Machine Lear...

    2024

  34. [34]

    Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, Ming Chen, and Ji-Rong Wen. 2024. Adapting Large Language Models by Integrating Col- laborative Semantics for Recommendation. InProceedings of the 40th IEEE Inter- national Conference on Data Engineering (ICDE)

    2024

  35. [35]

    Guorui Zhou, Honghui Bao, Jiaming Huang, Jiaxin Deng, Jinghao Zhang, Junda She, Kuo Cai, Lejian Ren, Lu Ren, Qiang Luo, Qianqian Wang, Qigen Hu, Rongzhou Zhang, Ruiming Tang, Shiyao Wang, Wuchao Li, Xiangyu Wu, Xinchen Luo, Xingmei Wang, Yifei Hu, Yunfan Wu, Zhanyu Liu, Zhiyang Zhang, Zixing Zhang, et al. 2025. OpenOneRec Technical Report. arXiv preprint ...

    arXiv 2025

  36. [36]

    Guorui Zhou, Jiaxin Deng, Jinghao Zhang, Kuo Cai, Lejian Ren, Qiang Luo, Qian- qian Wang, Qigen Hu, Rui Huang, Shiyao Wang, Weifeng Ding, Wuchao Li, Xinchen Luo, Xingmei Wang, Zexuan Cheng, Zixing Zhang, et al. 2025. OneRec Technical Report. arXiv preprint arXiv:2506.13695. arXiv:2506.13695 [cs.IR] https://arxiv.org/abs/2506.13695

    arXiv 2025

  37. [37]

    mega-collision

    Jing Zhu, Mingxuan Ju, Yozen Liu, Danai Koutra, Neil Shah, and Tong Zhao. 2025. Beyond Unimodal Boundaries: Generative Recommendation with Multimodal Semantics. arXiv preprint arXiv:2503.23333. arXiv:2503.23333 [cs.IR] https: //arxiv.org/abs/2503.23333 A Contributions Author roles are as follows: •Core Contributors:Liam Collins, Jiwen Ren •Contributors:Do...

    Pith/arXiv arXiv 2025

  38. [2409]

    doi:10.1145/3627673.3679569

    arXiv