REVIEW 5 major objections 4 minor 39 references
Cold items stay out of reach for semantic-ID recommenders
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
2026-08-01 08:26 UTC pith:WGWGH5MZ
load-bearing objection Useful temporal cold-start protocol for SID recommenders, but the token-level taxonomy's position-blind support definition weakens the headline structural claim. the 5 major comments →
Can Generative Recommendation Reach Cold Items? A Temporal Perspective on Semantic-ID Generation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On the paper's own terms, the central discovery is a reachability boundary: a cold item is reachable by a SID-based generative recommender only when its SID lies on a sufficiently supported semantic path. Using a token-level coldness taxonomy, the authors show that atomic token coverage is necessary but not sufficient — on one dataset 40.5% of cold items have all their tokens seen during training, yet Recall@20 is essentially zero — while items that share deep prefixes with training items are the ones occasionally hit. Oracle-prefix probing confirms the bottleneck is fine-grained path completion rather than coarse bucket selection: even with the correct first two tokens forced, unseen-item S
What carries the argument
The central object is the semantic ID (SID), a length-L sequence of discrete tokens assigned to each item, together with the training-time support sets of observed atomic tokens and observed prefixes. The argument hinges on reading autoregressive SID decoding as hierarchical semantic bucketing: token 1 selects a coarse semantic region, later tokens refine item-specific paths. This interpretation carries the diagnosis — seen/unseen splits, a token-level coldness taxonomy, oracle-prefix forcing, and distributional alignment with a discriminative scorer — and explains why a cold item is reachable only on a sufficiently supported path.
Load-bearing premise
The load-bearing premise is that the SID codebook is frozen and token support is defined only from items with training interactions; some training windows are also extremely thin, so if codebooks were updated with new-item content or training data grew, the reachability boundary could shrink or disappear.
What would settle it
Retrain or expand the SID codebook at each temporal window using only the content of items that will appear in the test window, while keeping the recommender's training interactions identical; if any-token-unseen items then receive non-negligible Recall@20, the claimed barrier is a consequence of the frozen-codebook protocol rather than of SID generation itself.
If this is right
- Under an absolute-time temporal split, both generative and discriminative recommenders collapse on truly unseen items; strong seen-target performance does not imply cold-start ability.
- A cold item whose SID tokens were all observed during training is still hard to reach, because residual-style SIDs form ordered paths; token coverage alone does not guarantee compositionality.
- Cold-item hits concentrate in deep-prefix-supported regimes; reachability is path-bounded, not just token-bounded.
- Changing the SID space to a factorized codebook, replacing exact decoding with candidate scoring, and injecting dynamic contextual signals expand different parts of the boundary, but none removes it entirely.
- Temporal evaluation should report seen/unseen splits and token-level support, or it will overstate open-world generalization.
Where Pith is reading between the lines
- I infer that a large part of the observed barrier may be an artifact of freezing the codebook: because the SID tokenizer is trained on item content, one could tokenize future items' content before any interaction occurs and update the token support set, which would shrink the any-token-unseen category. The paper does not test this.
- I infer that the hierarchical-bucketing view makes SID length a tunable reachability knob: shorter IDs with coarser buckets should improve cold-item expressibility at the cost of discrimination, which is a testable design trade-off the paper leaves open.
- I infer that the token-level taxonomy can serve as a pre-deployment diagnostic: by measuring what fraction of incoming items fall into all-token-seen versus any-token-unseen, an operator could predict whether a generative recommender will handle the next catalog wave.
- I infer that the scoring-interface results suggest a hybrid: use generation for candidate recall and scoring for final ranking, combining the semantic retrieval strength of SIDs with the softer reach of a scorer; the paper's variants point this way but do not combine them.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes an absolute-time sliding-window protocol for evaluating semantic-ID-based generative recommendation under item cold start. Using TIGER on seven datasets, it partitions test targets into seen/unseen items, introduces a token-level coldness taxonomy (all-token-seen, any-token-unseen, prefix-seen), performs oracle-prefix probing, compares TIGER's prefix generation with SASRec's bucket-aggregated scoring, and tests three controlled variants (TIGER-SID, TIGER-Scorer, TIGER-Edge). The main empirical finding is that unseen-item recall is near zero across datasets; even all-token-seen cold items are rarely reached, while reachability improves only for deep prefix-supported cases. The paper concludes that SID generation is compositional but not fully open-ended: cold-item reachability is bounded by the token vocabulary and path support observed in training.
Significance. If the conclusions hold, the paper is a useful diagnostic contribution: it exposes the gap between leave-one-out and temporal evaluation for generative retrieval, provides a reusable temporal protocol, and separates coarse bucket selection from path completion through oracle-prefix probing. The released code and pipeline, transparent category definitions, and one-component-at-a-time variants are strengths. However, the central 'structural reachability boundary' claim currently rests on a token-support definition that is position-agnostic, and the paper overstates the distinction between codebook availability and target-supervision support. The empirical pattern is likely robust, but the as-stated conclusion needs revision before the paper can be accepted.
major comments (5)
- [§3.3, Eq. (3.3); §4.2, Table 2 and Table 13] V_w^train is defined as a single position-independent set of atomic tokens, and Eq. (10) classifies a cold item as all-token-seen if each token appears anywhere in this set. In TIGER/RQ-VAE-style SIDs, each token position has its own output distribution and the later codes are residual-conditioned on earlier ones; observing a token at level ℓ does not make that token a valid or supervised target at level ℓ'. Thus an item can be counted as all-token-seen while requiring a level-2 token that was never observed at level 2. The near-zero Recall@20 for this category therefore does not by itself establish that 'atomic token coverage is necessary but not sufficient'; it may reflect missing position-specific supervision. The prefix-seen categories are position-aware and do show higher recall, which supports path-dependence, but the headline all-token-seen claim is weakened. Please redefine all-t
- [§4.1 and Introduction] The paper motivates the temporal protocol by stating that 'Compared with leave-one-out evaluation, TIGER suffers substantial degradation in Recall and NDCG' (§4.1). No leave-one-out results are reported in the main text or in the appendix (Tables 1–17 are all under the temporal protocol). This motivating comparison is therefore unverifiable. Please add the leave-one-out baseline numbers or remove/qualify the claim.
- [Table 6; §4.1, Table 1] Several test windows are extremely small: Beauty W3 has 12 training sequences, Sports W3 has 3, WeiboTech W3 has 134, and at W4 Beauty and Sports have only 86 and 69 training sequences. Near-zero unseen Recall in these regimes can be a floor effect from undertraining rather than a structural reachability boundary. Although the W5 windows and the larger DyTAG datasets still show near-zero unseen Recall, the paper should report whether the main conclusions hold when W3/W4 are excluded, and should provide variance or seed information. The word 'structural' is too strong unless training-size effects are controlled or analyzed.
- [§5, Eq. (15)–(17), Figure 4, Table 15] The bucket-alignment analysis computes P_SASRec over a candidate item set I_cand via item-level scores. Since SASRec is an ID-based model, items outside the training item universe have no representation and cannot be scored; unless I_cand is explicitly defined to include cold items with content-based embeddings, the coarse-to-fine alignment in Figure 4/Table 15 characterizes only the warm-item regime. As written, the analysis does not directly establish that 'a cold item is reachable only when its SID lies on a sufficiently supported semantic path.' Please specify I_cand and either provide a cold-item version of the alignment or soften the claim.
- [§3.3 and §6.1] The manuscript uses 'unseen atomic tokens' to mean tokens not present in training-period SIDs, but it never states how or when the SID codebook is learned. If the RQ-VAE/OPQ tokenizer is trained on the full item corpus, including post-training content, then those tokens are in the codebook vocabulary but absent only from training targets. The conclusion that the boundary is a property of 'current SID generation' would then not apply to systems that retrain or expand the codebook online with future content. Please state the tokenizer training window, and ideally run a variant where the codebook is retrained per window, to separate a supervision artifact from an intrinsic representational limit.
minor comments (4)
- [Table 3 and Table 14] The column layout for next-item vs next-token generation is hard to parse: the repeated 'Acc. Δ Acc. Δ' headers and the placement of the 'Free' columns should be reformatted for readability.
- [Figure 1 caption] The caption says 'all tokens seen, but new path contains unseen token,' which is self-contradictory. Clarify that the unseen element is the path or the position-specific token, not an atomic token.
- [References and template] The ACM template placeholders ('Conference acronym ’XX', '2018', 'https://doi.org/XXXXXXX.XXXXXXX') should be cleaned before submission. Several references are dated 2026; ensure the bibliography is in final form.
- [§3.2] The terms 'seen' and 'unseen' are used for both item-level and token-level conditions. Consider consistently using 'item-unseen' vs 'token-unseen' to avoid ambiguity.
Circularity Check
Core diagnostic is self-contained; one acknowledged definitional barrier is not load-bearing
specific steps
-
self definitional
[§4.2 (Table 2 paragraph), with Eq. (3.3) and Eq. (9)]
"Unseen atomic tokens form a harder barrier. For any-token-unseen cold items, performance is consistently zero across datasets. This is expected, since the decoder receives little or no supervision for generating tokens absent from the training targets."
Eq. (9) trains p_theta only on (u,i) in D_train, and Eq. (3.3) defines V_w^train as the tokens appearing in I_train. A token outside V_w^train therefore never occurs as a decoder target in training, so the observed zero/near-zero generation for any-token-unseen items is entailed by the objective and the taxonomy, not independently discovered. The paper itself calls it 'expected.' This tautology is not the load-bearing part of the central boundary claim: the all-token-seen and prefix-seen regimes are empirical measurements, and the SASRec bucket alignment is an external comparison.
full rationale
The paper is an empirical diagnostic rather than a derivation from fitted constants. The absolute-time protocol, seen/unseen split, token taxonomy, oracle-prefix probing, bucket alignment with SASRec, and the three controlled variants are all external measurements or ablations; no fitted parameter is renamed as a prediction. The only step that reduces by construction is the unsurprising near-zero performance on any-token-unseen items, which follows from the training loss (Eq. 9) and the definition of V_w^train (Eq. 3.3) and which the authors explicitly label as expected. The central contribution—that all-token-seen items are still hard while deeper prefix support helps—is an empirical result, not a consequence of the definitions. The GDGB datasets [18] are reused from the authors' own benchmark, but as data resources they are independent evidence, not an unverified self-citation theorem. A separate validity caveat (not circularity): V_w^train is position-agnostic, so 'all-token-seen' may mix positions, potentially confounding the structural-boundary interpretation; but this is a measurement-validity concern, not a circular derivation.
Axiom & Free-Parameter Ledger
free parameters (2)
- TIGER-Scorer temperature τ =
not reported
- temporal window boundaries =
5 windows; boundaries not formula-derived
axioms (3)
- domain assumption SID codebook is fixed at training time and token support is defined only over items in I_train (§3.3).
- domain assumption TIGER-Scorer has access to the candidate item set C_t at test time, including items that only appear during the test window (§A.2).
- domain assumption Bucket-aligned SASRec scores are treated as ground-truth semantic preference (§5, Eq. 16-17).
Cite this review
Pith. "Pith review of Can Generative Recommendation Reach Cold Items? A Temporal Perspective on Semantic-ID Generation." pith.science (2026). https://pith.science/paper/WGWGH5MZ
@misc{pith2026260721101,
author = {Pith},
title = {Pith review of: Can Generative Recommendation Reach Cold Items? A Temporal Perspective on Semantic-ID Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/WGWGH5MZ}},
note = {Machine review of arXiv:2607.21101}
}
read the original abstract
Semantic-ID-based generative recommendation represents items as sequences of shared semantic tokens, enabling token recombination beyond isolated item IDs. However, closed-world recombination does not necessarily imply temporal open-token cold-start induction, where new items enter the item catalog with unseen atomic tokens or weakly supported SID paths. In this work, we revisit SID-based generative recommendation under an absolute-time temporal protocol that separates seen and unseen targets and diagnoses the cold item reachability at the token level. Through seen/unseen-hit analysis, coldness taxonomy, and oracle-prefix probing, we show that current SID-based models can occasionally reach future items supported by observed tokens and prefixes, but struggle with unseen atomic tokens and unsupported SID paths. We further explain this boundary by interpreting SID generation as hierarchical semantic bucketing: early tokens select coarse semantic regions, while later tokens refine item-specific paths. These findings show that SID generation is compositional but not fully open-ended, and suggest future directions in more independent SID spaces, scoring-based interfaces, and dynamic textual context.
Figures
Reference graph
Works this paper leans on
-
[1]
Haoyue Bai, Le Wu, Min Hou, Miaomiao Cai, Zhuangzhuang He, Yuyang Zhou, Richang Hong, and Meng Wang. 2024. Multimodality invariant learning for multimedia-based new item recommendation. InProceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval. 677–686
2024
-
[2]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al
-
[3]
Yijie Ding, Zitian Guo, Jiacheng Li, Letian Peng, Shuai Shao, Wei Shao, Xiaoqiang Luo, Luke Simon, Jingbo Shang, Julian McAuley, et al . 2026. How Well Does Generative Recommendation Generalize?arXiv preprint arXiv:2603.19809(2026)
arXiv 2026
-
[4]
Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013. Optimized product quantization.IEEE transactions on pattern analysis and machine intelligence36, 4 (2013), 744–755
2013
-
[5]
Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2013. Optimized product quantization for approximate nearest neighbor search. InProceedings of the IEEE conference on computer vision and pattern recognition. 2946–2953
2013
-
[6]
Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). InProceedings of the 16th ACM conference on recommender systems. 299–315
2022
-
[7]
Danil Gusak, Anna Volodkevich, Anton Klenitskiy, Alexey Vasilev, and Evgeny Frolov. 2025. Time to split: Exploring data splitting strategies for offline evaluation of sequential recommenders. InProceedings of the Nineteenth ACM Conference on Recommender Systems. 874–883
2025
-
[8]
Yupeng Hou, Jiacheng Li, Ashley Shin, Jinsung Jeon, Abhishek Santhanam, Wei Shao, Kaveh Hassani, Ning Yao, and Julian McAuley. 2025. Generating long semantic ids in parallel for recommendation. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 956–966
2025
-
[9]
Yupeng Hou, Jianmo Ni, Zhankui He, Noveen Sachdeva, Wang-Cheng Kang, Ed H Chi, Julian McAuley, and Derek Zhiyuan Cheng. 2025. Actionpiece: Contextually tokenizing action sequences for generative recommendation.arXiv preprint arXiv:2502.13581(2025)
Pith/arXiv arXiv 2025
-
[10]
Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search.IEEE transactions on pattern analysis and machine intelligence33, 1 (2010), 117–128
2010
-
[11]
Yitong Ji, Aixin Sun, Jie Zhang, and Chenliang Li. 2023. A critical study on data leakage in recommender system offline evaluation.ACM Transactions on Information Systems41, 3 (2023), 1–27
2023
-
[12]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recom- mendation. In2018 IEEE international conference on data mining (ICDM). IEEE, 197–206
2018
-
[13]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization tech- niques for recommender systems.Computer42, 8 (2009), 30–37
2009
-
[14]
Xuan Nhat Lam, Thuc Vu, Trong Duc Le, and Anh Duc Duong. 2008. Addressing cold-start problem in recommendation systems. InProceedings of the 2nd inter- national conference on Ubiquitous information management and communication. 208–211
2008
-
[15]
Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. 2022. Autoregressive image generation using residual quantization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11523–11532
2022
-
[16]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel
-
[17]
Zaiqiao Meng, Richard McCreadie, Craig Macdonald, and Iadh Ounis. 2020. Ex- ploring data splitting strategies for the evaluation of recommendation models. In Proceedings of the 14th acm conference on recommender systems. 681–686
2020
-
[18]
Jie Peng, Jiarui Ji, Runlin Lei, Zhewei Wei, Yongchao Liu, and Chuntao Hong
-
[19]
Jie Peng, Rui Wang, Qiang Wang, Zhewei Wei, Bin Tong, Guan Wang, and Bo Zheng. 2025. Beyond Leakage and Complexity: Towards Realistic and Efficient Information Cascade Prediction.arXiv preprint arXiv:2510.25348(2025)
Pith/arXiv arXiv 2025
-
[20]
Jie Peng, Zhewei Wei, and Yuhang Ye. 2025. TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V
2025
-
[21]
Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, et al
-
[22]
Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock
-
[23]
Anima Singh, Trung Vu, Nikhil Mehta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Devansh Tandon, et al. 2024. Better generalization with semantic ids: A case study in ranking for recommendations. InProceedings of the 18th ACM Conference on Recommender Systems. 1039–1044
2024
-
[24]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang
-
[25]
Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. 2022. Transformer memory as a differentiable search index.Advances in neural information processing systems35 (2022), 21831–21843
2022
-
[26]
Aaron Van Den Oord, Oriol Vinyals, et al. 2017. Neural discrete representation learning.Advances in neural information processing systems30 (2017)
2017
-
[27]
Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. 2017. Dropoutnet: Ad- dressing cold start in recommender systems.Advances in neural information processing systems30 (2017)
2017
-
[28]
Wenjie Wang, Honghui Bao, Xinyu Lin, Jizhi Zhang, Yongqi Li, Fuli Feng, See- Kiong Ng, and Tat-Seng Chua. 2024. Learnable item tokenization for generative recommendation. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management. 2400–2409
2024
-
[29]
Jian Wei, Jianhua He, Kai Chen, Yi Zhou, and Zuoyin Tang. 2017. Collaborative filtering and deep learning based recommendation system for cold start items. Expert systems with applications69 (2017), 29–39
2017
-
[30]
Zhen Zhang, Jujia Zhao, Xinyu Ma, Xin Xin, Maarten de Rijke, and Zhaochun Ren. 2026. Cold-Starts in Generative Recommendation: A Reproducibility Study. arXiv preprint arXiv:2603.29845(2026)
Pith/arXiv arXiv 2026
-
[31]
Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending what video to watch next: a multitask ranking system. InPro- ceedings of the 13th ACM conference on recommender systems. 43–51
2019
-
[32]
Jieming Zhu, Mengqun Jin, Qijiong Liu, Zexuan Qiu, Zhenhua Dong, and Xiu Li
-
[2002]
InProceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval
Methods and metrics for cold-start recommendations. InProceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval. 253–260
-
[2015]
InProceedings of the 38th international ACM SIGIR conference on research and development in information retrieval
Image-based recommendations on styles and substitutes. InProceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 43–52
-
[2016]
InProceedings of the 1st workshop on deep learning for recommender systems
Wide & deep learning for recommender systems. InProceedings of the 1st workshop on deep learning for recommender systems. 7–10
-
[2019]
InProceedings of the 28th ACM international conference on information and knowledge management
BERT4Rec: Sequential recommendation with bidirectional encoder rep- resentations from transformer. InProceedings of the 28th ACM international conference on information and knowledge management. 1441–1450
-
[2023]
Recommender systems with generative retrieval.Advances in Neural Information Processing Systems36 (2023), 10299–10315
2023
-
[2024]
Cost: Contrastive quantization based semantic tokenization for generative recommendation. InProceedings of the 18th ACM Conference on Recommender Systems. 969–974. Can Generative Recommendation Reach Cold Items? A Temporal Perspective on Semantic-ID Generation Conference acronym ’XX, June 03–05, 2018, Woodstock, NY A TIGER-based Variant A.1 TIGER-SID: Enl...
arXiv 2018
-
[2026]
InThe Fourteenth International Conference on Learning Representations
GDGB: A Benchmark for Generative Dynamic Text-Attributed Graph Learning. InThe Fourteenth International Conference on Learning Representations. https://openreview.net/forum?id=5UFUHUC5qP
This paper was first reviewed by deepseek-v4-flash on August 1, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.