REVIEW 3 major objections 5 minor 43 references
Temporal Collaborative Ranking Via Personalized Transformer
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read SSE-PT, a personalized Transformer that concatenates user embeddings with item embeddings and regularizes them via stochastic shared embeddings, outperforms the unpersonalized SASRec by almost 5% in NDCG@10 on five real-world datasets.
desk verdict A solid, incremental extension of SASRec that makes personalization work via SSE-SE regularization; the empirical claim is plausible but the sampled-negative evaluation protocol leaves the headline 4-5% gain less certain than the abstract suggests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Personalized Transformer (SSE-PT): a modified Transformer encoder in which each position t of the input sequence is the concatenation [v_{j_t}; u_i] plus a positional encoding p_t, so that the self-attention blocks are conditioned on who the user is. The prediction layer scores every candidate item l by the dot product of the last output representation with [v_l; u_i]. The load-bearing regularizer is SSE-SE (Stochastic Shared Embeddings, 'shared' because an embedding is stochastically replaced by another embedding during SGD with probability p), applied to user embeddings, input item embeddings, and output item embeddings. The long-sequence variant SSE-PT++ modifies how sequences are fed in: with probability p_s it samples a starting index uniformly from the available history and takes a window of length T; otherwise it uses the last T items, so that the model can use much longer histories while T stays small.
What would settle it
Run SSE-PT and SASRec on the same five datasets but compute NDCG@10 and Recall@10 by scoring every item in the catalog (no negative sampling), using the same hyperparameters; if the gain over SASRec shrinks to near zero or reverses, the central claim is an artifact of the sampled-candidate evaluation.
Extended reading notes
Core claim
In the paper's own terms, the discovery is that personalization, implemented as user embeddings concatenated with item embeddings at the input and reused in the prediction layer, substantially improves temporal collaborative ranking, and this only becomes possible when a stochastic-shared-embedding regularizer is applied to the embedding layers. The authors report that SSE-PT beats SASRec on Beauty, Games, Steam, Movielens1M, and Movielens10M, with double-digit relative improvements in NDCG@10 on most datasets, and that ablations show removing SSE-SE collapses the gain and can drop below the unpersonalized baseline. They further show that attention heat maps place more mass on recent items in long sequences than SASRec's, and that the SSE-PT++ variant, which truncates long histories by either sampling a window or taking the most recent T items, preserves most of the benefit at roughly the same training speed.
Load-bearing premise
The reported gains are measured under a negative-sampling evaluation where each user's next item is ranked against C sampled negatives rather than the full catalog, so the numbers assume that success on this sampled ranking transfers to true top-K ranking over all items.
Editorial extensions
If this is right
- On all five datasets, the personalized model beats the unpersonalized one, and the margin persists when the number of sampled negatives is varied from 100 to 2000.
- Removing SSE-SE regularization drops the personalized model below the unpersonalized baseline, so the regularizer, not just the extra parameters, is causally responsible for the gain.
- When the history window is shortened from 200 to 100, SSE-PT++ keeps most of its performance while SASRec degrades more, so the recency-based sampling strategy is more robust to truncation.
- The attention maps associate the model's better ranking with larger attention mass on recent items, giving an interpretable explanation rather than only a metric improvement.
Reading between the lines
- A natural next test is whether SSE-SE personalization transfers to other sequence recommenders (e.g., RNN- or CNN-based models), since the paper establishes the principle on a Transformer but the regularizer is architecture-agnostic.
- A quantitative recency metric, such as the average attention weight on the last 10% of positions versus a uniform baseline, could formalize the qualitative attention-map claim and make it testable across users.
- Because the reported evaluation samples negatives, the ~5% advantage is only guaranteed under that protocol; full-catalog NDCG@10 evaluations would show whether the gain survives a harder ranking task.
- The user-in-the-input move suggests that a Transformer could also treat the user id as a special token, or that session embeddings could be learned for anonymous visits, which the paper does not explore.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SSE-PT, a self-attentive sequential recommendation model that augments the SASRec Transformer architecture with per-user embedding vectors at both the input and prediction layers, regularized by Stochastic Shared Embeddings (SSE-SE). The authors report consistent improvements over SASRec and other baselines on Beauty, Games, Steam, ML-1M, and ML-10M, claim roughly 5% NDCG@10 gains, and introduce SSE-PT++, a variant that samples subsequences to handle long user histories. The paper also presents attention-map visualizations and an ablation study that isolates the contribution of personalization.
Significance. If the reported gains hold under full-catalog evaluation, this is a practically valuable contribution: it shows that a carefully regularized user embedding can be added to a Transformer-based sequential recommender without hurting training speed, and the open-sourced code and data support reproducibility. The personalization ablation in Table 11, the speed comparison in Figure 4, and the attention visualizations are useful concrete evidence for the proposed mechanism. The main limitation is that the headline quantitative claim rests entirely on a sampled-negative evaluation protocol, and the manuscript does not report variance or full-catalog numbers, so the magnitude of the improvement over SASRec is not yet firmly established.
major comments (3)
- [Abstract; Section 4.4; Tables 2 and 7] The abstract's claim of "outperforming SASRec by almost 5% in terms of NDCG@10 on 5 real-world datasets" is not supported by the reported tables. Table 2 contains only four datasets (Beauty, Games, Steam, ML-1M); ML-10M appears separately in Table 7, where the best NDCG@10 gain over SASRec is around 1.6% (0.7529 vs 0.7413). The claim should be restricted to the actual datasets and percentages, or the fifth dataset should be included in the main comparison table.
- [Section 4.2; Section 4.4; Table 11] All NDCG@10 and Recall@10 numbers are computed over a candidate set that consists of one known positive item plus C randomly sampled negatives, rather than over the full item pool. This is a load-bearing issue for the central comparison because the prediction score in Eq. (8) contains a user embedding that can act as a user-specific bias, lowering scores for uniformly sampled irrelevant items in a way that an unpersonalized model such as SASRec cannot. Table 11 extends C to 2000 on ML-1M but stops short of the full catalog of 3,416 items and compares only the personalized model against its unpersonalized ablation, not against SASRec. To substantiate the ranking claim, the authors should report full-catalog metrics (or at least C equal to the item count) and provide means and standard deviations over multiple seeds.
- [Tables 2, 3, 4, and 5; Section 4.4] The reported SASRec baseline numbers are internally inconsistent across tables for what appear to be the same ML-1M configuration: Table 2 reports NDCG@10 of 0.5936, Table 3 reports 0.5936 and 0.5919, Table 4 reports 0.5919 and 0.5769, and Table 5 reports 0.5961, 0.5941, and 0.5996. In addition, the text states that "SSE-PT++ achieves even higher 0.9292," which conflicts with the SSE-PT++ NDCG@10 value of 0.6292 in Table 5. These inconsistencies make the headline comparison difficult to audit and should be corrected.
minor comments (5)
- [Table 7] The table header says "Movielens1m" but the dataset is Movielens10M; this should be corrected.
- [Section 3.2.5, Eq. (8)] The sentence "r_itl is the predicted score of item l by user l at time point t" should refer to user i, not user l.
- [Section 3.2.3, Eq. (5)] The text says the resulting S is in R^{n x d}; this should be consistent with the sequence length T, typically R^{T x d} for one sequence or a batched form.
- [Table 8] The first row of the "USER-SIDE SSE-SE PROBABILITY" column is labeled "PARAMETERSHARING," which is not a probability value; this makes the table hard to read.
- [Section 4.5 and Figures 2, 3] The interpretability claim is based on a single user example and unquantified attention heat maps; this should be described as anecdotal or supplemented with a quantitative measure of attention concentration on recent items.
Circularity Check
No circularity: the claimed NDCG gain is an empirical benchmark comparison, and the main borrowed component (SSE-SE) is isolated by ablations rather than assumed.
full rationale
The paper's central claim ('outperforming SASRec by almost 5% in terms of NDCG@10 on 5 real-world datasets') is an empirical result measured against an external baseline under fixed train/validation/test splits. The architecture is SASRec plus user embeddings and SSE-SE regularization; the paper does not define the evaluation metric in terms of the model's own outputs, nor does it fit a parameter and then rename the fit as a prediction. Hyperparameters such as dropout, SSE probabilities, and the SSE-PT++ sampling probability are tuned on validation and reported on test (Section 4.4), which is standard practice rather than a circular reduction. The only substantial self-citation is [41] (Stochastic Shared Embeddings), but it is not load-bearing as a citation: Section 4.7 and Tables 6, 8, and 11 provide direct ablations showing that SSE-SE helps and that personalization helps even when regularization is held fixed. There is no uniqueness theorem imported from the authors' prior work and no ansatz smuggled in solely through a citation. The sampled-negative evaluation protocol (Section 4.2) is a shared evaluation limitation that could affect external validity, but both SSE-PT and SASRec are evaluated under the same protocol, so it does not make the comparison equivalent to the model's own inputs. Accordingly, no circular step is established and the paper is best scored as self-contained against external benchmarks.
Assumptions & free parameters
free parameters (5)
- SSE probability for user embeddings (p_u) =
0.92 (best on ML-1M, Table 5)
- SSE probabilities for input/output item embeddings (p_i, p_y) =
0.1 / 0.1 (best on ML-1M, Table 5)
- Sampling probability p_s (SSE-PT++) =
0.3 (best on ML-1M, Table 9)
- Evaluation candidate count C =
100 for most datasets, 500 for ML-10M
- Architecture hyperparameters (T, d_u, d_i, B, dropout) =
T=200 for ML-1M, T=50 for others; d_u=50, d_i=100; B=2-6; dropout=0.2
assumptions (4)
- domain assumption The next item a user engages with is the correct positive label for ranking at each time step.
- domain assumption Metrics computed on C sampled negative items approximate true top-K ranking metrics over all m items.
- domain assumption Single-split, single-seed results with validation-based hyperparameter selection are sufficient to establish the claimed improvement.
- domain assumption Baseline results taken from [16] are directly comparable because the same data splits and item embeddings are used.
Cite this review
Pith. "Pith review of Temporal Collaborative Ranking Via Personalized Transformer." pith.science (2026). https://pith.science/paper/HHVJUT24
@misc{pith2026190805435,
author = {Pith},
title = {Pith review of: Temporal Collaborative Ranking Via Personalized Transformer},
year = {2026},
howpublished = {\url{https://pith.science/paper/HHVJUT24}},
note = {Machine review of arXiv:1908.05435}
}
read the original abstract
The collaborative ranking problem has been an important open research question as most recommendation problems can be naturally formulated as ranking problems. While much of collaborative ranking methodology assumes static ranking data, the importance of temporal information to improving ranking performance is increasingly apparent. Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed us to make better use of the temporal ordering of items that each user has engaged with. In particular, the SASRec model, inspired by the popular Transformer model in natural languages processing, has achieved state-of-art results in the temporal collaborative ranking problem and enjoyed more than 10x speed-up when compared to earlier CNN/RNN-based methods. However, SASRec is inherently an un-personalized model and does not include personalized user embeddings. To overcome this limitation, we propose a Personalized Transformer (SSE-PT) model, outperforming SASRec by almost 5% in terms of NDCG@10 on 5 real-world datasets. Furthermore, after examining some random users' engagement history and corresponding attention heat maps used during the inference stage, we find our model is not only more interpretable but also able to focus on recent engagement patterns for each user. Moreover, our SSE-PT model with a slight modification, which we call SSE-PT++, can handle extremely long sequences and outperform SASRec in ranking results with comparable training speed, striking a balance between performance and speed requirements. Code and data are open sourced at https://github.com/wuliwei9278/SSE-PT.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016
arXiv 2016
-
[2]
Neural machine translation by jointly learning to align and translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014
arXiv 2014
-
[3]
Empirical evaluation of gated recurrent neural networks on sequence modeling
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014
arXiv 2014
-
[4]
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 14
arXiv 2018
-
[5]
MIT press Cambridge, 2016
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio.Deep learning, volume 1. MIT press Cambridge, 2016
2016
-
[6]
The movielens datasets: History and context
F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):19, 2016
work page 2016
-
[7]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
2016
-
[8]
Translation-based recommendation
Ruining He, Wang-Cheng Kang, and Julian McAuley. Translation-based recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems, pages 161–169. ACM, 2017
work page 2017
Show all 43 references
-
[9]
Recurrent neural networks with top-k gains for session-based recommendations
Balázs Hidasi and Alexandros Karatzoglou. Recurrent neural networks with top-k gains for session-based recommendations. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pages 843–852. ACM, 2018
2018
-
[10]
Session-based recommendations with recurrent neural networks
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939, 2015
2015 arXiv
-
[11]
Recommending and evaluating choices in a virtual community of use
Will Hill, Larry Stead, Mark Rosenstein, and George Furnas. Recommending and evaluating choices in a virtual community of use. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 194–201. ACM Press/Addison-Wesley Publishing Co., 1995
1995
-
[12]
Improving neural networks by preventing co-adaptation of feature detectors
Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhut- dinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012
2012 arXiv
-
[13]
Ridge regression: Biased estimation for nonorthogonal problems
Arthur E Hoerl and Robert W Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55–67, 1970
1970
-
[14]
Collaborative filtering for implicit feedback datasets
Yifan Hu, Yehuda Koren, and Chris V olinsky. Collaborative filtering for implicit feedback datasets. In Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on, pages 263–272. Ieee, 2008
2008
-
[15]
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015
2015 arXiv
-
[16]
Self-attentive sequential recommendation
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. arXiv preprint arXiv:1808.09781, 2018
2018 arXiv
-
[17]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
-
[18]
Factorization meets the neighborhood: a multifaceted collaborative filtering model
Yehuda Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 426–434. ACM, 2008
2008
-
[19]
The bellkor solution to the netflix grand prize
Yehuda Koren. The bellkor solution to the netflix grand prize. Netflix prize documentation, 81(2009):1–10, 2009
2009
-
[20]
Collaborative filtering with temporal dynamics
Yehuda Koren. Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 447–456. ACM, 2009
2009
-
[21]
Matrix factorization techniques for recom- mender systems
Yehuda Koren, Robert Bell, and Chris V olinsky. Matrix factorization techniques for recom- mender systems. Computer, (8):30–37, 2009
2009
-
[22]
A simple weight decay can improve generalization
Anders Krogh and John A Hertz. A simple weight decay can improve generalization. In Advances in neural information processing systems, pages 950–957, 1992. 15
1992
-
[23]
Stamp: short-term attention/memory priority model for session-based recommendation
Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. Stamp: short-term attention/memory priority model for session-based recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1831–1839. ACM, 2018
2018
-
[24]
Probabilistic matrix factorization
Andriy Mnih and Ruslan R Salakhutdinov. Probabilistic matrix factorization. In Advances in neural information processing systems, pages 1257–1264, 2008
2008
-
[25]
On the difficulty of training recurrent neural networks
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. In International Conference on Machine Learning, pages 1310–1318, 2013
2013
-
[26]
Bpr: Bayesian personalized ranking from implicit feedback
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence, pages 452–461. AUAI Press, 2009
2009
-
[27]
Factorizing personalized markov chains for next-basket recommendation
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web, pages 811–820. ACM, 2010
2010
-
[28]
Item-based collaborative filtering recommendation algorithms
Badrul Munir Sarwar, George Karypis, Joseph A Konstan, John Riedl, et al. Item-based collaborative filtering recommendation algorithms. Www, 1:285–295, 2001
2001
-
[29]
Collaborative filtering recom- mender systems
J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. Collaborative filtering recom- mender systems. In The adaptive web, pages 291–324. Springer, 2007
2007
-
[30]
Mining recommendations from the web
Guy Shani, Max Chickering, and Christopher Meek. Mining recommendations from the web. In Proceedings of the 2008 ACM conference on Recommender systems, pages 35–42. ACM, 2008
2008
-
[31]
Maximum-margin matrix factorization
Nathan Srebro, Jason Rennie, and Tommi S Jaakkola. Maximum-margin matrix factorization. In Advances in neural information processing systems, pages 1329–1336, 2005
2005
-
[32]
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958, 2014
1929
-
[33]
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112, 2014
2014
-
[34]
Personalized top-n sequential recommendation via convolutional sequence embedding
Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 565–573. ACM, 2018
2018
-
[35]
Regression shrinkage and selection via the lasso
Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267–288, 1996
1996
-
[36]
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems, pages 5998–6008, 2017
2017
-
[37]
Unifying user-based and item-based collaborative filtering approaches by similarity fusion
Jun Wang, Arjen P De Vries, and Marcel JT Reinders. Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, pages 501–5...
2006
-
[38]
Cofi rank-maximum margin matrix factorization for collaborative ranking
Markus Weimer, Alexandros Karatzoglou, Quoc V Le, and Alex J Smola. Cofi rank-maximum margin matrix factorization for collaborative ranking. In Advances in neural information processing systems, pages 1593–1600, 2008
2008
-
[39]
Large-scale collaborative ranking in near- linear time
Liwei Wu, Cho-Jui Hsieh, and James Sharpnack. Large-scale collaborative ranking in near- linear time. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 515–524. ACM, 2017. 16
2017
-
[40]
Sql-rank: A listwise approach to collaborative ranking
Liwei Wu, Cho-Jui Hsieh, and James Sharpnack. Sql-rank: A listwise approach to collaborative ranking. In Proceedings of Machine Learning Research (35th International Conference on Machine Learning), volume 80, 2018
2018
-
[41]
Stochastic shared embeddings: Data-driven regularization of embedding layers
Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. Stochastic shared embeddings: Data-driven regularization of embedding layers. arXiv preprint arXiv:1905.10630, 2019
1905 arXiv
-
[42]
Graph dna: Deep neighborhood aware graph encoding for collaborative filtering.arXiv preprint arXiv:1905.12217, 2019
Liwei Wu, Hsiang-Fu Yu, Nikhil Rao, James Sharpnack, and Cho-Jui Hsieh. Graph dna: Deep neighborhood aware graph encoding for collaborative filtering.arXiv preprint arXiv:1905.12217, 2019
1905 arXiv
-
[43]
Deep learning based recommender system: A survey and new perspectives
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), 52(1):5, 2019. 17
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.