{"id":"1b9cde4d-e0f7-4826-be84-862c509d2e9e","arxiv_id":"1908.05435","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Adding SSE-regularized user embeddings to the SASRec transformer improves next-item ranking accuracy on benchmark datasets, and a long-sequence variant preserves most of the gain at lower cost.","lead":"This paper adds a personal user profile to a transformer-based recommendation model and reports better next-item ranking than the previous SASRec method on benchmark datasets. It also adds a sampling trick for very long user histories, and releases code and data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline NDCG gain rests on a sampled-negative evaluation protocol; a full-catalog re-run is needed to confirm SSE-PT beats SASRec at true top-K ranking.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing threat to the paper's central empirical claim: the evaluation protocol in Section 4.2 restricts ranking to a small candidate set with the positive guaranteed and random negatives, and this can interact with SSE-PT's user embeddings to inflate apparent gains. The paper provides a partial robustness check in Table 11, which is evidence in its favor but is not a substitute for full-catalog evaluation or repeated-run statistics. I would not escalate to REJECT, because the method is well-motivated, code is released, and consistent gains appear in multiple tables; I would also not lower the conditionality, because the key comparison has not yet been demonstrated under the evaluation the title implies (ranking over all items). The secondary discrepancy with ML-10M (Table 7 gains around 1.6-2.0%, and Hit Ratio rather than Recall) reinforces the concern that the abstract's 'almost 5% on 5 datasets' is an overstatement, but the sampled-negative protocol is the more consequential issue. Therefore the verdict stays CONDITIONAL/UNCHANGED, with the condition being a full-catalog confirmation.","tokens_in":14963,"tokens_out":9582,"duration_ms":97471,"concrete_test":"Using the released code and the Section 4.4 hyperparameters, run a full-catalog evaluation on MovieLens-1M: for every test user, score all m=3,416 items with the trained SSE-PT and SASRec models, then compute NDCG@10 and Recall@10 without sampling, using the same test split. Repeat with at least 5 random seeds (or a paired bootstrap over users) to get confidence intervals. If SSE-PT's NDCG@10 gain over SASRec under full ranking is still in the 2-4% range, the sampled-negative concern is largely resolved; if the gain shrinks to near zero or reverses, the central claim is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is measured under the candidate-sampling protocol of Section 4.2: for each test user, C=100 or 500 negatives are sampled and the known positive is forced into the candidate set, and NDCG@10/Recall@10 are computed by ranking only those C+1 items. This does not rank the full item pool, so it can overstate or distort model differences. The mechanism is concrete: SSE-PT adds a user embedding to the prediction score in Eq. (8), giving a user-specific bias that can push down uniformly random negatives (most of which are irrelevant to that user) even when it does not improve ordering among realistic candidates. SASRec lacks this bias, so the 4-5% gain may be partly an artifact of the restricted candidate set. The paper's own robustness check (Table 11) varies C on ML-1M and shows the personalized variant still leads, but it compares against an unpersonalized ablation rather than SASRec, stops at C=2000 rather than the full m=3,416, and reports no variance. No error bars or seeds appear anywhere in the experiments, so the stability of the claimed gains is unknown.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SSE-PT, a self-attentive sequential recommendation model that augments the SASRec Transformer architecture with per-user embedding vectors at both the input and prediction layers, regularized by Stochastic Shared Embeddings (SSE-SE). The authors report consistent improvements over SASRec and other baselines on Beauty, Games, Steam, ML-1M, and ML-10M, claim roughly 5% NDCG@10 gains, and introduce SSE-PT++, a variant that samples subsequences to handle long user histories. The paper also presents attention-map visualizations and an ablation study that isolates the contribution of personalization.","tokens_in":15191,"tokens_out":4442,"duration_ms":43059,"significance":"If the reported gains hold under full-catalog evaluation, this is a practically valuable contribution: it shows that a carefully regularized user embedding can be added to a Transformer-based sequential recommender without hurting training speed, and the open-sourced code and data support reproducibility. The personalization ablation in Table 11, the speed comparison in Figure 4, and the attention visualizations are useful concrete evidence for the proposed mechanism. The main limitation is that the headline quantitative claim rests entirely on a sampled-negative evaluation protocol, and the manuscript does not report variance or full-catalog numbers, so the magnitude of the improvement over SASRec is not yet firmly established.","major_comments":[{"comment":"The abstract's claim of \"outperforming SASRec by almost 5% in terms of NDCG@10 on 5 real-world datasets\" is not supported by the reported tables. Table 2 contains only four datasets (Beauty, Games, Steam, ML-1M); ML-10M appears separately in Table 7, where the best NDCG@10 gain over SASRec is around 1.6% (0.7529 vs 0.7413). The claim should be restricted to the actual datasets and percentages, or the fifth dataset should be included in the main comparison table.","section":"Abstract; Section 4.4; Tables 2 and 7"},{"comment":"All NDCG@10 and Recall@10 numbers are computed over a candidate set that consists of one known positive item plus C randomly sampled negatives, rather than over the full item pool. This is a load-bearing issue for the central comparison because the prediction score in Eq. (8) contains a user embedding that can act as a user-specific bias, lowering scores for uniformly sampled irrelevant items in a way that an unpersonalized model such as SASRec cannot. Table 11 extends C to 2000 on ML-1M but stops short of the full catalog of 3,416 items and compares only the personalized model against its unpersonalized ablation, not against SASRec. To substantiate the ranking claim, the authors should report full-catalog metrics (or at least C equal to the item count) and provide means and standard deviations over multiple seeds.","section":"Section 4.2; Section 4.4; Table 11"},{"comment":"The reported SASRec baseline numbers are internally inconsistent across tables for what appear to be the same ML-1M configuration: Table 2 reports NDCG@10 of 0.5936, Table 3 reports 0.5936 and 0.5919, Table 4 reports 0.5919 and 0.5769, and Table 5 reports 0.5961, 0.5941, and 0.5996. In addition, the text states that \"SSE-PT++ achieves even higher 0.9292,\" which conflicts with the SSE-PT++ NDCG@10 value of 0.6292 in Table 5. These inconsistencies make the headline comparison difficult to audit and should be corrected.","section":"Tables 2, 3, 4, and 5; Section 4.4"}],"minor_comments":[{"comment":"The table header says \"Movielens1m\" but the dataset is Movielens10M; this should be corrected.","section":"Table 7"},{"comment":"The sentence \"r_itl is the predicted score of item l by user l at time point t\" should refer to user i, not user l.","section":"Section 3.2.5, Eq. (8)"},{"comment":"The text says the resulting S is in R^{n x d}; this should be consistent with the sequence length T, typically R^{T x d} for one sequence or a batched form.","section":"Section 3.2.3, Eq. (5)"},{"comment":"The first row of the \"USER-SIDE SSE-SE PROBABILITY\" column is labeled \"PARAMETERSHARING,\" which is not a probability value; this makes the table hard to read.","section":"Table 8"},{"comment":"The interpretability claim is based on a single user example and unquantified attention heat maps; this should be described as anecdotal or supplemented with a quantitative measure of attention concentration on recent items.","section":"Section 4.5 and Figures 2, 3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and practical problem, and the personalization mechanism is plausible with a genuine ablation control. However, the central performance claim depends on a sampled-negative evaluation protocol, and the absence of full-catalog results and variance estimates is a substantive gap. The internal inconsistencies in the SASRec baseline numbers and the overstatement in the abstract should also be fixed. I would not reject the paper, but these issues need to be addressed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid, honest incremental paper. The new ingredient is applying SSE-SE regularization to user embeddings in a SASRec-style transformer, which is exactly what lets them add personalization without the overfitting that Kang & McAuley reported. The empirical story is consistent across the datasets they show, and the ablations are the right kind of evidence: Table 6 shows SSE-SE is doing real work, and Table 11 shows the personalized model beats its unpersonalized control under several candidate-set sizes. That's a legitimate new result, not just a rehash.\n\nThe main soft spot is the evaluation protocol. They sample C=100 or 500 negatives and force the positive item in, then compute NDCG@10 and Recall@10 over that small set. That doesn't rank the full pool. Table 11 does vary C on ML-1M up to 2000, which is a good-faith robustness check, but it's still not the full 3,416 items and it compares against their own unpersonalized variant, not against SASRec. So the headline \"almost 5% over SASRec\" could shrink under full-catalog evaluation, though I doubt it would vanish given how consistent the pattern is. What bothers me more: there are no error bars or multiple seeds anywhere, so we have no idea if the 0.4–4.8% gains are stable or noise. The abstract also oversells; \"almost 5%\" is the best case, not the typical gain.\n\nBert4Rec is missing from the baselines. It was published around the same time and is also a masked-transformer sequential recommender; leaving it out makes the \"state-of-the-art\" claim weaker.\n\nThe attention-map interpretability section is qualitative but pleasant, and the SSE-PT++ long-sequence variant is a useful practical tweak. The code is open source, which earns credit. I'd like to see the authors add error bars, a full-catalog evaluation on at least one dataset, and ideally the Bert4Rec comparison before I'd fully trust the headline numbers.\n\nWho is this for? Anyone working on sequential recommendation or on regularization for embedding layers. It's not paradigm-shifting, but it's a useful, reproducible result that deserves a serious referee rather than a desk rejection. I'd send it to review with a request for the missing variance reporting and a stronger evaluation on at least one dataset.","headline":"A solid, incremental extension of SASRec that makes personalization work via SSE-SE regularization; the empirical claim is plausible but the sampled-negative evaluation protocol leaves the headline 4-5% gain less certain than the abstract suggests.","tokens_in":15776,"tokens_out":1723,"would_cite":true,"duration_ms":18845,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SSE-PT, a personalized Transformer that concatenates user embeddings with item embeddings and regularizes them via stochastic shared embeddings, outperforms the unpersonalized SASRec by almost 5% in NDCG@10 on five real-world datasets.","keywords":["temporal collaborative ranking","self-attention","personalization","stochastic shared embeddings","sequential recommendation","transformers","implicit feedback"],"falsifier":"Run SSE-PT and SASRec on the same five datasets but compute NDCG@10 and Recall@10 by scoring every item in the catalog (no negative sampling), using the same hyperparameters; if the gain over SASRec shrinks to near zero or reverses, the central claim is an artifact of the sampled-candidate evaluation.","tokens_in":14722,"feed_emoji":"🎯","tokens_out":7973,"duration_ms":72389,"temperature":0.7,"pith_summary":"The paper's central claim is that the self-attentive sequence model SASRec underperforms in recommendation because it has no per-user parameters, and that adding user embeddings, regularized by Stochastic Shared Embeddings (SSE-SE), yields a personalized Transformer, SSE-PT, that outperforms SASRec by almost 5% in NDCG@10 across five real-world datasets. The authors argue that standard regularizers such as dropout, weight decay, and layer normalization are insufficient to train these extra parameters, and that SSE-SE's stochastic embedding replacement is what makes personalization feasible. A long-sequence variant, SSE-PT++, samples either a random window or the most recent T items so that it matches or exceeds SASRec while allowing shorter sequence length and faster training. Attention-map inspection suggests the model concentrates on recent engagement per user, which the paper links to its better interpretability.","feed_headline":"Personalized Transformer beats SASRec by almost 5% in ranking","feed_subtitle":"Per-user embeddings, regularized by stochastic sharing, improve next-item ranking on five datasets.","key_machinery":"The central object is the Personalized Transformer (SSE-PT): a modified Transformer encoder in which each position t of the input sequence is the concatenation [v_{j_t}; u_i] plus a positional encoding p_t, so that the self-attention blocks are conditioned on who the user is. The prediction layer scores every candidate item l by the dot product of the last output representation with [v_l; u_i]. The load-bearing regularizer is SSE-SE (Stochastic Shared Embeddings, 'shared' because an embedding is stochastically replaced by another embedding during SGD with probability p), applied to user embeddings, input item embeddings, and output item embeddings. The long-sequence variant SSE-PT++ modifies how sequences are fed in: with probability p_s it samples a starting index uniformly from the available history and takes a window of length T; otherwise it uses the last T items, so that the model can use much longer histories while T stays small.","core_discovery":"In the paper's own terms, the discovery is that personalization, implemented as user embeddings concatenated with item embeddings at the input and reused in the prediction layer, substantially improves temporal collaborative ranking, and this only becomes possible when a stochastic-shared-embedding regularizer is applied to the embedding layers. The authors report that SSE-PT beats SASRec on Beauty, Games, Steam, Movielens1M, and Movielens10M, with double-digit relative improvements in NDCG@10 on most datasets, and that ablations show removing SSE-SE collapses the gain and can drop below the unpersonalized baseline. They further show that attention heat maps place more mass on recent items in long sequences than SASRec's, and that the SSE-PT++ variant, which truncates long histories by either sampling a window or taking the most recent T items, preserves most of the benefit at roughly the same training speed.","pith_inferences":["A natural next test is whether SSE-SE personalization transfers to other sequence recommenders (e.g., RNN- or CNN-based models), since the paper establishes the principle on a Transformer but the regularizer is architecture-agnostic.","A quantitative recency metric, such as the average attention weight on the last 10% of positions versus a uniform baseline, could formalize the qualitative attention-map claim and make it testable across users.","Because the reported evaluation samples negatives, the ~5% advantage is only guaranteed under that protocol; full-catalog NDCG@10 evaluations would show whether the gain survives a harder ranking task.","The user-in-the-input move suggests that a Transformer could also treat the user id as a special token, or that session embeddings could be learned for anonymous visits, which the paper does not explore."],"forward_implications":["On all five datasets, the personalized model beats the unpersonalized one, and the margin persists when the number of sampled negatives is varied from 100 to 2000.","Removing SSE-SE regularization drops the personalized model below the unpersonalized baseline, so the regularizer, not just the extra parameters, is causally responsible for the gain.","When the history window is shortened from 200 to 100, SSE-PT++ keeps most of its performance while SASRec degrades more, so the recency-based sampling strategy is more robust to truncation.","The attention maps associate the model's better ranking with larger attention mass on recent items, giving an interpretable explanation rather than only a metric improvement."],"supporting_citations":[{"why":"SASRec, the unpersonalized self-attention baseline that SSE-PT extends; the paper reuses its architecture, datasets, and train/dev/test splits.","marker":"[16]"},{"why":"Stochastic Shared Embeddings (SSE-SE), the regularization of embedding layers that the paper identifies as essential for training user embeddings without overfitting.","marker":"[41]"},{"why":"The Transformer self-attention architecture that the model adapts, with masked attention blocks.","marker":"[36]"}],"fun_headline_variants":["Personalized Transformer tops SASRec by 5% in temporal ranking","User-aware transformer wins ranking race with 5% boost over SASRec","SSE-PT: Personalized transformer improves ranking by 5% over SASRec","Temporal ranking gets personal: SSE-PT beats SASRec by 5%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains are measured under a negative-sampling evaluation where each user's next item is ranked against C sampled negatives rather than the full catalog, so the numbers assume that success on this sampled ranking transfers to true top-K ranking over all items.","fun_headline_variants_meta":{"raw":{"variants":["Personalized Transformer tops SASRec by 5% in temporal ranking","User-aware transformer wins ranking race with 5% boost over SASRec","SSE-PT: Personalized transformer improves ranking by 5% over SASRec","Temporal ranking gets personal: SSE-PT beats SASRec by 5%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000624,"raw_usage":{"total_tokens":2922,"prompt_tokens":1011,"completion_tokens":1911,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":1829}},"tokens_in":627,"tokens_out":1911,"duration_ms":13078,"temperature":1.0,"reasoning_tokens":1829,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:14:06.416312+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SSE-PT and SASRec on the same five datasets but compute NDCG@10 and Recall@10 by scoring every item in the catalog (no negative sampling), using the same hyperparameters; if the gain over SASRec shrinks to near zero or reverses, the central claim is an artifact of the sampled-candidate evaluation.","supporting_citations":[{"cited_title":"Stochastic Shared Embeddings: Data-driven Regularization of Embedding Layers","cited_arxiv_id":"1905.10630","evidence_quote":"Stochastic Shared Embeddings (SSE-SE), the regularization of embedding layers that the paper identifies as essential for training user embeddings without overfitting."}],"review_version":1}