{"id":"20cc6ca7-5f11-4f06-9d1b-82f556dc9afe","arxiv_id":"2508.20945","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"For cross-domain sequential recommendation, restricting transformer attention to intra-domain items plus TAPE and DDSR cross-domain state mechanisms improves retrieval on Amazon 5-domain data over HSTU baselines.","lead":"A recommender system can learn each shopping domain separately while two new mechanisms, TAPE and DDSR, pass hints between domains. The Amazon 5-domain results beat the HSTU baseline on recall, but the efficiency claim is not measured and the method skips comparisons with existing cross-domain systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8) lacks an explicit causal mask; if DDSR attends to future K/V, reported HR/NDCG gains may be leakage artifacts.","rationale":"The reader's weakest assumption identifies exactly the load-bearing gap. The paper's strongest claim has two parts: reduced cost and preserved performance. The efficiency part is already qualified ('not yet optimized', 'theoretical analysis indicates'), but the performance part is presented as experimental evidence. Eq. (8) is the only place where cross-domain information flows into the next-item representation, and its mask is unspecified. If the softmax over K/V includes positions j>i, then at training and evaluation time the model sees the target item's hidden state through V, which would trivially inflate all retrieval metrics. This would explain the observed gains and invalidate the 'preserving strong overall performance' claim. The manuscript itself flags missing masking details in the footnote ('If accepted, we will provide more details on the Flex Attention block masking used in this work and also how to generate this domain state representation'), which is an explicit gap in support. A second-order concern is that DDSR as written has complexity O(D·n^2) (D domain-state queries attending to all n keys/values), which is not accounted for in Section 3.5; even if leakage is absent, the asymptotic efficiency claim is incomplete. However, the mask question is more directly fatal: it determines whether the central experimental numbers mean anything. The proposed test—checking the actual mask or retraining with an explicit causal mask—would settle it. No ad hominem is intended; the issue is a missing specificity in the formulation, not a claim about author intent. Given this unverified load-bearing assumption plus the paper's own preliminary label, the reader's REJECT verdict stands unchanged.","tokens_in":8217,"tokens_out":7211,"duration_ms":74105,"concrete_test":"Instrument or reimplement the DDSR attention in Eq. (8) and check the attention mask. Concretely: (a) if code is available, assert that every attention weight for j>i is exactly zero in all heads and layers of a trained checkpoint; (b) if code is not available, retrain 'Ours' with an explicit causal mask (set m_{i,j} = -inf for j>i before softmax) and compare HR@100 and NDCG@100. If the masked model falls back to roughly the intra-domain masking baseline (HR@100 ≈ 9.09, NDCG@100 ≈ 3.15), leakage is confirmed and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that intra-domain attention plus TAPE/DDSR reduces cost while preserving performance. The performance half rests on DDSR Eq. (8), which is written as Softmax(Q·K^T/sqrt(k))·V with no masking. Because C_L is added to H_L at every position, a future token's V vector at j>i can enter the representation used to predict the next item at position i, directly leaking the target. The paper states in Section 4 that a causal mask is applied 'in the self-attention layers', and Section 3.4 calls DDSR 'an extra self-attention layer', but it never states that the mask applies to Eq. (8); the footnote even defers FlexAttention masking details to an accepted version. As written, the equation permits leakage, and the reported HR@100 improvement from 9.09 to 10.25 could be an artifact of seeing future tokens. This is the least secure assumption connecting the experiments to the claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to scale multi-domain sequential recommendation with transformer backbones by replacing full inter-domain attention with (i) Transition-Aware Positional Embeddings (TAPE), which encode domain-transition information, and (ii) a Dynamic Domain State Representation (DDSR) module, which summarizes each domain's recent hidden states and injects cross-domain context. The authors claim that this reduces computational cost relative to full attention while preserving or improving next-item prediction performance, and they report experiments on Amazon multi-domain data with HSTU as the base model.","tokens_in":8434,"tokens_out":5538,"duration_ms":58913,"significance":"If the claims were fully supported, the paper would make a useful contribution: it identifies a real bottleneck in applying transformer recommenders to multi-domain sequences and proposes a modular alternative that keeps intra-domain attention parallel and cheap while transferring cross-domain information through a compact state. The ablation design is sensible and the use of FlexAttention for intra-domain masking is appropriate. However, as submitted, the central efficiency claim is supported only by an incomplete complexity expression, and the central performance claim is threatened by a missing causal-mask specification in the DDSR module. These are load-bearing issues that need to be resolved before the paper's conclusions can be accepted.","major_comments":[{"comment":"The DDSR module as written does not specify a causal mask. Eq. (8) computes Softmax(Q_L K_L^T / sqrt(k)) V_L with Q_L derived from the domain state at position i but K_L, V_L derived from H^{L-1} across all positions. If no causal mask is applied, C_L at position i can include information from positions j > i, i.e., future tokens. Since C_L is added to H_L at every position, this would directly leak the target item into the representation used for next-item prediction. The text only states that a causal mask is applied 'in the self-attention layers' (Section 4), and Section 3.4 merely says DDSR 'follows a similar pattern' without specifying masking. Footnote 1 defers the masking details. This is not a cosmetic issue: the reported HR@100 gain from 9.09 to 10.25 could be an artifact of future-token leakage. Please specify the exact masking used in Eq. (8) and, if it is not causal, rerun th","section":"Sec. 3.5"},{"comment":"The complexity analysis considers only the intra-domain self-attention cost, O(sum_d s_d^2), and compares it with full attention O(S^2). However, the DDSR module in Eq. (8) adds a separate attention computation: for each of the |D| domain-state queries, the module attends to all n positions in K_L, V_L. As written, this adds O(|D| n^2) per layer, which is a factor of |D| larger than full attention O(n^2) and dominates the claimed intra-domain saving when |D| > 1. If the DDSR attention is intended to be masked (e.g., restricted to intra-domain positions or to a subset of keys), that restriction must be stated explicitly and included in the complexity expression. As written, the theoretical case for 'reduced computational cost' is incomplete and could be reversed.","section":"Sec. 3.5"},{"comment":"The paper's central claim is that the method 'reduces computational cost whilst preserving strong overall performance'. Yet Section 5 states that the current implementation is 'not yet optimized' and that runtime benefits are only predicted theoretically, not measured. No wall-clock time, FLOP count, or memory measurement is reported. Given that the complexity claim is also incomplete (see above), the efficiency half of the central claim is not supported by evidence. Please provide at least one runtime or FLOP comparison against full-attention HSTU on the same hardware.","section":"Sec. 5"},{"comment":"The evaluation is limited to ablations of the proposed model against HSTU variants. There is no comparison with existing cross-domain sequential recommendation methods such as MAN, C2DSR, or DREAM, which are cited in the related work. The paper frames itself as a cross-domain recommendation contribution, so the absence of any SOTA comparison makes the performance claim 'strong overall performance' difficult to assess. Adding at least one or two representative cross-domain baselines would substantially strengthen the paper.","section":"Sec. 4.3"}],"minor_comments":[{"comment":"The acronym DDSR is introduced in the abstract, but the Introduction refers to 'DDRS'. Please standardize the spelling.","section":"Sec. 1"},{"comment":"The sentence 'only including DDSR as above significantly drops our model's performance' appears to be a wording error: the subsection describes the model without DDSR (i.e., with only TAPE). Please clarify.","section":"Sec. 4.2.1"},{"comment":"The text says that the full model 'demonstrates consistent improvements across all evaluation metrics', but Table 1 shows that compared with the intra-domain masking baseline, HR@1 drops from 0.99 to 0.80 and MRR drops from 1.88 to 1.77. The discussion later acknowledges these drops. Please correct the earlier statement to avoid contradiction.","section":"Sec. 4.2"},{"comment":"Important details—the FlexAttention masking and the domain-state generation, and the precise form of the complexity bound—are deferred to an 'accepted version'. Since this is a submitted manuscript, these details should be in the current text.","section":"Footnotes"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly early work: Section 5 calls the results 'preliminary', and Section 4.3 explicitly limits evaluation to baseline comparisons. The missing causal mask in Eq. (8) is the most serious issue because it undermines the validity of the reported performance gains. I do not recommend outright rejection because the issue is testable and fixable, and the core idea has merit. However, the authors must (a) specify and/or enforce a causal mask for DDSR and confirm that results do not change materially, (b) extend the complexity analysis to include DDSR, and (c) provide runtime or FLOP evidence for the efficiency claim. Without these, the central claims remain unverified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper has a genuinely interesting idea and an evidential gap big enough to matter. The combination of intra-domain-only attention with transition-aware positional embeddings (TAPE) and a per-domain dynamic state (DDSR) is new relative to the cited literature, and the high-level motivation is sound: cross-domain transfer need not require full quadratic attention over all tokens. The complexity analysis is straightforward and correct as far as it goes, and the experimental setup (leave-one-out on five Amazon domains, HSTU baseline) is standard for the area.\n\nThe soft spots are serious, though. The stress-test note is on target: Eq. (8) computes DDSR attention over all sequence positions without an explicit causal mask. Q at position i uses only history, but K and V are not masked, so future hidden states can enter the representation used for next-item prediction. The reported HR@100 gain from 9.09 to 10.25 could be partly or entirely a leakage artifact. The text says the module is \"an extra self-attention layer\" and says causal masking is applied in the self-attention layers, but it never states that the mask covers Eq. (8), and the footnote defers details to an accepted version. That ambiguity is load-bearing because the paper's central claim is that TAPE+DDSR preserves or improves performance while cutting cost.\n\nThe efficiency half of the claim is also unsubstantiated: no runtime measurements anywhere, only a complexity expression. The paper itself admits the implementation is not optimized and calls the results preliminary. Ablations are single-run with no error bars, and no state-of-the-art CDSR methods (pi-Net, MAN, C2DSR, DREAM) are compared, which the authors acknowledge in a footnote. The relation to pi-Net's cross-domain state is acknowledged in related work, but DDSR's specific form seems distinct enough.\n\nOn the credit side: the paper is clearly written, the problem is important, the intra-domain masking baseline itself improves over full HSTU attention in the reported results, and the functional split between TAPE (better ranking) and DDSR (better recall) is a plausible, interesting finding—if the leakage is fixed. The citation pattern looks fine; no self-citation inflation or missing obvious prior work.\n\nWho is this for? A researcher working on industrial multi-domain sequential recommenders, particularly people thinking about scaling transformers beyond a single domain. A serious reviewer could usefully ask the authors to add a causal mask to Eq. (8), rerun, add runtime numbers, and compare against one or two non-transformer CDSR baselines. This is a promising workshop-tier or short-paper submission, not yet a solid conference paper. I would send it to peer review rather than desk-reject, because the idea is fresh and the fix is presumably simple, but I would not cite it in its current form.","headline":"TAPE+DDSR is a fresh, sensible approach to cross-domain sequential recommendation, but the reported gains are not yet trustworthy because Eq. (8) as written can leak future tokens and the efficiency claim has no runtime evidence.","tokens_in":8903,"tokens_out":2448,"would_cite":false,"duration_ms":28802,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A transformer recommender can attend only within each domain and still beat full attention, provided transition cues and domain states carry the cross-domain signal.","keywords":["multi-domain recommendation","sequential recommendation","autoregressive models","transformers","retrieval","cross-domain knowledge transfer","positional embeddings","efficient attention"],"falsifier":"Instrument the softmax in Eq. (8) at prediction time: for each position i and each domain row, record the largest key index j that receives non-negligible attention weight; if any j>i appears, future items leak into earlier predictions, and the method's gains over the masked-only baseline would not establish cross-domain transfer. Conversely, if j≤i always, the causal assumption is verified.","tokens_in":8160,"feed_emoji":"⚡","tokens_out":9643,"duration_ms":87768,"temperature":0.7,"pith_summary":"The paper tries to show that a transformer-based recommender can handle several domains without paying the full cross-domain attention cost. It restricts self-attention to items within the same domain, then uses two lightweight modules to recover the information that full attention would have carried across domains: Transition-Aware Positional Embeddings (TAPE) mark where and how the domain shifts, and Dynamic Domain State Representations (DDSR) keep the latest hidden state of each domain and let every position query it with a small cross-attention step. The authors report that on a five-domain benchmark this combination raises HR@100 from 9.09 to 10.25 and NDCG@100 from 3.15 to 3.26 over the masked-only setup, with attention complexity dropping from quadratic in total sequence length to quadratic in each domain's length only. If true, this gives a scalable path for industrial cross-domain recommendation, although the paper itself labels the results preliminary and reports the runtime benefit as theoretical rather than measured.","feed_headline":"Domain-only attention lifts recall, cuts cost","feed_subtitle":"On a five-domain benchmark, domain-masked attention lifts HR@100 from 9.09 to 10.25 at lower theoretical cost.","key_machinery":"Dynamic Domain State Representation (DDSR): a matrix that stores, for each domain and each sequence position, the last hidden state belonging to that domain; a cross-attention layer treats these stored states as queries over the current sequence, giving each position access to compressed cross-domain context without full attention. Transition-Aware Positional Embeddings (TAPE): additive positional terms that are nonzero exactly at domain boundaries, signalling both that a transition occurs and what the transition is. Together they replace the inter-domain attention map: TAPE prepares the input, DDSR propagates the state, and the main self-attention layers compute only intra-domain scores.","core_discovery":"On the paper's own terms, the core discovery is that intra-domain attention alone is not enough for multi-domain sequential recommendation, but intra-domain attention plus two cheap explicit cross-domain signals is enough to outperform full attention. TAPE injects a transition-aware term into the item embeddings exactly when the next item belongs to another domain, so the model knows a shift is happening. DDSR maintains a per-domain state matrix holding the most recent hidden state for each domain up to each position, and a small cross-attention layer uses those states as queries against the current keys and values; its output is added to the main attention output. With both mechanisms, retr","pith_inferences":["The paper defers the exact FlexAttention masking and domain-state generation details to a later version (footnote 1); the causal behaviour of DDSR therefore cannot be verified from the text, and the falsifier above is the appropriate check.","If the causal assumption is confirmed, the same pattern—masks over known segment boundaries plus a compressed state read by cross-attention—could transfer to other sequence tasks with explicit segment structure, such as multi-document or multi-session language modelling.","The theoretical factor-of-|D| saving assumes each domain's subsequence is processed with an efficient sparse-attention implementation; wall-clock speedup depends on kernel-level support and remains untested here.","A direct extension would vary the number of domains beyond five: the complexity argument predicts the benefit of intra-domain masking grows with |D|, so the recall gap over full attention should widen if the mechanism is doing the claimed work."],"forward_implications":["Attention cost for a multi-domain session drops from O(S^2) to O(sum_d s_d^2); with balanced domains that is roughly a |D|-fold reduction, making more domains computationally cheaper per item rather than more expensive.","Cross-domain knowledge transfer can be carried by a compact per-domain state and transition embeddings instead of pairwise attention between every item in every domain, so the approach scales to settings with many domains or long histories.","The reported trade-off—lower HR@1 and MRR but higher HR@100 and NDCG@100—positions the method as a candidate-pool generator for downstream ranking stages rather than as a final ranker.","TAPE and DDSR are complementary: ablations show TAPE mainly protects ranking sharpness while DDSR mainly expands recall, so both are needed for the combined gain."],"supporting_citations":[{"why":"Supplies the autoregressive transformer recommender baseline, its next-item training objective, and the gating formulation the method extends.","marker":"[Zhai et al. 2024]"},{"why":"Establishes the causal next-item prediction setup and the SASRec-style evaluation paradigm.","marker":"[Kang and McAuley 2018]"},{"why":"Provides the ALiBi distance bias used in the self-attention layer of the baseline and proposed model.","marker":"[Press et al. 2021]"},{"why":"Provides the FlexAttention operation used to mask attention to intra-domain tokens only.","marker":"[Li et al. 2024]"},{"why":"Supplies the multi-domain dataset the experiments are run on.","marker":"[McAuley et al. 2015]"},{"why":"Provides the SampledSoftmax negative-sampling objective used for training and evaluation.","marker":"[Covington et al. 2016]"},{"why":"Introduces the cross-domain state-update idea that DDSR adapts for parallel transformer processing.","marker":"[Ma et al. 2019]"},{"why":"Presents the parallel split-join state-transfer approach that motivates treating domain shifts with a separate state mechanism.","marker":"[Sun et al. 2021]"}],"fun_headline_variants":["Cheap cross-domain signals beat full attention","Two tricks replace attention in multi-domain recsys","Dynamic states slash attention cost, lift recall","Domain shifts encoded, not attended to","Recall up, compute down with TAPE and DDSR"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The central result assumes the domain-state cross-attention module in Eq. (8) never lets an earlier sequence position attend to a future hidden state; the paper specifies a causal mask for the main self-attention layers but not for this module, so without that assumption the reported gains could come from target leakage.","fun_headline_variants_meta":{"raw":{"variants":["Cheap cross-domain signals beat full attention","Two tricks replace attention in multi-domain recsys","Dynamic states slash attention cost, lift recall","Domain shifts encoded, not attended to","Recall up, compute down with TAPE and DDSR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000298,"raw_usage":{"total_tokens":1565,"prompt_tokens":749,"completion_tokens":816,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":760}},"tokens_in":493,"tokens_out":816,"duration_ms":7438,"temperature":1.0,"reasoning_tokens":760,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T14:41:11.060512+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Instrument the softmax in Eq. (8) at prediction time: for each position i and each domain row, record the largest key index j that receives non-negligible attention weight; if any j>i appears, future items leak into earlier predictions, and the method's gains over the masked-only baseline would not establish cross-domain transfer. Conversely, if j≤i always, the causal assumption is verified.","supporting_citations":[],"review_version":1}