{"id":"8d0b5536-ad92-4d06-827b-0d7efba42c02","arxiv_id":"2506.05625","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A sequel-aware graph neural network that explicitly models item-series relationships improves next-item recommendation on datasets where sequels are common, but it can underperform a strong baseline when sequel information is sparse.","lead":"This paper introduces a graph neural network that adds explicit links between items that are part of a series or franchise, such as movie sequels, and uses those links alongside user-item interactions for next-item recommendations. The authors report higher accuracy than several existing recommender systems on datasets with many series items, but the gains shrink or reverse when only a few items have sequels.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sequel-aware message passing may leak future information: §4.3.2 aggregates 'subsequent sequel items' and §4.1 builds sequel edges without timestamp constraints, so improvements over DGSR may reflect access to future/target items rather than temporal sequence modeling.","rationale":"The paper's central claim is empirical: sequel-aware GNNs are better or comparable to graph-based baselines that ignore sequel information. The reader correctly flags sequel-edge quality and the ML-100K contradiction. My read identifies a more specific and more damaging issue: the construction in §4.1 and §4.3.2 lets sequel edges carry information across time without temporal gating. In leave-one-out evaluation, the target item's embedding can be influenced by interactions that occur after the user's test timestamp via other users, and the explicit edge from a predecessor in the user's history to the successor target gives the model a direct handle on the answer. The fact that improvements are largest where sequel edges are most complete (Goodreads series metadata) and negative where they are inferred/noisy (ML-100K) is consistent with this leakage story. This does not require assuming any bad faith; it follows from the graph construction and message-passing equations as written. A controlled re-run with time-restricted or predecessor-only sequel propagation would settle it. Because the central claim is not established as reported, I keep the reader's reject-level concern; my only difference is the locus of the problem.","tokens_in":79,"tokens_out":7072,"duration_ms":120190,"concrete_test":"Re-run the Table 3 experiments with a leakage-controlled variant of §4.3.2: (a) restrict Ns(i) to immediate predecessors in the series, or (b) include a sequel edge (i,j) only if the earliest timestamp of any interaction with j is ≤ t_k of the user being scored, using the same splits and seeds as the paper. If the Goodreads NDCG@10 gain over DGSR (6.46%) and the ML-1m gains shrink or change sign, the reported benefit is leakage-driven. Additionally, report the fraction of test ground truths that are direct successors, via sequel edges, of items in the user's training history; if that fraction is high, the explicit edge nearly identifies the answer and the comparison to DGSR is not a fair test of sequence modeling.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The single most load-bearing concern is temporal leakage in the sequel-edge construction. Section 4.3.2 defines hseq_i = (1/|Ns(i)|) Σ_{j∈Ns(i)} h_j P(j), where Ns(i) is the set of 'subsequent items in the sequel series,' and §4.1 creates sequel edges over the full series (e.g., (i2,i5,A,2), (i5,i7,A,3)) with no timestamp restriction. In the leave-one-out protocol, the held-out next item is often a direct successor of an item already in the user's history. Because message passing is run before any temporal split, the candidate item's embedding and the user's representation can both contain information from items interacted with after t_k through other users, including the successor that is the target. Thus the observed advantage of HSAL-GNN over DGSR may be largely an artifact of unconstrained sequel edges propagating future/target information, not of a temporal 'sequel-aware' representation. The pattern in Table 3—large Goodreads gains, small ML-1m gains, and losses on ML-100K, where inferred sequel sets are smallest and noisiest—is exactly what this leakage would look like. The central claim in the abstract is therefore not supported by the experiments as reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HSAL-GNN, a heterogeneous graph neural network for sequential recommendation that augments the user-item bipartite graph with item-item edges representing sequel relationships. The architecture combines long-term, short-term, and sequel-aware item embeddings through several fusion strategies and uses sinusoidal positional encodings. The authors evaluate on three synthetic and three real-world datasets against GRU4Rec, SASRec, BERT4Rec, SVAE, ACVAE, DiffuRec, STOSA, and DGSR, using a leave-one-out protocol. The central claim is that explicitly modeling sequel relations improves or matches state-of-the-art graph-based sequential recommenders, especially when sequel information is abundant. The paper also includes an ablation study and a sensitivity analysis of hyperparameters.","tokens_in":14390,"tokens_out":12163,"duration_ms":107037,"significance":"The problem of exploiting item-series structure is timely and under-explored, and the idea of encoding such structure in a heterogeneous GNN is reasonable. The paper offers a broad empirical comparison and a detailed ablation, and it makes code available. If the claimed improvements were established with a sound evaluation protocol, the contribution would be useful to the recommender-systems community. However, the current evaluation suffers from a serious temporal-leakage issue that casts doubt on the reported gains, several reported results contradict the text, and internal inconsistencies appear in the ablation table. As a result, the significance of the contribution is not established by the experiments as reported.","major_comments":[{"comment":"The sequel-edge construction in §4.1 and the subgraph sampling in Algorithm 1 create item-item edges without any timestamp restriction. Section 4.3.2 defines hseq_i by aggregating over 'subsequent items in the sequel series.' In the leave-one-out protocol (§5.4), the held-out next item is frequently a direct successor of an item already in the user's history; because the graph contains all user interactions and sequel edges are formed over the full series, the candidate item's embedding can incorporate information from interactions that occur after the prediction time, including the target item itself. This gives HSAL-GNN access to future information that DGSR and other baselines do not have, so the reported gains in Tables 2 and 3 do not measure the value of sequel-aware representation. The authors should restrict sequel edges to pairs whose interaction timestamps are both before the prediction time, or re-run the evaluation under a strict temporal split (e.g., training only on edges with t < t_k for each user). Without such a correction, the central claim in the abstract is not supported.","section":"§4.1, §4.2, §4.3.2"},{"comment":"Section 5.5 states that ML-100K 'sees little to no improvement' and that the Non-Sequential synthetic dataset shows 'minimal improvements.' Table 3 shows that on ML-100K HSAL-GNN is worse than DGSR on every metric, with NDCG@10 0.4347 vs 0.4439 and Hit@10 0.7241 vs 0.7288. Table 2 shows that on the Non-Sequential dataset HSAL-GNN is worse than DGSR on all metrics, including NDCG@10 0.0432 vs 0.0476. Also on the Sequential dataset, HSAL-GNN is slightly worse than DGSR on most metrics (e.g., NDCG@10 0.9529 vs 0.9553). These results contradict the text and weaken the abstract's claim of 'better (or comparable)' performance. The report should be corrected and the negative results discussed honestly.","section":"§5.5, Table 3, Table 2"},{"comment":"In Table 4, the 'Sinusoidal' positional-embedding row reports ML-1m NDCG@10 = 0.4347, while the 'Sum' fusion row (which uses the default sinusoidal embedding) reports 0.4795 and the corresponding HSAL-GNN result in Table 3 is 0.4796. The value 0.4347 is identical to the ML-100K NDCG@10 in Table 3, suggesting a copy-paste error. This inconsistency undermines the ablation claim that sinusoidal embeddings outperform rotary embeddings on ML-1m. The ablation table should be verified and corrected.","section":"Table 4"},{"comment":"Section 5.2 states that experiments are run five times and mean performance is reported, but no standard deviations or significance tests are provided. Many of the claimed improvements are small, for example ML-1m NDCG@10 0.4796 vs 0.4764, Goodreads NDCG@10 0.2074 vs 0.1948, and Mixed NDCG@10 0.4539 vs 0.4290. Without variance reporting, these differences may be due to random seed variation. The authors should report mean ± std over the five seeds and perform paired significance tests for the main comparisons.","section":"§5.2"}],"minor_comments":[{"comment":"The density formula is unclear and the values appear inconsistent: for Goodreads, 200k interactions / (2000 users × 6474 items) is about 1.5%, not 0.01%, and for ML-1m, 1.0M / (6040 × 3900) is about 4.2%, not 4.79%. Please clarify the formula and correct the numbers.","section":"§5.1, Table 1"},{"comment":"The text says 'as illustrated in Figure 4' when discussing the effect of the number of layers, but the layer experiments are shown in Figure 3; Figure 4 is the bar-chart comparison. Please correct the cross-reference.","section":"§5.7"},{"comment":"The abbreviation 'LTSM' is a typo and should be 'LSTM'.","section":"§1"},{"comment":"The definition of ia and ib in Eq. (3) is confusing; please clarify that ia is the immediate predecessor and ib the immediate successor of ip in the series, and define the edge-tuple notation consistently.","section":"§4.1"},{"comment":"The 'SeHGNN-based transformation' is described only by name; please provide the exact equation or a precise reference to the specific layer so that the fusion strategy is reproducible.","section":"§4.3.4"},{"comment":"There are footnote markers 3 and 4 with no accompanying footnote text in the submission; please either supply the footnotes or remove the markers.","section":"§3.1"},{"comment":"The maximum sequence length is set to 50, while the average sequence length in Goodreads is 260.9 (Table 1); please explain how longer sequences are truncated and whether this affects the comparison.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"This paper faces substantial problems that may ultimately warrant rejection if they are not fully addressed. The temporal-leakage issue is fundamental: the current evaluation does not test the claimed contribution, and the negative results on several datasets are misreported. I recommend major revision only because the evaluation protocol can in principle be corrected. If the authors cannot provide a clean temporal-split evaluation with error bars and corrected text, the paper should be rejected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The first thing you should know: the paper's central claim—that sequel-aware GNNs are better or comparable to graph baselines—is not supported by its own experiments, and there is a plausible temporal leakage in the sequel-edge construction that would explain the positive results. Still, the core idea is genuinely new and worth a second look.\n\nWhat's new: the authors inject explicit series edges with sinusoidal positional encoding into a heterogeneous GNN. That's a real departure from DGSR's item-item transitions, which don't preserve order. The problem is well-motivated, and the synthetic setup with mixed/sequential/standalone splits is a sensible way to isolate the effect. The ablation of fusion strategies and positional encodings is thorough for an architecture paper.\n\nThe soft spots are serious. In §4.1, sequel edges are built over the full series with no timestamp restriction. In §4.3.2, the model aggregates 'subsequent sequel items' for each item. Under the leave-one-out protocol, the held-out target is often a direct successor of an item already in the user's history. The user representation can therefore contain the target item's own embedding before scoring. That's leakage, not temporal modeling. The pattern in Table 3—large gains on Goodreads, smaller on ML-1m, losses on ML-100K—is exactly what you'd expect if the edge construction is giving the model access to future items. The paper does not discuss or rule this out.\n\nSecond, the ML-100K numbers are misreported. Table 3 shows HSAL-GNN worse than DGSR on every metric (NDCG@10 0.4347 vs 0.4439), but the text calls this 'little to no improvement.' That's a direct contradiction. Also missing: error bars despite five seeds, the code, and details about the Goodreads sampling.\n\nWho is this for: people working on item-series structure in sequential recommendation. The idea has potential, but as reported, the evidence doesn't demonstrate an effect. If the authors rerun with timestamp-constrained sequel edges and show gains survive, the paper could be worth publishing. As is, I'd expect a reviewer to reject on leakage grounds.\n\nRecommendation: I would not accept the current version. But I'd send it to a serious referee rather than desk-reject it, because the construction is novel and the leakage concern is testable. With a careful revision, there's a path to a useful contribution.","headline":"A novel sequel-edge construction, but the evaluation leaks future information and the ML-100K results contradict the abstract.","tokens_in":14973,"tokens_out":4003,"would_cite":false,"duration_ms":39980,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Explicitly modeling item-sequel links in a heterogeneous GNN improves next-item recommendation when enough items belong to series, with gains scaling with sequel density.","keywords":["sequential recommendation","sequel-aware","graph neural networks","heterogeneous graphs","next-item prediction","temporal item sequences","user-item bipartite graphs"],"falsifier":"Run HSAL-GNN and the strongest graph baseline on a dataset with publisher-verified series metadata, holding the user-item graph fixed. If the sequel-aware model does not beat the baseline on series-rich item slices, or if randomly shuffling sequel links does not degrade its scores, then the claimed benefit of sequel information is not real.","tokens_in":13897,"feed_emoji":"🎬","tokens_out":7221,"duration_ms":67660,"temperature":0.7,"pith_summary":"The paper asks whether next-item recommendation improves when the model is explicitly told which items belong to an ordered series, and answers yes under the right conditions. It builds a heterogeneous graph with two edge types \\u2014 user-item interactions and sequel links between items in a series \\u2014 and propagates messages along both. On three synthetic and three real-world datasets, the resulting HSAL-GNN beats or matches a strong graph-based recommender that ignores sequels. The size of the gain tracks the share of items that actually belong to series: substantial on Goodreads, marginal on MovieLens 1m, and absent on MovieLens 100k, where the sequel-aware model trails a graph baseline.","feed_headline":"Sequel-aware GNN lifts next-item recommendation","feed_subtitle":"Gains grow with the share of series items in the data; with few or noisy sequels the edge disappears.","key_machinery":"The load-bearing mechanism is sequel-aware message propagation. For each item $i$, the model computes $h^{\\mathrm{seq}}_i = \\frac{1}{|N_s(i)|} \\sum_{j \\in N_s(i)} h^{(l-1)}_j \\cdot P(j)$, where $N_s(i)$ is the set of items that follow $i$ in its series and $P(j)$ is a sinusoidal positional encoding of $j$'s position in the series. This term is what injects ordered series structure into item embeddings; it is fused with long-term and short-term interaction embeddings and the fused representation updates the item node. When this term is removed or the sequel edges are noisy, the model's advantage over a user-item GCN shrinks.","core_discovery":"The central claim is that temporal item-sequence structure is a usable signal for next-item recommendation. HSAL-GNN represents each item as either standalone or sequel; for sequel items it aggregates messages from the items that come later in the same series, weighted by a sinusoidal positional embedding, and then fuses this sequel-aware representation with long-term and short-term user-item signals before predicting the next item as a link in a sampled subgraph. The paper's empirical thesis is that this fusion improves ranking quality over graph-based recommendation without sequel information. The improvement concentrates in sequel-rich data: NDCG@10 rises by 6.46% over the strongest baseline on Goodreads, by a small margin on ML-1m, and not at all on ML-100K, where the model underperforms a graph baseline.","pith_inferences":["The paper's pattern of results implies a practical decision rule the authors do not state: estimate the share of sequel items and the reliability of series metadata before adopting sequel-aware modeling; if either is low, a standard user-item GCN is likely the safer choice.","A stronger test of the mechanism would use verified series metadata on MovieLens; the title-matching inference used here may understate the potential benefit of clean sequel edges.","Because the paper reports only leave-one-out evaluation on sampled data, a live deployment would reveal whether the NDCG gains translate into user-facing recommendation quality."],"forward_implications":["On platforms where items naturally form ordered series (books, films, games, courses), adding sequel edges to a recommender GNN should improve next-item ranking, with gains proportional to series density.","Below a certain fraction of series items \\u2014 around 4\\u20138% in the paper's data \\u2014 sequel-aware modeling stops helping and can lose to a plain graph model, so metadata coverage should be checked before adopting it.","Fusion strategy matters: simple sum-based fusion of sequel and interaction signals outperformed mean, concatenation, and a more complex heterogeneous transformation on the tested datasets.","The method extends to any directed, ordered item relation, not only entertainment sequels; prerequisite chains in education and versioned products are natural candidates for the same graph construction."],"supporting_citations":[{"why":"Supplies the main graph-based baseline (DGSR) that does not use sequel information and is the key comparison for the claim.","marker":"[2]"},{"why":"SASRec transformer baseline; one of the strongest sequence models the method must beat.","marker":"[24]"},{"why":"BERT4Rec bidirectional transformer baseline used in the comparison tables.","marker":"[19]"},{"why":"GRU4Rec recurrent baseline in the comparison set.","marker":"[1]"},{"why":"STOSA stochastic self-attention baseline providing item-distribution representations that the paper contrasts with its sequel-aware fusion.","marker":"[26]"},{"why":"DiffuRec diffusion-model baseline in the uncertainty-aware category.","marker":"[25]"},{"why":"Provides the MovieLens 100K and 1M datasets used to test the method on real interaction sequences.","marker":"[29]"},{"why":"Provides the Goodreads dataset used in the real-world evaluation, with explicit series information.","marker":"[27]"},{"why":"Companion Goodreads source used for the sampled subset construction in the real-world experiments.","marker":"[28]"}],"fun_headline_variants":["Temporal item sequences power GNN recommenders","Sequel-aware GNN improves sequential recommendations","Graph nets gain from item sequence context","Next-item prediction boosted by sequel-aware GNN"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole advantage depends on the sequel edges in the data being accurate and complete; MovieLens sequels are inferred by matching titles like 'Part 1' and 'Part 2', which can miss or mislabel series entries, and the Goodreads sample is described without reporting how it was drawn.","fun_headline_variants_meta":{"raw":{"variants":["Temporal item sequences power GNN recommenders","Sequel-aware GNN improves sequential recommendations","Graph nets gain from item sequence context","Next-item prediction boosted by sequel-aware GNN"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000405,"raw_usage":{"total_tokens":2057,"prompt_tokens":844,"completion_tokens":1213,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":460,"completion_tokens_details":{"reasoning_tokens":1157}},"tokens_in":460,"tokens_out":1213,"duration_ms":13111,"temperature":1.0,"reasoning_tokens":1157,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:13:04.649332+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run HSAL-GNN and the strongest graph baseline on a dataset with publisher-verified series metadata, holding the user-item graph fixed. If the sequel-aware model does not beat the baseline on series-rich item slices, or if randomly shuffling sequel links does not degrade its scores, then the claimed benefit of sequel information is not real.","supporting_citations":[{"cited_title":"Dynamic Graph Neural Networks for Sequential Recommendation","cited_arxiv_id":null,"evidence_quote":"Supplies the main graph-based baseline (DGSR) that does not use sequel information and is the key comparison for the claim."},{"cited_title":"and McAuley, J., 2018, November","cited_arxiv_id":null,"evidence_quote":"SASRec transformer baseline; one of the strongest sequence models the method must beat."},{"cited_title":"BERT4Rec: Sequential recommendation with bidirectional encoder representations from Transformer","cited_arxiv_id":null,"evidence_quote":"BERT4Rec bidirectional transformer baseline used in the comparison tables."},{"cited_title":"and Yu, P.S., 2022, April","cited_arxiv_id":null,"evidence_quote":"STOSA stochastic self-attention baseline providing item-distribution representations that the paper contrasts with its sequel-aware fusion."},{"cited_title":"and Li, C., 2023","cited_arxiv_id":null,"evidence_quote":"DiffuRec diffusion-model baseline in the uncertainty-aware category."},{"cited_title":"While all the details have been described in the main paper, the appendix presents a visual represen- tation of the algorithm framework","cited_arxiv_id":null,"evidence_quote":"Provides the MovieLens 100K and 1M datasets used to test the method on real interaction sequences."},{"cited_title":"Item Recommendation on Monotonic Behavior Chains","cited_arxiv_id":null,"evidence_quote":"Provides the Goodreads dataset used in the real-world evaluation, with explicit series information."},{"cited_title":"Fine-Grained Spoiler Detection from Large-Scale Review Corpora","cited_arxiv_id":null,"evidence_quote":"Companion Goodreads source used for the sampled subset construction in the real-world experiments."}],"review_version":1}