{"id":"8d1a6c53-d82e-4890-a40d-fd4b22167875","arxiv_id":"1908.05611","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Stage-wise training with neighbor resampling improves KGCN and RippleNet on most tested datasets and helps KGCN converge at higher graph hops, though a few Recall@K entries drop.","lead":"This paper proposes GraphSW, a training protocol that retrains graph-based recommender models on a new random slice of the knowledge graph at each stage and carries learned entity representations forward. It reports that this stage-wise resampling improves accuracy of two knowledge-graph recommender models on six datasets and helps deep-hop graph networks converge.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GraphSW's gains are confounded: no control isolates stage-wise embedding transfer from longer training and neighbor resampling, so the central causal claim is unsupported.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the paper does not control for total training budget or resampling alone, so the improvement attributed to GraphSW's stage-wise transfer is confounded. My reading of §2.2–2.3, §3.3.1, and Tables 2, 6, and 7 confirms that no equal-budget or resample-only control exists, and no epoch or stage counts are reported. This is the single most load-bearing weakness because the paper's central claim is causal (stage-wise embedding transfer produces the gains), not merely correlational. The proposed test would settle whether the transfer matters by comparing GraphSW against single-stage baselines with identical update counts and resampling schedules. The missing appendix containing RippleNet hop results is secondary; it undermines reproducibility but does not change the primary confound. Since my concern matches the reader's and does not overturn the CONDITIONAL verdict, no verdict adjustment is needed.","tokens_in":11602,"tokens_out":4735,"duration_ms":43159,"concrete_test":"Using the released code (github.com/mengruwu/graphsw), run KGCN on Last.FM and Book-Crossing with H=4 under three conditions: (A) GraphSW as published; (B) single-stage KGCN trained for the same total number of gradient steps as the summed GraphSW stages, with the same per-step neighbor-resampling schedule (resample the fixed-size neighbor set at the same frequency, but never stop, reinitialize, or transfer embeddings); (C) single-stage KGCN trained for the same number of steps with the neighbor set fixed as in the original baseline. Compare AUC across at least 5 seeds and report mean ± std. If (B) ≈ (A) > (C), the improvement is due to resampling or longer training, not stage-wise embedding transfer. Also report per-stage epoch counts, the number of stages S, and the early-stopping criteria for all runs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper claims that GraphSW's stage-wise embedding transfer improves KGCN and RippleNet and fixes KGCN's high-hop convergence (Tables 2, 3, 6, 7). The protocol in §2.2–2.3 changes three factors simultaneously: (i) training proceeds in multiple stages with early stopping per stage (§3.3.1), so total epochs can exceed the baseline's early-stopping budget; (ii) each stage samples a fresh fixed-size neighbor set G_{s+1}, letting the model see more KG entities than a baseline that samples once; (iii) KG embeddings are carried from W_s to W^{init}_{s+1} while aggregator parameters are retrained. The paper never removes (iii) or equalizes (i) and (ii). Table 7 only compares transferring whole parameters versus transferring only KG representation; there is no 'resample-only, no-transfer' arm, and no epoch counts, stage counts, or training budgets are reported anywhere. Since the contribution is specifically that transferring embeddings lets the model 'learn the information gradually' (Abstract, §2.3), the measured gains could be an artifact of simply training for more updates on more diverse neighbor samples. The high-hop result in Table 6 is especially vulnerable: the KGCN baseline collapses at H=4 (Last.FM AUC 0.5803), and a 34.8% relative improvement over a collapsed baseline may reflect additional training rather than the proposed protocol. Without controls matching total updates and neighbor-resampling schedule while withholding the transfer, the central causal attribution to stage-wise embedding transfer is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GraphSW, a stage-wise training protocol for KG-aware GNN-based recommender systems. The protocol divides training into stages; in each stage the model observes a fresh fixed-size neighbor set sampled from the knowledge graph, is trained with early stopping, and the learned KG embedding parameters are carried into the next stage while the aggregator parameters are retrained. The authors instantiate GraphSW on RippleNet and KGCN and report AUC/ACC and Recall@K results on six datasets, together with sensitivity studies over neighbor sampling size and hop number. The central claims are that GraphSW improves both models across all datasets and that it mitigates KGCN's difficulty in converging at high hop counts. A code repository is announced for reproducibility.","tokens_in":11823,"tokens_out":6705,"duration_ms":57993,"significance":"If the causal claim were established, GraphSW would be a simple, model-agnostic training protocol with practical value for GNN-based recommenders: it would reduce the cost of exploring large knowledge graphs and stabilize high-order propagation. The paper has several strengths: it evaluates on six real-world datasets, covers two representative models, releases code, and makes a falsifiable empirical prediction (stage-wise embedding transfer yields consistent gains). The sensitivity analysis over neighbor sampling size is also informative. However, the significance is currently contingent on an ablation that isolates the embedding-transfer mechanism from the other changes introduced by the protocol; without that ablation, the central attribution is not supported.","major_comments":[{"comment":"The central attribution of the reported gains to stage-wise embedding transfer is not supported by the current experiments. The protocol changes three factors simultaneously: (i) training proceeds in multiple stages with per-stage early stopping (§3.3.1), so the total number of updates can exceed the baseline's budget; (ii) each stage samples a fresh fixed-size neighbor set G_{s+1}, exposing the model to more KG entities than a single-stage baseline; and (iii) KG embeddings learned at stage s are transferred as W^{init}_{s+1} while aggregator parameters are retrained (§2.2). Table 7 only contrasts transferring all parameters with transferring KG embeddings only; it does not include a control that resamples neighbors and matches the total number of updates while withholding the transfer. Without such controls, the improvements in Tables 2, 3, and 6 could be explained by additional training or by neighbor resampling alone. Please add an equal-update, resample-only, no-transfer arm and report the stage count and per-stage epoch budget.","section":"§2.2–2.3, Table 7"},{"comment":"The experimental reporting is insufficient to assess the reliability of the improvements. The paper states that hyperparameters are chosen by optimizing AUC on a validation set and that each experiment is repeated at least 5 times, but it does not report the final per-dataset hyperparameters (learning rate, L2 weight, embedding dimension, neighbor sampling size K, number of stages S, and early-stopping patience), nor does it report standard deviations or significance tests. This matters because many reported gains are 1–2% in AUC (e.g., KGNN-SW on MovieLens-1M: 0.9223 vs 0.9171), which could be within run-to-run noise. Please provide the hyperparameter values used for each table and the variance or confidence intervals over the repeated runs.","section":"§3.3.1, Tables 2–7"},{"comment":"The claims that GraphSW 'improves the recommendation performance ... on all dataset' (§3.4.1) and 'improves KGCN and RippleNet on every dataset' (Conclusion) are contradicted by several Recall@K cells in Table 3. For example, Book-Crossing KGNN-SW has R@25 0.0478 versus baseline 0.0483, and Last.FM RippleNet-SW has R@25 0.1158 versus baseline 0.1177 and R@50 equal at 0.1917. The wording should be corrected to describe the pattern accurately, and the discussion should address why top-K Recall is not uniformly improved even when CTR metrics improve.","section":"Table 3, §3.4.1, Conclusion"},{"comment":"The high-hop convergence claim is evaluated only through end-point AUC of models trained with the full protocol, and the baseline collapses at H=4 (e.g., Last.FM AUC drops from 0.7865 at H=1 to 0.5803 at H=4). The reported 34.8% relative improvement for Last.FM is therefore measured against a nearly broken baseline, and the endpoint difference alone does not establish that the protocol 'assist[s] KGCN to converge effectively' unless the training budget and neighbor resampling are matched. Please report learning/validation curves for the baseline and GraphSW at H=4, as well as the exact training budgets, so the convergence claim can be evaluated independently of the final AUC numbers.","section":"Table 6, §3.4.3"}],"minor_comments":[{"comment":"The dataset names are inconsistent: the abstract lists 'movie', while the experiments use MovieLens-1M; please unify the naming.","section":"§1, Abstract"},{"comment":"The symbol S is used both for the number of training stages in §2.2 and for the neighbor sampling size in Tables 4 and 5; please use different symbols to avoid ambiguity.","section":"§2.2, Table 4"},{"comment":"The definition T_s = (u, v, G_s, Y) is informal: T_s appears to denote the whole training data available at stage s, but the tuple notation mixes a pair (u,v), a sampled neighbor set, and the interaction matrix. Please define the user-item set explicitly.","section":"§2.2"},{"comment":"The sentence 'the average improvement on hop number 4 is 34.8%, ...' does not specify whether these are relative or absolute improvements and over which baseline; please state the formula.","section":"§3.4.3"},{"comment":"The reference list contains duplicate entries (e.g., [2] and [7], [4] and [8], [10] and [14]); please consolidate them.","section":"References"},{"comment":"The text contains numerous OCR-like artifacts (e.g., 'pa/t_ttern', '/f_ixed-size', 'bene/f_icial', 'SOTA'), which should be cleaned in the camera-ready version.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads as a short workshop paper, and the experimental evidence is suggestive rather than conclusive. In its present form, the central causal claim—that stage-wise embedding transfer, not extra training or resampling alone, drives the gains—is untested. This is fixable with additional control experiments, so I would not reject, but the paper needs a substantial revision and more complete reporting. I would also ask the editor to check whether the journal's standards for reproducibility require the final hyperparameter settings to be included."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: GraphSW is a reasonable empirical idea, and the six-dataset comparison is useful, but the paper doesn't isolate the thing it claims to be testing. If the question is whether stage-wise embedding transfer helps KGCN and RippleNet, this paper doesn't yet answer it.\n\nWhat is actually new: applying stage-wise neighbor resampling with embedding carry-over to KGCN and RippleNet, evaluating on six datasets, and the observation that KGCN performs best with small sampled neighbor sets. Code is released. Tables 2–6 show consistent AUC/ACC gains in most cells, and the high-hop collapse of KGCN at H=4 is partly mitigated. That deserves credit.\n\nThe soft spots land on the central claim. The protocol changes three things at once: multiple stages with early stopping per stage, fresh neighbor samples each stage, and transfer of KG embeddings while retraining aggregators. The baseline trains with a single sampling and a single early-stopped run. There is no control with equal total updates and equal resampling but no transfer, so the gains could simply be more training on more diverse samples. Table 7 only compares transferring all parameters versus transferring KG representation; it doesn't remove transfer while keeping resampling. The paper also doesn't report stage counts, epoch counts, or final per-dataset hyperparameters, so the protocol is under-specified. Variance estimates are absent, and the claim that GraphSW improves 'all datasets' is contradicted by a few Recall@K decreases in Table 3 (e.g., Last.FM RippleNet R@25). The high-hop improvement over a collapsed baseline is especially vulnerable to the budget confound.\n\nThat said, this is a short workshop-style paper, not a deep theoretical claim. The references to prior work look fine, and the empirical pattern is plausible. The right fix is straightforward: add a resample-only, no-transfer arm with matched total updates, and report per-stage budgets and hyperparameters. With those controls, the paper would be solid.\n\nWho is this for: people working on KG-based GNN recommenders, especially those trying to stabilize high-hop KGCN training. A serious referee could give useful feedback. I'd send it to review, but I wouldn't cite the central claim until the controls are run.","headline":"Plausible empirical idea with useful six-dataset results, but the central claim—that stage-wise embedding transfer, not extra training and resampling, drives the gains—is not yet supported by the experiments.","tokens_in":12440,"tokens_out":2146,"would_cite":false,"duration_ms":22846,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A stage-wise schedule that resamples knowledge-graph neighbors and carries learned embeddings forward improves KGCN and RippleNet on six datasets and prevents KGCN's four-hop AUC collapse.","keywords":["knowledge graph recommendation","stage-wise training","graph neural networks","KGCN","RippleNet","neighbor sampling","embedding transfer","high-order graph features"],"falsifier":"Train KGCN and RippleNet under GraphSW's schedule but initialize every stage from random weights instead of transferring the learned KG embeddings, keeping the same total number of updates and the same neighbor resampling; if the AUC matches or exceeds GraphSW on the six datasets, the central claim that stage-wise embedding transfer is the cause of the gains is refuted. The control must use the same stage counts and per-stage epochs as GraphSW to be meaningful.","tokens_in":11331,"feed_emoji":"📈","tokens_out":10823,"duration_ms":93191,"temperature":0.7,"pith_summary":"GraphSW is a training schedule for knowledge-graph-aware recommender models that breaks a single training run into stages, gives each stage a different fixed-size sample of the knowledge graph, and carries the learned entity embeddings from one stage to the next. The paper argues this staged exposure lets RippleNet and KGCN absorb more of the graph's information without full-graph aggregation, and it specifically fixes KGCN's failure to converge when the number of hops reaches three or four. On six public datasets, the protocol raises AUC and Recall@K for both models; for KGCN it also changes the optimal configuration, with the best AUC appearing at small neighbor sampling sizes. The contribution is a training protocol rather than a new model, positioned as a drop-in improvement for fixed-size-neighbor GNN recommenders.","feed_headline":"Stage-wise training lifts KG-aware recommenders, tames high hops","feed_subtitle":"Carrying learned embeddings between stages lifts AUC on six datasets and stops high-hop collapse.","key_machinery":"The machinery is the stage-wise training loop defined by $W^{\\mathrm{init}}_{s+1} := W_s$ applied only to the knowledge-graph representation, while each stage $s$ draws a new fixed-size neighbor set $G_s$ from the KG. Each stage first fine-tunes the KG embedding on the sampled subgraph, then fine-tunes the aggregator and prediction head using that learned representation as initialization. The argument turns on this separation: the KG representation accumulates a gradually broader view of the graph across stages, while the aggregator parameters are retrained per stage, and the paper's whole-parameter-transfer experiment shows that collapsing this separation destroys the benefit.","core_discovery":"The paper's central claim is that stage-wise training—each stage sampling a fresh fixed-size neighbor set from the knowledge graph and initializing the next stage's KG embeddings with the previous stage's learned values—improves both accuracy and training stability for GNN-based KG-aware recommenders. On RippleNet the gains are largest on datasets with large KGs; on KGCN the protocol produces its best AUC with small neighbor sampling sizes, so the usual assumption that more neighbors per hop is better does not hold under this schedule. The sharpest result is at high hop counts: at four hops KGCN's AUC falls from 0.7865 to 0.5803 on Last.FM 2011 and from 0.6750 to 0.5628 on Book-Crossing without GraphSW, while with GraphSW the four-hop AUC stays near the one-hop value. Transferring the whole parameter set between stages degrades performance, so the paper concludes that transferring only the KG representation is what allows the model to build a broad view of the graph gradually.","pith_inferences":["The paper leaves untested whether the same total number of updates with per-step neighbor resampling but no embedding transfer would reproduce the gains; an equal-compute control would settle whether stage-wise transfer is the causal ingredient.","One testable interpretation is that stage-wise training acts as a denoising curriculum: early stages see small random neighbor samples that filter noisy entities, and later stages build high-order structure on a cleaner representation; this could be probed by measuring gradient or representation drift between stages.","The same principle may extend to other fixed-size-neighbor graph recommenders beyond knowledge-graph settings, provided only the entity or item representations are transferred between stages rather than all parameters.","The whole-parameter-transfer result suggests a design rule: keep the aggregator or prediction head fresh while letting the embedding layer carry memory across stages; ablating which parameter groups transfer, and when, would clarify the mechanism."],"forward_implications":["Under GraphSW, stage-wise training with embedding transfer is a drop-in schedule: it improves AUC for KGCN and RippleNet on all six datasets without changing the model architecture.","KGCN's collapse at three and four hops is treatable by training protocol rather than requiring a new architecture.","Under GraphSW, KGCN's best AUC occurs at small neighbor sampling sizes, so per-hop computation can be reduced without sacrificing accuracy.","RippleNet's largest gains appear on datasets with large knowledge graphs, so the protocol's value scales with the amount of side information."],"supporting_citations":[{"why":"RippleNet is one of the two baseline recommenders; GraphSW modifies its training protocol.","marker":"[15]"},{"why":"KGCN is the other baseline and is the source of the high-hop convergence problem that stage-wise training addresses.","marker":"[16]"},{"why":"Supplies the label-smoothness regularized variant of KGCN used in the experiments.","marker":"[17]"},{"why":"Provides dataset construction and the experimental setting for Yelp2018 and other benchmarks.","marker":"[18]"},{"why":"PinSage motivates the fixed-size neighbor sampling strategy and the resampling question that GraphSW investigates.","marker":"[21]"},{"why":"Stage-wise training is the learning strategy that GraphSW adapts to KG-aware recommenders.","marker":"[23]"},{"why":"Supplies the title-matching method used to build the LFM-1b and Amazon-book knowledge graphs.","marker":"[26]"}],"fun_headline_variants":["Stage-wise training prevents high-hop collapse in KG recommenders","GraphSW: carry learned embeddings across stages for better KG recs","Stage-wise embedding transfer staves off KG recommender collapse","Transfer embeddings, not weights: stage-wise wins for KG recs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim rests on the assumption that the measured gains come from transferring learned embeddings between stages, rather than from the larger total number of training updates or from resampling a different neighbor set in each pass; the paper reports no control that isolates these factors.","fun_headline_variants_meta":{"raw":{"variants":["Stage-wise training prevents high-hop collapse in KG recommenders","GraphSW: carry learned embeddings across stages for better KG recs","Stage-wise embedding transfer staves off KG recommender collapse","Transfer embeddings, not weights: stage-wise wins for KG recs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000775,"raw_usage":{"total_tokens":3493,"prompt_tokens":1076,"completion_tokens":2417,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":692,"completion_tokens_details":{"reasoning_tokens":2346}},"tokens_in":692,"tokens_out":2417,"duration_ms":18253,"temperature":1.0,"reasoning_tokens":2346,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:40:56.295359+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train KGCN and RippleNet under GraphSW's schedule but initialize every stage from random weights instead of transferring the learned KG embeddings, keeping the same total number of updates and the same neighbor resampling; if the AUC matches or exceeds GraphSW on the six datasets, the central claim that stage-wise embedding transfer is the cause of the gains is refuted. The control must use the same stage counts and per-stage epochs as GraphSW to be meaningful.","supporting_citations":[{"cited_title":"RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems","cited_arxiv_id":"1803.03467","evidence_quote":"RippleNet is one of the two baseline recommenders; GraphSW modifies its training protocol."},{"cited_title":"Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems","cited_arxiv_id":"1905.04413","evidence_quote":"Supplies the label-smoothness regularized variant of KGCN used in the experiments."},{"cited_title":"KGAT: Knowledge Graph Attention Network for Recommendation","cited_arxiv_id":"1905.07854","evidence_quote":"Provides dataset construction and the experimental setting for Yelp2018 and other benchmarks."},{"cited_title":"Stage-wise training: An improved feature learning strategy for deep models,","cited_arxiv_id":null,"evidence_quote":"Stage-wise training is the learning strategy that GraphSW adapts to KG-aware recommenders."},{"cited_title":"KB4Rec: A Dataset for Linking Knowledge Bases with Recommender Systems","cited_arxiv_id":"1807.11141","evidence_quote":"Supplies the title-matching method used to build the LFM-1b and Amazon-book knowledge graphs."}],"review_version":1}