REVIEW 4 major objections 6 minor 1 cited by
GraphSW: a training protocol based on stage-wise training for GNN-based Recommender Model
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A stage-wise schedule that resamples knowledge-graph neighbors and carries learned embeddings forward improves KGCN and RippleNet on six datasets and prevents KGCN's four-hop AUC collapse.
desk verdict Plausible empirical idea with useful six-dataset results, but the central claim—that stage-wise embedding transfer, not extra training and resampling, drives the gains—is not yet supported by the experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the stage-wise training loop defined by $W^{\mathrm{init}}_{s+1} := W_s$ applied only to the knowledge-graph representation, while each stage $s$ draws a new fixed-size neighbor set $G_s$ from the KG. Each stage first fine-tunes the KG embedding on the sampled subgraph, then fine-tunes the aggregator and prediction head using that learned representation as initialization. The argument turns on this separation: the KG representation accumulates a gradually broader view of the graph across stages, while the aggregator parameters are retrained per stage, and the paper's whole-parameter-transfer experiment shows that collapsing this separation destroys the benefit.
What would settle it
Train KGCN and RippleNet under GraphSW's schedule but initialize every stage from random weights instead of transferring the learned KG embeddings, keeping the same total number of updates and the same neighbor resampling; if the AUC matches or exceeds GraphSW on the six datasets, the central claim that stage-wise embedding transfer is the cause of the gains is refuted. The control must use the same stage counts and per-stage epochs as GraphSW to be meaningful.
Extended reading notes
Core claim
The paper's central claim is that stage-wise training—each stage sampling a fresh fixed-size neighbor set from the knowledge graph and initializing the next stage's KG embeddings with the previous stage's learned values—improves both accuracy and training stability for GNN-based KG-aware recommenders. On RippleNet the gains are largest on datasets with large KGs; on KGCN the protocol produces its best AUC with small neighbor sampling sizes, so the usual assumption that more neighbors per hop is better does not hold under this schedule. The sharpest result is at high hop counts: at four hops KGCN's AUC falls from 0.7865 to 0.5803 on Last.FM 2011 and from 0.6750 to 0.5628 on Book-Crossing without GraphSW, while with GraphSW the four-hop AUC stays near the one-hop value. Transferring the whole parameter set between stages degrades performance, so the paper concludes that transferring only the KG representation is what allows the model to build a broad view of the graph gradually.
Load-bearing premise
The central claim rests on the assumption that the measured gains come from transferring learned embeddings between stages, rather than from the larger total number of training updates or from resampling a different neighbor set in each pass; the paper reports no control that isolates these factors.
Editorial extensions
If this is right
- Under GraphSW, stage-wise training with embedding transfer is a drop-in schedule: it improves AUC for KGCN and RippleNet on all six datasets without changing the model architecture.
- KGCN's collapse at three and four hops is treatable by training protocol rather than requiring a new architecture.
- Under GraphSW, KGCN's best AUC occurs at small neighbor sampling sizes, so per-hop computation can be reduced without sacrificing accuracy.
- RippleNet's largest gains appear on datasets with large knowledge graphs, so the protocol's value scales with the amount of side information.
Reading between the lines
- The paper leaves untested whether the same total number of updates with per-step neighbor resampling but no embedding transfer would reproduce the gains; an equal-compute control would settle whether stage-wise transfer is the causal ingredient.
- One testable interpretation is that stage-wise training acts as a denoising curriculum: early stages see small random neighbor samples that filter noisy entities, and later stages build high-order structure on a cleaner representation; this could be probed by measuring gradient or representation drift between stages.
- The same principle may extend to other fixed-size-neighbor graph recommenders beyond knowledge-graph settings, provided only the entity or item representations are transferred between stages rather than all parameters.
- The whole-parameter-transfer result suggests a design rule: keep the aggregator or prediction head fresh while letting the embedding layer carry memory across stages; ablating which parameter groups transfer, and when, would clarify the mechanism.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GraphSW, a stage-wise training protocol for KG-aware GNN-based recommender systems. The protocol divides training into stages; in each stage the model observes a fresh fixed-size neighbor set sampled from the knowledge graph, is trained with early stopping, and the learned KG embedding parameters are carried into the next stage while the aggregator parameters are retrained. The authors instantiate GraphSW on RippleNet and KGCN and report AUC/ACC and Recall@K results on six datasets, together with sensitivity studies over neighbor sampling size and hop number. The central claims are that GraphSW improves both models across all datasets and that it mitigates KGCN's difficulty in converging at high hop counts. A code repository is announced for reproducibility.
Significance. If the causal claim were established, GraphSW would be a simple, model-agnostic training protocol with practical value for GNN-based recommenders: it would reduce the cost of exploring large knowledge graphs and stabilize high-order propagation. The paper has several strengths: it evaluates on six real-world datasets, covers two representative models, releases code, and makes a falsifiable empirical prediction (stage-wise embedding transfer yields consistent gains). The sensitivity analysis over neighbor sampling size is also informative. However, the significance is currently contingent on an ablation that isolates the embedding-transfer mechanism from the other changes introduced by the protocol; without that ablation, the central attribution is not supported.
major comments (4)
- [§2.2–2.3, Table 7] The central attribution of the reported gains to stage-wise embedding transfer is not supported by the current experiments. The protocol changes three factors simultaneously: (i) training proceeds in multiple stages with per-stage early stopping (§3.3.1), so the total number of updates can exceed the baseline's budget; (ii) each stage samples a fresh fixed-size neighbor set G_{s+1}, exposing the model to more KG entities than a single-stage baseline; and (iii) KG embeddings learned at stage s are transferred as W^{init}_{s+1} while aggregator parameters are retrained (§2.2). Table 7 only contrasts transferring all parameters with transferring KG embeddings only; it does not include a control that resamples neighbors and matches the total number of updates while withholding the transfer. Without such controls, the improvements in Tables 2, 3, and 6 could be explained by additional training or by neighbor resampling alone. Please add an equal-update, resample-only, no-transfer arm and report the stage count and per-stage epoch budget.
- [§3.3.1, Tables 2–7] The experimental reporting is insufficient to assess the reliability of the improvements. The paper states that hyperparameters are chosen by optimizing AUC on a validation set and that each experiment is repeated at least 5 times, but it does not report the final per-dataset hyperparameters (learning rate, L2 weight, embedding dimension, neighbor sampling size K, number of stages S, and early-stopping patience), nor does it report standard deviations or significance tests. This matters because many reported gains are 1–2% in AUC (e.g., KGNN-SW on MovieLens-1M: 0.9223 vs 0.9171), which could be within run-to-run noise. Please provide the hyperparameter values used for each table and the variance or confidence intervals over the repeated runs.
- [Table 3, §3.4.1, Conclusion] The claims that GraphSW 'improves the recommendation performance ... on all dataset' (§3.4.1) and 'improves KGCN and RippleNet on every dataset' (Conclusion) are contradicted by several Recall@K cells in Table 3. For example, Book-Crossing KGNN-SW has R@25 0.0478 versus baseline 0.0483, and Last.FM RippleNet-SW has R@25 0.1158 versus baseline 0.1177 and R@50 equal at 0.1917. The wording should be corrected to describe the pattern accurately, and the discussion should address why top-K Recall is not uniformly improved even when CTR metrics improve.
- [Table 6, §3.4.3] The high-hop convergence claim is evaluated only through end-point AUC of models trained with the full protocol, and the baseline collapses at H=4 (e.g., Last.FM AUC drops from 0.7865 at H=1 to 0.5803 at H=4). The reported 34.8% relative improvement for Last.FM is therefore measured against a nearly broken baseline, and the endpoint difference alone does not establish that the protocol 'assist[s] KGCN to converge effectively' unless the training budget and neighbor resampling are matched. Please report learning/validation curves for the baseline and GraphSW at H=4, as well as the exact training budgets, so the convergence claim can be evaluated independently of the final AUC numbers.
minor comments (6)
- [§1, Abstract] The dataset names are inconsistent: the abstract lists 'movie', while the experiments use MovieLens-1M; please unify the naming.
- [§2.2, Table 4] The symbol S is used both for the number of training stages in §2.2 and for the neighbor sampling size in Tables 4 and 5; please use different symbols to avoid ambiguity.
- [§2.2] The definition T_s = (u, v, G_s, Y) is informal: T_s appears to denote the whole training data available at stage s, but the tuple notation mixes a pair (u,v), a sampled neighbor set, and the interaction matrix. Please define the user-item set explicitly.
- [§3.4.3] The sentence 'the average improvement on hop number 4 is 34.8%, ...' does not specify whether these are relative or absolute improvements and over which baseline; please state the formula.
- [References] The reference list contains duplicate entries (e.g., [2] and [7], [4] and [8], [10] and [14]); please consolidate them.
- [Throughout] The text contains numerous OCR-like artifacts (e.g., 'pa/t_ttern', '/f_ixed-size', 'bene/f_icial', 'SOTA'), which should be cleaned in the camera-ready version.
Circularity Check
No significant circularity: reported gains are held-out measurements of a training protocol against external baselines, not fitted predictions or self-citation-derived results.
full rationale
The paper's central claim is that GraphSW, a stage-wise training protocol with embedding transfer and neighbor resampling, improves KGCN and RippleNet on multiple datasets. This claim is supported by empirical AUC, ACC, and Recall measurements on held-out test splits (Tables 2-7). Equations (1) and (2) merely define the stage-wise weight-transfer update W_s = A(T_s, W_init_s) and W_init_{s+1} := W_s; they do not analytically derive the reported performance nor define any output metric in terms of a fitted constant. The paper does not fit a parameter to one subset and then 'predict' a quantity that is algebraically identical to that fit. There are no self-citations that are load-bearing: the baselines (RippleNet, KGCN) and the stage-wise training concept are external prior work, cited normally. The paper also includes an ablative comparison in Table 7 (transferring only KG representations versus transferring whole parameters), which supports rather than assumes the mechanism. The skeptic's concern that the protocol changes multiple factors simultaneously—total training budget, neighbor resampling, and embedding transfer—without isolating them is a valid experimental-control threat, but it is not circularity: the claimed improvements are not true by construction or by definition, and no step reduces to its own input. The absence of such controls affects causal attribution and correctness risk, not the circularity score. Under the rules, a non-finding is appropriate.
Assumptions & free parameters
free parameters (5)
- learning rate =
not reported per dataset
- L2 regularization weight =
not reported per dataset
- embedding dimension =
8 for RippleNet, 16 for KGCN
- neighbor sampling size K =
swept over 2, 4, 8, 16, 32, 64
- number of training stages and per-stage budget =
not specified
assumptions (3)
- domain assumption The public knowledge graphs and the 20/50-core filtering produce a fair testbed for recommender accuracy.
- domain assumption Validation-set hyperparameter tuning transfers to the held-out test distribution.
- domain assumption Adam optimization with early stopping finds comparable local optima for the baseline and GraphSW models.
Cite this review
Pith. "Pith review of GraphSW: a training protocol based on stage-wise training for GNN-based Recommender Model." pith.science (2026). https://pith.science/paper/BJCHM5NL
@misc{pith2026190805611,
author = {Pith},
title = {Pith review of: GraphSW: a training protocol based on stage-wise training for GNN-based Recommender Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/BJCHM5NL}},
note = {Machine review of arXiv:1908.05611}
}
read the original abstract
Recently, researchers utilize Knowledge Graph (KG) as side information in recommendation system to address cold start and sparsity issue and improve the recommendation performance. Existing KG-aware recommendation model use the feature of neighboring entities and structural information to update the embedding of currently located entity. Although the fruitful information is beneficial to the following task, the cost of exploring the entire graph is massive and impractical. In order to reduce the computational cost and maintain the pattern of extracting features, KG-aware recommendation model usually utilize fixed-size and random set of neighbors rather than complete information in KG. Nonetheless, there are two critical issues in these approaches: First of all, fixed-size and randomly selected neighbors restrict the view of graph. In addition, as the order of graph feature increases, the growth of parameter dimensionality of the model may lead the training process hard to converge. To solve the aforementioned limitations, we propose GraphSW, a strategy based on stage-wise training framework which would only access to a subset of the entities in KG in every stage. During the following stages, the learned embedding from previous stages is provided to the network in the next stage and the model can learn the information gradually from the KG. We apply stage-wise training on two SOTA recommendation models, RippleNet and Knowledge Graph Convolutional Networks (KGCN). Moreover, we evaluate the performance on six real world datasets, Last.FM 2011, Book-Crossing,movie, LFM-1b 2015, Amazon-book and Yelp 2018. The result of our experiments shows that proposed strategy can help both models to collect more information from the KG and improve the performance. Furthermore, it is observed that GraphSW can assist KGCN to converge effectively in high-order graph feature.
Figures
Forward citations
Cited by 1 Pith paper
-
Knowledge Graphs Meet Graph Neural Networks: A Comprehensive Survey
A two-level taxonomy (KG pipeline stages × GNN architectures) systematically reviews GNN methods for knowledge-graph construction, embedding, reasoning, and applications.
Reference graph
Works this paper leans on
-
[6]
Y. Cao, X. Wang, X. He, Z. Hu, and T. Chua, “Unifying knowledge graph learning and recommendation: Towards a be/t_ter understanding of user preferences, ”CoRR, vol. abs/1902.06236, 2019. [Online]. Available: h/t_tp://arxiv.org/abs/1902.06236
work page Pith review arXiv 1902
-
[7]
Improving sequential recommendation with knowledge-enhanced memory networks,
J. Huang, W. X. Zhao, H. Dou, J.-R. Wen, and E. Y. Chang, “Improving sequential recommendation with knowledge-enhanced memory networks, ” in /T_he 41st International ACM SIGIR Conference on Research & Development in Information Retrieval , ser. SIGIR ’18. New York, NY, USA: ACM, 2018, pp. 505–514. [Online]. Available: h/t_tp://doi.acm.org/10.1145/3209...
arXiv 2018
-
[8]
Collaborative knowledge base embedding for recommender systems,
F. Zhang, N. J. Yuan, D. Lian, X. Xie, and W.-Y. Ma, “Collaborative knowledge base embedding for recommender systems, ” in Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ser. KDD ’16. New York, NY, USA: ACM, 2016, pp. 353–362. [Online]. Available: h/t_tp://doi.acm.org/10.1145/2939672.2939673
arXiv 2016
-
[9]
Dkn: Deep knowledge-aware network for news recommendation,
H. Wang, F. Zhang, X. Xie, and M. Guo, “Dkn: Deep knowledge-aware network for news recommendation, ” inProceedings of the 2018 World Wide Web Conference, ser. WWW ’18. Republic and Canton of Geneva, Switzerland: International World Wide Web Conferences Steering Commi/t_tee, 2018, pp. 1835–1844. [Online]. Available: h/t_tps://doi.org/10.1145/3178876.3186175
arXiv 2018
-
[10]
Leveraging meta-path based context for top- n recommendation with a neural co-a/t_tention model,
B. Hu, C. Shi, W. X. Zhao, and P. S. Yu, “Leveraging meta-path based context for top- n recommendation with a neural co-a/t_tention model, ” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’18. New York, NY, USA: ACM, 2018, pp. 1531–1540. [Online]. Available: h/t_tp://doi.acm.org/10.1145/32...
arXiv 2018
-
[11]
Recurrent knowledge graph embedding for effective recommendation,
Z. Sun, J. Yang, J. Zhang, A. Bozzon, L.-K. Huang, and C. Xu, “Recurrent knowledge graph embedding for effective recommendation, ” in Proceedings of the 12th ACM Conference on Recommender Systems , ser. RecSys ’18. New York, NY, USA: ACM, 2018, pp. 297–305. [Online]. Available: h/t_tp: //doi.acm.org/10.1145/3240323.3240361
arXiv 2018
-
[12]
Explainable reasoning over knowledge graphs for recommendation,
X. Wang, D. Wang, C. Xu, X. He, Y. Cao, and T. Chua, “Explainable reasoning over knowledge graphs for recommendation, ”CoRR, vol. abs/1811.04540, 2018. [Online]. Available: h/t_tp://arxiv.org/abs/1811.04540
arXiv 2018
-
[13]
Personalized entity recommendation: A heterogeneous information network approach,
X. Yu, X. Ren, Y. Sun, Q. Gu, B. Sturt, U. Khandelwal, B. Norick, and J. Han, “Personalized entity recommendation: A heterogeneous information network approach, ” inProceedings of the 7th ACM International Conference on Web Search and Data Mining, ser. WSDM ’14. New York, NY, USA: ACM, 2014, pp. 283–292. [Online]. Available: h/t_tp://doi.acm.org/10.1145/2...
arXiv 2014
Show all 21 references
-
[14]
Meta-graph based recommendation fusion over heterogeneous information networks,
H. Zhao, Q. Yao, J. Li, Y. Song, and D. L. Lee, “Meta-graph based recommendation fusion over heterogeneous information networks, ” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ser. KDD ’17. New York, NY, USA: ACM, 2017...
2017
-
[15]
Ripple network: Propagating user preferences on the knowledge graph for recommender systems,
H. Wang, F. Zhang, J. Wang, M. Zhao, W. Li, X. Xie, and M. Guo, “Ripple network: Propagating user preferences on the knowledge graph for recommender systems, ”CoRR, vol. abs/1803.03467, 2018. [Online]. Available: h/t_tp://arxiv.org/abs/1803.03467
2018 arXiv
-
[16]
Knowledge graph convolutional networks for recommender systems,
H. Wang, M. Zhao, X. Xie, W. Li, and M. Guo, “Knowledge graph convolutional networks for recommender systems, ”CoRR, vol. abs/1904.12575, 2019. [Online]. Available: h/t_tp://arxiv.org/abs/1904.12575
1904 arXiv
-
[17]
Knowledge graph convolutional networks for recommender systems with label smoothness regularization,
H. Wang, F. Zhang, M. Zhang, J. Leskovec, M. Zhao, W. Li, and Z. Wang, “Knowledge graph convolutional networks for recommender systems with label smoothness regularization, ”CoRR, vol. abs/1905.04413, 2019. [Online]. Available: h/t_tp://arxiv.org/abs/1905.04413
1905 arXiv
-
[18]
KGAT: knowledge graph a/t_tention network for recommendation,
X. Wang, X. He, Y. Cao, M. Liu, and T. Chua, “KGAT: knowledge graph a/t_tention network for recommendation, ” CoRR, vol. abs/1905.07854, 2019. [Online]. Available: h/t_tp://arxiv.org/abs/1905.07854
1905 arXiv
-
[19]
A neural in/f_luence diffusion model for social recommendation,
L. Wu, P. Sun, Y. Fu, R. Hong, X. Wang, and M. Wang, “A neural in/f_luence diffusion model for social recommendation, ” CoRR, vol. abs/1904.10322, 2019. [Online]. Available: h/t_tp://arxiv.org/abs/1904.10322
1904 arXiv
-
[20]
Neural graph collaborative /f_iltering,
X. Wang, X. He, M. Wang, F. Feng, and T. Chua, “Neural graph collaborative /f_iltering, ”CoRR, vol. abs/1905.08108, 2019. [Online]. Available: h/t_tp://arxiv.org/abs/1905.08108
1905 arXiv
-
[21]
Graph convolutional neural networks for web-scale recommender systems,
R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, “Graph convolutional neural networks for web-scale recommender systems, ”CoRR, vol. abs/1806.01973, 2018. [Online]. Available: h/t_tp://arxiv.org/abs/1806.01973
2018 arXiv
-
[22]
Graph convolutional matrix completion for bi- partite edge prediction,
Y. Wu., H. Liu., and Y. Yang., “Graph convolutional matrix completion for bi- partite edge prediction, ” inProceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR,, INSTICC. SciTePress, 2018...
2018
-
[23]
Stage-wise training: An improved feature learning strategy for deep models,
E. Barshan and P. Fieguth, “Stage-wise training: An improved feature learning strategy for deep models, ” in Proceedings of the 1st International Workshop on Feature Extraction: Modern /Q_uestions and Challenges at NIPS 2015 , ser. Proceedings of Machine Learning Research, D. ...
2015
-
[24]
Densely connected pyramid dehazing network,
H. Zhang and V. M. Patel, “Densely connected pyramid dehazing network, ”CoRR, vol. abs/1803.08396, 2018. [Online]. Available: h/t_tp://arxiv.org/abs/1803.08396
2018 arXiv
-
[25]
Multimodal deep learning for robust RGB-D object recognition,
A. Eitel, J. T. Springenberg, L. Spinello, M. A. Riedmiller, and W. Burgard, “Multimodal deep learning for robust RGB-D object recognition, ” CoRR, vol. abs/1507.06821, 2015. [Online]. Available: h/t_tp://arxiv.org/abs/1507.06821
2015 arXiv
-
[26]
Kb4rec: A dataset for linking knowledge bases with recommender systems,
W. X. Zhao, G. He, H. Dou, J. Huang, S. Ouyang, and J. Wen, “Kb4rec: A dataset for linking knowledge bases with recommender systems, ” CoRR, vol. abs/1807.11141, 2018. [Online]. Available: h/t_tp://arxiv.org/abs/1807.11141
2018 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.