REVIEW 4 major objections 5 minor 38 references
Graph Embedding Based Hybrid Social Recommendation System
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A hybrid of three social-graph embeddings more than doubles recommendation coverage over the best single embedding on the Yelp graph.
desk verdict The hybrid's edge over single embeddings is an artifact of training and evaluating on the same 100 users; the paper is a clear but empirically unsupported incremental study. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a two-stage pipeline. First, the explicit Yelp friend graph is converted into an implicit weighted graph $G'$ by assigning edge weights $W_{ij}=(|L_i\cap L_j|+|D_i\cap D_j|)/(|L_i\cup L_j\cup D_i\cup D_j|)$ over liked and disliked restaurants. Three embeddings are computed on $G'$: spectral embedding from Laplacian eigenvectors, HOPE from a generalized SVD that preserves higher-order proximity, and node2vec from biased random walks. For each of 100 fixed users, the hybrid constructs a feature tensor $X$ of shape $(100,3,1434)$ where each entry records whether each embedding recommended a given restaurant from the union ground-truth set, with labels $Y$ marking the user's own high-rated restaurants; a dense neural network with layers of 32, 64, and 128 neurons, ReLU activations, MSE loss, and the Adam optimizer at learning rate $0.0001$, trained for 40 epochs, learns how to weight the three embeddings' votes.
What would settle it
Run the hybrid with a user-disjoint split: train on 80 of the 100 users and test on the remaining 20, while excluding the test users' high-rated restaurants from the training labels and from the top-user recommendation pool. If coverage on the held-out users falls back to roughly the 18–19% seen with individual embeddings, the claimed hybrid advantage is an artifact of training on the test users.
Extended reading notes
Core claim
The central discovery claimed is that a recommendation filter trained on the combined outputs of spectral embedding, HOPE, and node2vec outperforms each individual embedding on social-graph-based restaurant recommendation. On a Yelp-derived graph with 14,346 users and 407,495 weighted edges, the individual embeddings all land near 18–19% coverage at Top 100, while the hybrid model reports 48.53% coverage and a MAE of 0.514 on its test set. The paper interprets this as validating the hypothesis that different embedding methods capture complementary aspects of social structure—community, higher-order proximity, and random-walk neighborhoods—so their union carries more signal than any one view.
Load-bearing premise
The evaluation assumes that the same 100 users can be used both to train the hybrid model and to measure its test coverage, so if the model has effectively seen those users' high-rated restaurants during training, the reported 48.53% coverage is not a fair measure of how it would perform for a new user.
Editorial extensions
If this is right
- A social graph alone, with no item text or user demographics, can support restaurant recommendations: node2vec alone reaches 18.84% coverage and the hybrid reaches 48.53% at Top 100.
- Combining embeddings that preserve different graph structure—community, higher-order proximity, and random-walk neighborhoods—carries more recommendation signal than any single embedding.
- A small dense network (32-64-128 neurons, 40 epochs) can learn to weight per-restaurant votes from three embeddings using a training set of just 100 users.
- The paper reports a large gap between train coverage (63.21%) and test coverage (48.53%), and attributes it to users making individual choices that friends cannot predict, leaving room for better modeling of that residual randomness.
Reading between the lines
- Editorial inference: the reported gain would be more convincing if the hybrid were also compared against a pure neighborhood baseline that ignores embeddings, since the 'nearest ten neighbours weighted vote' step may carry much of the recommendation signal on its own.
- Editorial inference: a user-disjoint evaluation—train on some users, test on others—would tell whether the 48.53% coverage reflects a generalizable fusion rule or memorization of the 100 fixed users; the paper's own train-test gap suggests this matters.
- Editorial inference: if the complementary-aspects story is right, adding further embeddings (for example, graph convolutional or heterogeneous user-item embeddings) should push coverage higher, and the same architecture should transfer to movie or music social graphs without content features.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies three graph-embedding techniques (spectral embedding, HOPE, and node2vec) for restaurant recommendation from the Yelp social graph. It constructs a weighted user-user graph, obtains per-embedding restaurant recommendations from each user's nearest neighbors, and combines the three recommendation outputs with a deep neural network trained to predict each user's high-rated restaurants. The hybrid is reported to achieve 48.53% test coverage at Top-100 versus 18.84% for the best individual embedding (node2vec), and the authors conclude that the hybrid model validates their hypothesis that combining embeddings improves recommendation quality. The paper also reports MAE values, discusses future work, and provides a literature review.
Significance. If the reported comparison were trustworthy, the paper would provide simple evidence that fusing complementary graph embeddings improves social recommendation, which is a relevant and timely problem. The paper is clearly written and the problem is well motivated. However, the empirical evaluation is not reproducible as presented: no code or data are provided, the train/test protocol is unspecified, the similarity formula in Section III is malformed, and all results are single-run numbers from a hand-picked set of 100 users. The central empirical claim is therefore not established. The paper contains no machine-checked proofs, reproducible code, or parameter-free derivations that would compensate for these experimental weaknesses.
major comments (4)
- [Section IV-C and Table I] The paper never specifies how the 100 selected users are partitioned between training the deep network and computing the Hybrid(test) metric. Since X and Y are constructed for exactly the same 100 users (Section IV-C: 'A set of 100 well connected users are chosen for whom, restaurants are recommended and MAE is calculated'), any overlap between training rows and test rows allows the network to memorize which of the 1434 restaurants each user rated. The reported gap between Hybrid(test) at 48.53% and node2vec at 18.84% is therefore uninterpretable. Section VI's admission of a 'huge gap between train and test results' is consistent with memorization rather than generalization. Please specify a user-disjoint (or at least item-disjoint) split, report per-fold or cross-validated results, and apply the same protocol to all baselines.
- [Section III] The similarity weight in Section III is printed as W_ij = |L_i∩L_j| + (|D_i∩D_j| / |L_i∪L_j∪D_i∪D_j|, with an unbalanced parenthesis and no normalization applied to the first intersection term. Since this weight is used to construct the weighted graph G' that all embeddings receive, the formula must be corrected before the experimental setup is reproducible. As written, the formula also appears to give a liked-restaurant overlap and a normalized disliked-restaurant overlap, which is not the 'similarity score' described in the surrounding text.
- [Section V, Table I] The central comparison is structurally unfair. The hybrid is trained with supervision on the ground-truth labels Y, while the individual-embedding baselines receive no such training on those labels. The reported outperformance therefore conflates the benefit of fusing embeddings with the benefit of supervised fitting. To support the paper's hypothesis, the baselines need the same access to training labels (for example, a learned weighting of neighbor recommendations), or the hybrid must be evaluated in a label-free setting.
- [Section V] All reported numbers are single runs on one hand-picked set of 100 users, with no error bars, no cross-validation, and no significance tests. Given the many manually tuned hyperparameters (embedding dimension D, cluster count, number of nearest neighbors, rating bounds, DNN layer sizes, learning rate, and epoch count), the headline difference of 48.53% versus 18.84% cannot be assessed as statistically reliable. The paper should report variance across multiple user samples or random seeds.
minor comments (5)
- [Section III] The text says 'movies' where the domain is restaurants; for example, 'total number of movies both have seen' should read 'restaurants.'
- [Section IV-B] The number of clusters k for spectral embedding and the node2vec hyperparameters p and q are not reported, which prevents replication of the baseline results.
- [Section V] In the MAE formula, the variables N_r and N_hit are used but the surrounding text defines N as the test-set size; the definitions are inconsistent, and the word 'movies' appears again in a restaurant-recommendation context.
- [Table I] The hybrid rows only report Top-100 values; the Top-200 columns are empty, so the comparison against node2vec at Top-200 (35.76% coverage) is incomplete.
- [Figures] Figure captions are mismatched: Figure 1 is described as 'Architecture for hybrid Recommendation', Figure 2 as 'Deep learning Network Architecture', and Figure 3 as 'Coverage Graph', but the text in Sections IV-C and V does not clearly map these captions to the discussion.
Circularity Check
Hybrid 'test' coverage is measured against the same ground-truth labels used to train it; the comparison to untrained baselines makes the outperformance a fitted result rather than an independent prediction.
-
fitted input called prediction
[Section IV-C (Hybrid Recommendation), Y-label construction; Section V, Table I]
"we work on a constant ground by using a few selected users throughout the recommendation process. A set of 100 well connected users are chosen for whom, restaurants are recommended and MAE is calculated. ... In this vector Yi, the ground truth restaurants for ui are represented by 1 and rest by 0. The vector [Y1, Y2, ..Yn] is nothing but the labels Y for our data set, with the shape (100, 1434)."
The DNN is trained with MSE to map X (embedding recommendations for the 100 chosen users) to Y (the same users' ground-truth high-rated restaurants). Table I then reports 'Hybrid(test) 48.53' as if it were a prediction. Because the manuscript specifies no user-disjoint or item-disjoint split—only a 'constant ground' of the same 100 users for both dataset construction and MAE calculation—the test coverage is measured against the very labels the network was optimized to reproduce. The individual-embedding baselines are never fitted to Y, so the comparison is asymmetric: a fitted model is compared with unfitted baselines, and the claimed 48.53% coverage is partly a report of training fit rather than an independent validation of the hybrid hypothesis.
full rationale
The core empirical claim is that the hybrid DNN outperforms individual embeddings. The only load-bearing step that looks circular is the construction of the hybrid's supervision: Y is the ground-truth high-rated-restaurant vector for the same 100 users whose MAE/coverage is reported, and no split is described. That is pattern 2: the 'prediction' is the output of a network trained on the target labels, while baselines get no label training. This makes the superiority of hybrid in Table I a fitted value by construction, unless an undocumented split exists. No pattern 1, 3, 4, 5, or 6 is present: the embeddings are standard external methods, and the paper does not lean on self-citations or uniqueness theorems. The train-test gap noted in Section VI ('there is a huge gap between train and test results') is consistent with memorization rather than generalization and reinforces the concern. Score 6 because one or more reported 'predictions' reduce to a supervised fit on the evaluation labels; the paper is not wholly tautological, but the central comparison is structurally biased.
Assumptions & free parameters
free parameters (5)
- embedding dimension D =
25
- node2vec cluster count k =
3
- nearest neighbors count =
10
- high-rated restaurant bounds per user =
not specified
- DNN hyperparameters (layer sizes, learning rate, epochs) =
32,64,128; 0.0001; 40
assumptions (4)
- domain assumption Social influence alone drives item consumption, so recommendations from social-graph neighbors are suitable candidates
- domain assumption The three embedding methods (spectral, HOPE, node2vec) preserve the social structure needed for nearest-neighbor recommendations
- ad hoc to paper The similarity weight W_ij = |L_i∩L_j| + |D_i∩D_j| / |L_i∪L_j∪D_i∪D_j| is a valid measure of social proximity
- domain assumption Ground truth is the user's own high-rated restaurants, i.e., high ratings are the correct recommendation targets
Cite this review
Pith. "Pith review of Graph Embedding Based Hybrid Social Recommendation System." pith.science (2026). https://pith.science/paper/RNEF7UXT
@misc{pith2026190809454,
author = {Pith},
title = {Pith review of: Graph Embedding Based Hybrid Social Recommendation System},
year = {2026},
howpublished = {\url{https://pith.science/paper/RNEF7UXT}},
note = {Machine review of arXiv:1908.09454}
}
read the original abstract
Item recommendation tasks are a widely studied topic. Recent developments in deep learning and spectral methods paved a path towards efficient graph embedding techniques. But little research has been done on applying these graph embedding to social graphs for recommendation tasks. This paper focuses at performance of various embedding methods applied on social graphs for the task of item recommendation. Additionally, a hybrid model is proposed wherein chosen embedding models are combined together to give a collective output. We put forward the hypothesis that such a hybrid model would perform better than individual embedding for recommendation task. With recommendation using individual embedding as a baseline, performance for hybrid model for the same task is evaluated and compared. Standard metrics are used for qualitative comparison. It is found that the proposed hybrid model outperforms the baseline.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Social network data mining: Research questions, techniques, and applications,
N. Memon, J. J. Xu, D. L. Hicks, and H. Chen, “Social network data mining: Research questions, techniques, and applications,” in Data mining for social network data , pp. 1–7, Springer, 2010
work page 2010
-
[2]
Deep learning based rec- ommender system: A survey and new perspectives,
S. Zhang, L. Yao, A. Sun, and Y . Tay, “Deep learning based rec- ommender system: A survey and new perspectives,” ACM Computing Surveys (CSUR), vol. 52, no. 1, p. 5, 2019
work page 2019
-
[3]
S. Wasserman and K. Faust, Social network analysis: Methods and applications, vol. 8. Cambridge university press, 1994
work page 1994
-
[4]
A comprehensive survey of graph embedding: Problems, techniques, and applications,
H. Cai, V . W. Zheng, and K. C.-C. Chang, “A comprehensive survey of graph embedding: Problems, techniques, and applications,” IEEE Transactions on Knowledge and Data Engineering , vol. 30, no. 9, pp. 1616–1637, 2018
work page 2018
- [5]
-
[6]
A social network-based recommender system (snrs),
J. He and W. W. Chu, “A social network-based recommender system (snrs),” in Data mining for social network data , pp. 47–74, Springer, 2010
work page 2010
-
[7]
User’s interests- based movie recommendation in heterogeneous network,
W. Yang, X. Cui, J. Liu, Z. Wang, W. Zhu, and L. Wei, “User’s interests- based movie recommendation in heterogeneous network,” in 2015 In- ternational Conference on Identification, Information, and Knowledge in the Internet of Things (IIKI) , pp. 74–77, IEEE, 2015
work page 2015
-
[8]
A survey of active learning in collaborative filtering recommender systems,
M. Elahi, F. Ricci, and N. Rubens, “A survey of active learning in collaborative filtering recommender systems,”Computer Science Review, vol. 20, pp. 29–50, 2016
work page 2016
Show all 38 references
-
[9]
Introduction to recommender systems handbook,
F. Ricci, L. Rokach, and B. Shapira, “Introduction to recommender systems handbook,” in Recommender systems handbook , pp. 1–35, Springer, 2011
2011
-
[10]
Recommender systems survey,
J. Bobadilla, F. Ortega, A. Hernando, and A. Guti ´errez, “Recommender systems survey,”Knowledge-based systems, vol. 46, pp. 109–132, 2013
2013
-
[11]
Prof. a. thomas,survey on recommendation system meth- ods,
P. Nagarnaik, “Prof. a. thomas,survey on recommendation system meth- ods,” in International Conference On Eletronics And Communication System (ICECS), 2015
2015
-
[12]
A survey paper on recommender systems,
D. Almazro, G. Shahatah, L. Albdulkarim, M. Kherees, R. Martinez, and W. Nzoukou, “A survey paper on recommender systems,”arXiv preprint arXiv:1006.5278, 2010
2010 arXiv
-
[13]
Modeling the assimilation-contrast ef- fects in online product rating systems: Debiasing and recommendations,
X. Zhang, J. Zhao, and J. Lui, “Modeling the assimilation-contrast ef- fects in online product rating systems: Debiasing and recommendations,” in Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 98–106, ACM, 2017
2017
-
[14]
Hybrid recommender systems using social net- work analysis,
K.-J. Kim and H. Ahn, “Hybrid recommender systems using social net- work analysis,” in Proceedings of World Academy of Science, Engineer- ing and Technology , no. 64, World Academy of Science, Engineering and Technology, 2012
2012
-
[15]
Trust-based recommender systems: an overview,
A. Selmi, Z. Brahmi, and M. M. Gammoudi, “Trust-based recommender systems: an overview,” in Proceedings of 27th International Business Information Management Association (IBIMA) Conference, Milan, Italy, 2016
2016
-
[16]
Cross-domain recommendation via clustering on multi-layer graphs,
A. Farseev, I. Samborskii, A. Filchenkov, and T.-S. Chua, “Cross-domain recommendation via clustering on multi-layer graphs,” in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pp. 195–204, ACM, 2017
2017
-
[17]
Representation learning on graphs: Methods and applications,
W. L. Hamilton, R. Ying, and J. Leskovec, “Representation learning on graphs: Methods and applications,” arXiv preprint arXiv:1709.05584 , 2017
2017 arXiv
-
[18]
Graph embedding techniques, applications, and performance: A survey,
P. Goyal and E. Ferrara, “Graph embedding techniques, applications, and performance: A survey,”Knowledge-Based Systems, vol. 151, pp. 78–94, 2018
2018
-
[19]
Asymmetric transitivity preserving graph embedding,
M. Ou, P. Cui, J. Pei, Z. Zhang, and W. Zhu, “Asymmetric transitivity preserving graph embedding,” inProceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining , pp. 1105–1114, ACM, 2016
2016
-
[20]
Laplacian eigenmaps and spectral techniques for embedding and clustering,
M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embedding and clustering,” in Advances in neural information processing systems, pp. 585–591, 2002
2002
-
[21]
Distributed large-scale natural graph factorization,
A. Ahmed, N. Shervashidze, S. Narayanamurthy, V . Josifovski, and A. J. Smola, “Distributed large-scale natural graph factorization,” in Proceedings of the 22nd international conference on World Wide Web , pp. 37–48, ACM, 2013
2013
-
[22]
Deepwalk: Online learning of social representations,
B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: Online learning of social representations,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pp. 701–710, ACM, 2014
2014
-
[23]
node2vec: Scalable feature learning for networks,
A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining , pp. 855–864, ACM, 2016
2016
-
[24]
Semi-supervised classification with graph convolutional networks,
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907 , 2016
2016 arXiv
-
[25]
Distributed representations of words and phrases and their composition- ality,
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their composition- ality,” in Advances in neural information processing systems , pp. 3111– 3119, 2013
2013
-
[26]
Efficient estimation of word representations in vector space,
T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781 , 2013
2013 arXiv
-
[27]
Spectral clustering for link prediction in social networks with positive and negative links,
P. Symeonidis and N. Mantas, “Spectral clustering for link prediction in social networks with positive and negative links,” Social Network Analysis and Mining , vol. 3, no. 4, pp. 1433–1447, 2013
2013
-
[28]
Network embedding based recommendation method in social networks,
Y . Wen, L. Guo, Z. Chen, and J. Ma, “Network embedding based recommendation method in social networks,” in Companion of the The Web Conference 2018 on The Web Conference 2018 , pp. 11–12, International World Wide Web Conferences Steering Committee, 2018
2018
-
[29]
Real-time social recommendation based on graph embedding and temporal context,
P. Liu, L. Zhang, and J. A. Gulla, “Real-time social recommendation based on graph embedding and temporal context,” International Journal of Human-Computer Studies , vol. 121, pp. 58–72, 2019
2019
-
[30]
On spectral clustering: Anal- ysis and an algorithm,
A. Y . Ng, M. I. Jordan, and Y . Weiss, “On spectral clustering: Anal- ysis and an algorithm,” in Advances in neural information processing systems, pp. 849–856, 2002
2002
-
[31]
A tutorial on spectral clustering,
U. V on Luxburg, “A tutorial on spectral clustering,” Statistics and computing, vol. 17, no. 4, pp. 395–416, 2007
2007
-
[32]
On spectral graph embedding: A non-backtracking perspective and graph approximation,
F. Jiang, L. He, Y . Zheng, E. Zhu, J. Xu, and P. S. Yu, “On spectral graph embedding: A non-backtracking perspective and graph approximation,” in Proceedings of the 2018 SIAM International Conference on Data Mining, pp. 324–332, SIAM, 2018
2018
-
[33]
Spectral embedding of graphs,
B. Luo, R. C. Wilson, and E. R. Hancock, “Spectral embedding of graphs,” Pattern recognition, vol. 36, no. 10, pp. 2213–2230, 2003
2003
-
[34]
Hybrid recommender systems: A systematic literature review,
E. C ¸ ano and M. Morisio, “Hybrid recommender systems: A systematic literature review,” Intelligent Data Analysis , vol. 21, no. 6, pp. 1487– 1524, 2017
2017
-
[35]
Deep learning. nature 521 (7553): 436,
Y . LeCun, Y . Bengio, and G. Hinton, “Deep learning. nature 521 (7553): 436,” Google Scholar, 2015
2015
-
[36]
Continuous-time dynamic network embeddings,
G. H. Nguyen, J. B. Lee, R. A. Rossi, N. K. Ahmed, E. Koh, and S. Kim, “Continuous-time dynamic network embeddings,” in Companion of the The Web Conference 2018 on The Web Conference 2018 , pp. 969–976, International World Wide Web Conferences Steering Committee, 2018
2018
-
[37]
Dyngem: Deep embedding method for dynamic graphs,
P. Goyal, N. Kamra, X. He, and Y . Liu, “Dyngem: Deep embedding method for dynamic graphs,” arXiv preprint arXiv:1805.11273 , 2018
2018 arXiv
-
[38]
Attributed network embedding for learning in a dynamic environment,
J. Li, H. Dani, X. Hu, J. Tang, Y . Chang, and H. Liu, “Attributed network embedding for learning in a dynamic environment,” in Proceedings of the 2017 ACM on Conference on Information and Knowledge Manage- ment, pp. 387–396, ACM, 2017
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.