REVIEW 3 major objections 6 minor 32 references
Boosting the Rating Prediction with Click Data and Textual Contents
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper proposes TCF, a joint model of ratings, textual contents, and item co-click data, and reports lower prediction error than four baselines on sparse rating datasets.
desk verdict A plausible model combination with a train/test leak that undermines the headline empirical claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a shared latent item vector $\boldsymbol{\beta}_i$ that is pinned to three generative processes. Textually, $\boldsymbol{\beta}_i$ is drawn from the middle layer of an SDAE, so the bag-of-words description of item $i$ shapes it. For clicks, the same vector appears in a PPMI matrix factorization: for item pairs $(i,j)$, the model sets $\boldsymbol{\beta}_i^\top \boldsymbol{\alpha}_j \approx s_{ij}$, where $s_{ij} = \max\{\log \#(i,j)|D|/(\#(i)\#(j)), 0\}$; the context vectors $\boldsymbol{\alpha}_j$ absorb the role of the neighbor. For ratings, $\boldsymbol{\beta}_i$ is multiplied by a user vector $\boldsymbol{\theta}_u$ to predict $r_{ui}$. Alternating updates for $\boldsymbol{\theta}$, $\boldsymbol{\beta}$, $\boldsymbol{\alpha}$, and the SDAE weights minimize one joint objective (Eq. 23), with $\lambda_S$ balancing click versus rating information.
What would settle it
Re-run the MovieTweetings comparison with a click matrix built only from a pre-split click log or from the training portion of the binarized ratings, excluding test users' clicks; if TCF's RMSE advantage over CVAE shrinks or vanishes, the reported gain comes from the click matrix leaking test ratings, while if the advantage persists, the co-click signal is genuinely independent.
Extended reading notes
Core claim
On the paper's own terms, TCF establishes that item co-click information and textual contents should be learned jointly with the rating matrix rather than used as separate features or post-hoc regularizers. The generative model draws each item's feature vector from the middle layer of a stacked denoising autoencoder on bag-of-words text, then uses that same vector to predict both ratings and the entries of a positive pointwise mutual information (PPMI) matrix formed from co-clicks. The central identity is that the log ratio $p(i|j)/p(i)$ equals $\boldsymbol{\beta}_i^\top \boldsymbol{\alpha}_j$, meaning the pointwise mutual information of two items is the inner product of an item feature vector and an item context vector; TCF factorizes the PPMI version of this matrix together with the rating matrix. The authors report that this joint factorization beats text-only hybrids and pure matrix factorization in in-matrix prediction, and that it predicts out-of-matrix ratings for items with clicks but no prior ratings better than the text-only baselines.
Load-bearing premise
The load-bearing premise is that the click matrix is an independent auxiliary signal; on MovieTweetings it is constructed by turning the ratings into click indicators before splitting the data, so the co-click matrix can carry information about test ratings.
Editorial extensions
If this is right
- Items that have never been rated but have been clicked can still get predicted ratings, because their item vector is formed by the text encoder and the co-click factorization.
- The benefit over text-only baselines grows as the rating matrix gets sparser, so implicit feedback is most valuable precisely when explicit ratings are scarce.
- In out-of-matrix prediction, where test items have no ratings in the training set, TCF reports lower RMSE than CTR, CDL, and CVAE, meaning the click signal helps cold-start items.
- The learned item vectors also improve a downstream classification task compared with embeddings learned by other methods, so the co-click signal improves representational quality, not just rating loss.
- The model depends on a balance parameter $\lambda_S$: both too little and too much click influence degrade accuracy, so the two signals must be weighted against each other.
Reading between the lines
- A natural extension not tested in the paper is to replace the user-based context with session-based contexts, so items clicked in the same browsing session form the co-click matrix; this would generate a different item-item graph and may capture relationships that user histories miss.
- The same joint-loss template could be reused with a different text encoder, such as a transformer-based encoder, in place of the SDAE; the rating and co-click terms would remain unchanged, making it straightforward to test whether better text representations compound the gain.
- The classification results suggest the item vectors are useful beyond RMSE; one could test whether the co-click-augmented embeddings also improve retrieval or ranking metrics such as precision at k or normalized discounted cumulative gain.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TCF, a probabilistic matrix-factorization model that jointly learns item representations from explicit ratings, item textual content via a stacked denoising autoencoder (SDAE), and item co-click data via a PPMI item-item matrix. The generative process places the item feature vector beta_i near the SDAE encoder output, draws PPMI entries from beta_i^T alpha_j, and draws ratings from theta_u^T beta_i. The learning objective in Eq. (23) combines the rating likelihood, the PPMI factorization, and the SDAE reconstruction terms, with coordinate updates for theta, beta, alpha, and the SDAE weights in Eqs. (24)-(28). Experiments on MovieTweetings and Bookcrossing compare in-matrix and out-of-matrix RMSE with PMF, CTR, CDL, and CVAE across sparsity levels MT-10/20/50/80. The paper claims that TCF significantly outperforms the baselines, especially on extremely sparse data, and that the click signal helps cold-start rating prediction.
Significance. The proposed combination of an SDAE text model, a rating matrix factorization, and a PPMI click factorization is a plausible and coherent extension of CDL. The objective is not tautological: the PPMI factorization has its own auxiliary signal and is not derived from the rating target. The coordinate update derivations and the complexity analysis are useful contributions. However, the empirical evidence for the central claim is compromised on MovieTweetings because the click matrix is built from all rating data before the train/test split. This makes the reported sparse-data gains potentially artifacts of label leakage. The significance of the paper therefore rests on whether the experiments can be rerun with a leakage-free protocol; with such a rerun, the model may well be a meaningful contribution to cold-start rating prediction.
major comments (3)
- [Section 4.1-4.2, Eq. (23)] The MovieTweetings implicit feedback matrix is constructed by binarizing all rating data before the split: Section 4.1 states that 'the implicit feedback is obtained by binarizing the rating data' and that 'we use all the rating data by considering whether a user rated an item or not,' while Section 4.2 then splits the rating data into 80% training and 20% test. The PPMI matrix S used in Eq. (23) therefore encodes the test ratings. For in-matrix prediction, every test pair (u,i) contributes to the co-click statistics of item i; for out-of-matrix prediction, the test item's click vector is derived exclusively from ratings that later appear in the test set. Because the beta_i update in Eq. (25) and the SDAE gradients in Eq. (27) depend on S, test outcomes leak into the training signal. The reported gains on MT-10 and MT-20 in Fig. 2 may therefore be inflated. The experiments must be rerun with the implicit feedback constructed from the training split only, and the revised RMSE values must be reported.
- [Section 4.4, Tables 4a/4b, Fig. 2] All comparisons report a single RMSE value for each method and dataset, with no standard deviations, confidence intervals, or significance tests. Since the explicit feedback subsets are randomly sampled (Section 4.1) and the validation split is random, the reported differences, particularly the gaps between TCF and CVAE on Bookcrossing, could be within random variation. Please report means and standard deviations over multiple random splits, or otherwise establish that the improvements are statistically significant.
- [Section 4.2, out-of-matrix prediction] The out-of-matrix protocol says that test items have no ratings in the training set, but the click matrix for those same items is built from the full rating set. A genuinely cold-start item should also have no rating-derived implicit signal in the training click matrix; otherwise the out-of-matrix evaluation does not measure cold-start performance. The authors should define out-of-matrix prediction using only training-derived clicks, text, and ratings, and rerun the out-of-matrix experiments accordingly.
minor comments (6)
- [Abstract and Section 1] The model name is inconsistent: the abstract uses 'TCMF' while the rest of the paper uses 'TCF.' Please unify the terminology.
- [Section 3.3, Eqs. (22) and (23)] Eq. (22) writes r_{uj} on the left but beta_i on the right, and the first term of Eq. (23) uses theta_i^T beta_i instead of theta_u^T beta_i. These typos make the derivation harder to follow.
- [Fig. 2] The legend of Fig. 2a labels the proposed method as 'CTF' instead of 'TCF.' Please correct the label.
- [Section 1] The introduction says the experiments use 'three public datasets,' but Section 4.1 describes only two datasets. Please adjust the text to match the actual experiments.
- [Table 3] The density values in Table 3 are inconsistent with the counts in Table 2: with 630,000 ratings over 50,000 users and 28,000 items, taking 80% of the ratings yields approximately 0.036% density, not 1.9348% as reported. Please check the computation and clarify what the table reports.
- [Abstract] The abstract promises a comparison on a classification task using the learned item representations, but the full text contains no classification experiments. Either add the experiments or remove that claim.
Circularity Check
MovieTweetings evaluation is partly circular: the 'click matrix' is binarized from all ratings, so the PPMI item vectors are fitted to the very test ratings whose prediction is claimed as a gain.
-
fitted input called prediction
[Section 4.1 (MovieTweetings construction) with Section 4.2 split and Eq. (25)-(26)]
"Since MovieTweetings datasets contain only explicit feedback, we artificially create the implicit feedback and explicit feedback data following [20]. For the implicit feedback, we use all the rating data by considering whether a user rated an item or not. In other words, the implicit feedback is obtained by binarizing the rating data. For explicit feedback, we randomly pick 10%, 20%, 50%, and 80%, from the rating data and use as explicit feedback."
The same rating set is binarized to form the click matrix P and hence the PPMI matrix S (Eqs. 13-14). Explicit feedback is a random subset of that same rating set, and Section 4.2 splits the rating data 80/20 into train and test. Every test rating (u,i) is therefore a 1-entry in P, so it contributes to the co-occurrence counts #(i) and #(i,j) used in s_ij. Those s_ij entries enter the item-update Eq. (25) via the sum over S_i of s_ij alpha_j and the context update Eq. (26), so item representations are fitted using the withheld ratings. For out-of-matrix test items, all of their click profile comes from test ratings, and the SDAE encoder is additionally trained to match the resulting beta_i through the lambda_beta term in Eq. (23).
full rationale
The model equations themselves are not tautological: Eq. (21) defines an independent PPMI-factorization objective, Eq. (22) defines the rating objective, and the prediction E[r_ui]=theta_u^T beta_i is standard MF; no parameter is defined as the prediction target. The circularity is in the MovieTweetings experiment, where the auxiliary 'click' signal is constructed from all ratings, including the 20% held out for testing. This reduces the paper's central sparse-data claim to a leaky fit on that dataset. Bookcrossing uses genuinely separate click data and is not affected by this construction, but the paper's headline 'particularly for extremely sparse datasets' evidence comes from the contaminated MovieTweetings subsets. Self-citations (e.g., [12], [21]-[26]) are not load-bearing for the derivation, and no uniqueness theorem is imported from the authors' prior work; these do not contribute to the score. Overall: one central empirical claim is partially circular by construction, giving 6.
Assumptions & free parameters
free parameters (3)
- lambda_S =
not specified numerically; varied in Figure 3
- K =
64
- SDAE architecture and regularization hyperparameters =
not specified
assumptions (4)
- domain assumption Items clicked by the same user are similar enough that the co-click PPMI matrix can be factorized into item vectors.
- domain assumption Ratings and PPMI entries are normally distributed around the corresponding latent dot products.
- standard math The middle layer of the SDAE provides a useful item-text representation that can be trained jointly with the rating and click objectives.
- domain assumption Textual descriptions are available and usable for all items in both datasets.
Cite this review
Pith. "Pith review of Boosting the Rating Prediction with Click Data and Textual Contents." pith.science (2026). https://pith.science/paper/X5TX72ED
@misc{pith2026190807749,
author = {Pith},
title = {Pith review of: Boosting the Rating Prediction with Click Data and Textual Contents},
year = {2026},
howpublished = {\url{https://pith.science/paper/X5TX72ED}},
note = {Machine review of arXiv:1908.07749}
}
read the original abstract
Matrix factorization (MF) is one of the most efficient methods for rating predictions. MF learns user and item representations by factorizing the user-item rating matrix. Further, textual contents are integrated to conventional MF to address the cold-start problem. However, the textual contents do not reflect all aspects of the items. In this paper, we propose a model that leverages the information hidden in the item co-click (i.e., items that are often clicked together by a user) into learning item representations. We develop TCMF (Textual Co Matrix Factorization) that learns the user and item representations jointly from the user-item matrix, textual contents and item co-click matrix built from click data. Item co-click information captures the relationships between items which are not captured via textual contents. The experiments on two real-world datasets MovieTweetings, and Bookcrossing) demonstrate that our method outperforms competing methods in terms of rating prediction. Further, we show that the proposed model can learn effective item representations by comparing with state-of-the-art methods in classification task which uses the item representations as input vectors.
Figures
Reference graph
Works this paper leans on
-
[1]
Sarwar, George Karypis, Joseph A
Badrul M. Sarwar, George Karypis, Joseph A. Konstan, and John Reidl. Item-based collaborative filtering recommenda- tion algorithms. In World Wide Web, pages 285–295, 2001
work page 2001
-
[2]
A comprehensive survey of neighborhood-based recommendation methods
Xia Ning, Christian Desrosiers, and George Karypis. A comprehensive survey of neighborhood-based recommendation methods. In Recommender Systems Handbook, pages 37–76. Springer, 2015
work page 2015
-
[3]
Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 426–434. ACM, 2008
work page 2008
-
[4]
Probabilistic matrix factorization
Ruslan Salakhutdinov and Andriy Mnih. Probabilistic matrix factorization. InAdvances in Neural Information Processing Systems, volume 20, 2008
work page 2008
-
[5]
Collaborative filtering with temporal dynamics
Yehuda Koren. Collaborative filtering with temporal dynamics. Commun. ACM, 53(4):89–97, 2010
work page 2010
-
[6]
Chong Wang and David M. Blei. Collaborative topic modeling for recommending scientific articles. In KDD, pages 448–456. ACM, 2011
work page 2011
-
[7]
Collaborative deep learning for recommender systems
Hao Wang, Naiyan Wang, and Dit-Yan Yeung. Collaborative deep learning for recommender systems. In KDD, pages 1235–1244. ACM, 2015
work page 2015
-
[8]
Prem Gopalan, Laurent Charlin, and David M. Blei. Content-based recommendations with poisson factorization. In NIPS, pages 3176–3184, 2014
work page 2014
Show all 32 references
-
[9]
Collaborative variational autoencoder for recommender systems
Xiaopeng Li and James She. Collaborative variational autoencoder for recommender systems. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD ’17, pages 305–314, New York, NY, USA, 2017. ACM
2017
-
[10]
Corrado, and Jeffrey Dean
Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111–3119, 2013
2013
-
[11]
Glove: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532–1543, 2014
2014
-
[12]
A probabilistic model for the cold-start problem in rating prediction using click data
ThaiBinh Nguyen and Atsuhiro Takasu. A probabilistic model for the cold-start problem in rating prediction using click data. In International Conference on Neural Information Processing , pages 196–205. Springer, Cham, 2017. 14 ThaiBinh Nguyen and Atsuhiro Takasu
2017
-
[13]
Efficient estimation of word representations in vector space
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013
2013 arXiv
-
[14]
Distributed representations of sentences and documents
Quoc V Le and Tomas Mikolov. Distributed representations of sentences and documents. In ICML, volume 14, pages 1188–1196, 2014
2014
-
[15]
A generative word embedding model and its low rank positive semidefinite solution
Shaohua Li, Jun Zhu, and Chunyan Miao. A generative word embedding model and its low rank positive semidefinite solution. In EMNLP, pages 1599–1609. The Association for Computational Linguistics, 2015
2015
-
[16]
Neural word embedding as implicit matrix factorization
Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In Advances in Neural Information Processing Systems 27 , pages 2177–2185. Curran Associates, Inc., 2014
2014
-
[17]
K. W. Church and P. Hanks. Word association norms, mutual information, and lexicography.Computational Linguistics, 1(16):22–29, 1990
1990
-
[18]
Extracting semantic representations from word co-occurrence statistics: A computa- tional study
John Bullinaria and Joseph Levy. Extracting semantic representations from word co-occurrence statistics: A computa- tional study. Behavior Research Methods, 39(3):510–526, 2007
2007
-
[19]
Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on , pages 263–272. IEEE, 2008
2008
-
[20]
Bell and Yehuda Koren
Robert M. Bell and Yehuda Koren. Scalable collaborative filtering with jointly derived neighborhood interpolation weights. In ICDM, pages 43–52. IEEE Computer Society, 2007
2007
-
[21]
Npe: Neural personalized embedding for collaborative filtering
ThaiBinh Nguyen and Atsuhiro Takasu. Npe: Neural personalized embedding for collaborative filtering. In IJCAI 2018, pages 1583–1589, 2018
2018
-
[22]
City recommender system based on a latent topic model
Thai-Binh NGUYEN, Kenro AIHARA, and Atsuhiro TAKASU. City recommender system based on a latent topic model. IEICE technical report, 115(381):95–99, 2015
2015
-
[23]
Collaborative item embedding model for implicit feedback data
ThaiBinh Nguyen, Kenro Aihara, and Atsuhiro Takasu. Collaborative item embedding model for implicit feedback data. In International Conference on Web Engineering , pages 336–348. Springer, Cham, 2017
2017
-
[24]
A hierarchical bayesian factorization model for implicit and explicit feedback data
ThaiBinh Nguyen and Atsuhiro Takasu. A hierarchical bayesian factorization model for implicit and explicit feedback data. In International Conference on Advanced Data Mining and Applications , pages 104–118. Springer, 2017
2017
-
[25]
Kanjirathinkal, and Mohan Kankanhalli
Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose C. Kanjirathinkal, and Mohan Kankanhalli. Mmalfm: Explainable recommendation by leveraging reviews and images. ACM Transaction on Information Systems (TOIS) , 37(2):16:1–16:28, January 2019
2019
-
[26]
Learning representations from product titles for modeling shopping transac- tions
ThaiBinh Nguyen and Atsuhiro Takasu. Learning representations from product titles for modeling shopping transac- tions. In The AAAI-19 Workshop on Recommender Systems and Natural Language Processing (RecNLP) , 2019
2019
-
[27]
Vbpr: Visual bayesian personalized ranking from implicit feedback
Ruining He and Julian McAuley. Vbpr: Visual bayesian personalized ranking from implicit feedback. In Dale Schuurmans and Michael P. Wellman, editors, AAAI, pages 144–150. AAAI Press, 2016
2016
-
[28]
Unifying explicit and implicit feedback for collaborative filtering
Nathan Nan Liu, Evan Wei Xiang, Min Zhao, and Qiang Yang 0001. Unifying explicit and implicit feedback for collaborative filtering. In CIKM, pages 1445–1448. ACM, 2010
2010
-
[29]
Expectation-maximization collaborative filtering with explicit and implicit feedback
Bin Wang, Mohammadreza Rahimi, Dequan Zhou, and Xin Wang. Expectation-maximization collaborative filtering with explicit and implicit feedback. In PAKDD (1), volume 7301 of Lecture Notes in Computer Science , pages 604–616. Springer, 2012
2012
-
[30]
Transfer to rank for heterogeneous one-class collaborative filtering
Weike Pan, Qiang Yang, Wanling Cai, Yaofeng Chen, Qing Zhang, Xiaogang Peng, and Zhong Ming. Transfer to rank for heterogeneous one-class collaborative filtering. ACM Transaction on Information Systems (TOIS) , 37(1):10:1–10:20, January 2019
2019
-
[31]
Item2vec: Neural item embedding for collaborative filtering
Oren Barkan and Noam Koenigstein. Item2vec: Neural item embedding for collaborative filtering. In RecSys Posters, volume 1688 of CEUR Workshop Proceedings. CEUR-WS.org, 2016
2016
-
[32]
Dawen Liang, Jaan Altosaar, Laurent Charlin, and David M. Blei. Factorization meets the item embedding: Regularizing matrix factorization with item co-occurrence. In RecSys, pages 59–66. ACM, 2016
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.