Pith. sign in

REVIEW 3 major objections 6 minor 32 references

Boosting the Rating Prediction with Click Data and Textual Contents

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read This paper proposes TCF, a joint model of ratings, textual contents, and item co-click data, and reports lower prediction error than four baselines on sparse rating datasets.

desk verdict A plausible model combination with a train/test leak that undermines the headline empirical claim. read the letter →

arxiv 1908.07749 v2 pith:X5TX72ED submitted 2019-08-21 cs.IR cs.LG

classification cs.IRcs.LG
keywords recommendersystemsratingpredictionmatrixfactorizationimplicitfeedbackitemembeddingstackeddenoisingautoencodercold-startco-clickdata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rating prediction usually suffers when users rate few items, and text descriptions only capture part of what makes items similar. The paper proposes TCF (Textual Co-Factorization), a model that learns item representations from three signals at once: the rating matrix, the item's text, and a co-click matrix built from implicit feedback, meaning items clicked together by the same user. TCF shares one item vector across these signals, so an item with few or no ratings can still get a useful representation from its text and its click neighborhood. The experiments on MovieTweetings and Bookcrossing report lower root-mean-square error (RMSE) than plain matrix factorization and three text-aware collaborative baselines, with the largest gains on the sparsest data and for items that have clicks but no ratings.

What carries the argument

The machinery is a shared latent item vector $\boldsymbol{\beta}_i$ that is pinned to three generative processes. Textually, $\boldsymbol{\beta}_i$ is drawn from the middle layer of an SDAE, so the bag-of-words description of item $i$ shapes it. For clicks, the same vector appears in a PPMI matrix factorization: for item pairs $(i,j)$, the model sets $\boldsymbol{\beta}_i^\top \boldsymbol{\alpha}_j \approx s_{ij}$, where $s_{ij} = \max\{\log \#(i,j)|D|/(\#(i)\#(j)), 0\}$; the context vectors $\boldsymbol{\alpha}_j$ absorb the role of the neighbor. For ratings, $\boldsymbol{\beta}_i$ is multiplied by a user vector $\boldsymbol{\theta}_u$ to predict $r_{ui}$. Alternating updates for $\boldsymbol{\theta}$, $\boldsymbol{\beta}$, $\boldsymbol{\alpha}$, and the SDAE weights minimize one joint objective (Eq. 23), with $\lambda_S$ balancing click versus rating information.

What would settle it

Re-run the MovieTweetings comparison with a click matrix built only from a pre-split click log or from the training portion of the binarized ratings, excluding test users' clicks; if TCF's RMSE advantage over CVAE shrinks or vanishes, the reported gain comes from the click matrix leaking test ratings, while if the advantage persists, the co-click signal is genuinely independent.

Watch

Extended reading notes

Core claim

On the paper's own terms, TCF establishes that item co-click information and textual contents should be learned jointly with the rating matrix rather than used as separate features or post-hoc regularizers. The generative model draws each item's feature vector from the middle layer of a stacked denoising autoencoder on bag-of-words text, then uses that same vector to predict both ratings and the entries of a positive pointwise mutual information (PPMI) matrix formed from co-clicks. The central identity is that the log ratio $p(i|j)/p(i)$ equals $\boldsymbol{\beta}_i^\top \boldsymbol{\alpha}_j$, meaning the pointwise mutual information of two items is the inner product of an item feature vector and an item context vector; TCF factorizes the PPMI version of this matrix together with the rating matrix. The authors report that this joint factorization beats text-only hybrids and pure matrix factorization in in-matrix prediction, and that it predicts out-of-matrix ratings for items with clicks but no prior ratings better than the text-only baselines.

Load-bearing premise

The load-bearing premise is that the click matrix is an independent auxiliary signal; on MovieTweetings it is constructed by turning the ratings into click indicators before splitting the data, so the co-click matrix can carry information about test ratings.

Editorial extensions

If this is right

  • Items that have never been rated but have been clicked can still get predicted ratings, because their item vector is formed by the text encoder and the co-click factorization.
  • The benefit over text-only baselines grows as the rating matrix gets sparser, so implicit feedback is most valuable precisely when explicit ratings are scarce.
  • In out-of-matrix prediction, where test items have no ratings in the training set, TCF reports lower RMSE than CTR, CDL, and CVAE, meaning the click signal helps cold-start items.
  • The learned item vectors also improve a downstream classification task compared with embeddings learned by other methods, so the co-click signal improves representational quality, not just rating loss.
  • The model depends on a balance parameter $\lambda_S$: both too little and too much click influence degrade accuracy, so the two signals must be weighted against each other.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension not tested in the paper is to replace the user-based context with session-based contexts, so items clicked in the same browsing session form the co-click matrix; this would generate a different item-item graph and may capture relationships that user histories miss.
  • The same joint-loss template could be reused with a different text encoder, such as a transformer-based encoder, in place of the SDAE; the rating and co-click terms would remain unchanged, making it straightforward to test whether better text representations compound the gain.
  • The classification results suggest the item vectors are useful beyond RMSE; one could test whether the co-click-augmented embeddings also improve retrieval or ranking metrics such as precision at k or normalized discounted cumulative gain.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes TCF, a probabilistic matrix-factorization model that jointly learns item representations from explicit ratings, item textual content via a stacked denoising autoencoder (SDAE), and item co-click data via a PPMI item-item matrix. The generative process places the item feature vector beta_i near the SDAE encoder output, draws PPMI entries from beta_i^T alpha_j, and draws ratings from theta_u^T beta_i. The learning objective in Eq. (23) combines the rating likelihood, the PPMI factorization, and the SDAE reconstruction terms, with coordinate updates for theta, beta, alpha, and the SDAE weights in Eqs. (24)-(28). Experiments on MovieTweetings and Bookcrossing compare in-matrix and out-of-matrix RMSE with PMF, CTR, CDL, and CVAE across sparsity levels MT-10/20/50/80. The paper claims that TCF significantly outperforms the baselines, especially on extremely sparse data, and that the click signal helps cold-start rating prediction.

Significance. The proposed combination of an SDAE text model, a rating matrix factorization, and a PPMI click factorization is a plausible and coherent extension of CDL. The objective is not tautological: the PPMI factorization has its own auxiliary signal and is not derived from the rating target. The coordinate update derivations and the complexity analysis are useful contributions. However, the empirical evidence for the central claim is compromised on MovieTweetings because the click matrix is built from all rating data before the train/test split. This makes the reported sparse-data gains potentially artifacts of label leakage. The significance of the paper therefore rests on whether the experiments can be rerun with a leakage-free protocol; with such a rerun, the model may well be a meaningful contribution to cold-start rating prediction.

major comments (3)
  1. [Section 4.1-4.2, Eq. (23)] The MovieTweetings implicit feedback matrix is constructed by binarizing all rating data before the split: Section 4.1 states that 'the implicit feedback is obtained by binarizing the rating data' and that 'we use all the rating data by considering whether a user rated an item or not,' while Section 4.2 then splits the rating data into 80% training and 20% test. The PPMI matrix S used in Eq. (23) therefore encodes the test ratings. For in-matrix prediction, every test pair (u,i) contributes to the co-click statistics of item i; for out-of-matrix prediction, the test item's click vector is derived exclusively from ratings that later appear in the test set. Because the beta_i update in Eq. (25) and the SDAE gradients in Eq. (27) depend on S, test outcomes leak into the training signal. The reported gains on MT-10 and MT-20 in Fig. 2 may therefore be inflated. The experiments must be rerun with the implicit feedback constructed from the training split only, and the revised RMSE values must be reported.
  2. [Section 4.4, Tables 4a/4b, Fig. 2] All comparisons report a single RMSE value for each method and dataset, with no standard deviations, confidence intervals, or significance tests. Since the explicit feedback subsets are randomly sampled (Section 4.1) and the validation split is random, the reported differences, particularly the gaps between TCF and CVAE on Bookcrossing, could be within random variation. Please report means and standard deviations over multiple random splits, or otherwise establish that the improvements are statistically significant.
  3. [Section 4.2, out-of-matrix prediction] The out-of-matrix protocol says that test items have no ratings in the training set, but the click matrix for those same items is built from the full rating set. A genuinely cold-start item should also have no rating-derived implicit signal in the training click matrix; otherwise the out-of-matrix evaluation does not measure cold-start performance. The authors should define out-of-matrix prediction using only training-derived clicks, text, and ratings, and rerun the out-of-matrix experiments accordingly.
minor comments (6)
  1. [Abstract and Section 1] The model name is inconsistent: the abstract uses 'TCMF' while the rest of the paper uses 'TCF.' Please unify the terminology.
  2. [Section 3.3, Eqs. (22) and (23)] Eq. (22) writes r_{uj} on the left but beta_i on the right, and the first term of Eq. (23) uses theta_i^T beta_i instead of theta_u^T beta_i. These typos make the derivation harder to follow.
  3. [Fig. 2] The legend of Fig. 2a labels the proposed method as 'CTF' instead of 'TCF.' Please correct the label.
  4. [Section 1] The introduction says the experiments use 'three public datasets,' but Section 4.1 describes only two datasets. Please adjust the text to match the actual experiments.
  5. [Table 3] The density values in Table 3 are inconsistent with the counts in Table 2: with 630,000 ratings over 50,000 users and 28,000 items, taking 80% of the ratings yields approximately 0.036% density, not 1.9348% as reported. Please check the computation and clarify what the table reports.
  6. [Abstract] The abstract promises a comparison on a classification task using the learned item representations, but the full text contains no classification experiments. Either add the experiments or remove that claim.

Circularity Check

1 steps flagged · score 6.0 of 10

MovieTweetings evaluation is partly circular: the 'click matrix' is binarized from all ratings, so the PPMI item vectors are fitted to the very test ratings whose prediction is claimed as a gain.

  1. fitted input called prediction [Section 4.1 (MovieTweetings construction) with Section 4.2 split and Eq. (25)-(26)]
    "Since MovieTweetings datasets contain only explicit feedback, we artificially create the implicit feedback and explicit feedback data following [20]. For the implicit feedback, we use all the rating data by considering whether a user rated an item or not. In other words, the implicit feedback is obtained by binarizing the rating data. For explicit feedback, we randomly pick 10%, 20%, 50%, and 80%, from the rating data and use as explicit feedback."

    The same rating set is binarized to form the click matrix P and hence the PPMI matrix S (Eqs. 13-14). Explicit feedback is a random subset of that same rating set, and Section 4.2 splits the rating data 80/20 into train and test. Every test rating (u,i) is therefore a 1-entry in P, so it contributes to the co-occurrence counts #(i) and #(i,j) used in s_ij. Those s_ij entries enter the item-update Eq. (25) via the sum over S_i of s_ij alpha_j and the context update Eq. (26), so item representations are fitted using the withheld ratings. For out-of-matrix test items, all of their click profile comes from test ratings, and the SDAE encoder is additionally trained to match the resulting beta_i through the lambda_beta term in Eq. (23).

full rationale

The model equations themselves are not tautological: Eq. (21) defines an independent PPMI-factorization objective, Eq. (22) defines the rating objective, and the prediction E[r_ui]=theta_u^T beta_i is standard MF; no parameter is defined as the prediction target. The circularity is in the MovieTweetings experiment, where the auxiliary 'click' signal is constructed from all ratings, including the 20% held out for testing. This reduces the paper's central sparse-data claim to a leaky fit on that dataset. Bookcrossing uses genuinely separate click data and is not affected by this construction, but the paper's headline 'particularly for extremely sparse datasets' evidence comes from the contaminated MovieTweetings subsets. Self-citations (e.g., [12], [21]-[26]) are not load-bearing for the derivation, and no uniqueness theorem is imported from the authors' prior work; these do not contribute to the score. Overall: one central empirical claim is partially circular by construction, giving 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard probabilistic MF assumptions plus the domain assumption that co-click similarity transfers to rating prediction. No new physical or conceptual entities are introduced; the model reuses latent user, item, and context vectors. The main free choice is lambda_S, the click-rating balance, which the paper itself shows to be influential.

free parameters (3)
  • lambda_S = not specified numerically; varied in Figure 3
    Balance parameter between rating loss and click loss in Eq. 23. The paper shows RMSE varies strongly with it, so the reported performance depends on this choice.
  • K = 64
    Latent dimensionality is fixed to 64 in Section 4.4 for all methods. It is a chosen hyperparameter, not derived.
  • SDAE architecture and regularization hyperparameters = not specified
    Layer sizes, vocabulary size, and the regularization coefficients lambda_theta, lambda_beta, lambda_alpha, lambda_X, and lambda_W are tuned via a validation set but their final values are not reported.
assumptions (4)
  • domain assumption Items clicked by the same user are similar enough that the co-click PPMI matrix can be factorized into item vectors.
    Stated in Section 3.1 as the word-embedding analogy. If co-clicks do not carry rating-relevant similarity, the click term adds noise.
  • domain assumption Ratings and PPMI entries are normally distributed around the corresponding latent dot products.
    Equations (21) and (22) in the generative process. This is a standard probabilistic MF assumption, not proven on either dataset.
  • standard math The middle layer of the SDAE provides a useful item-text representation that can be trained jointly with the rating and click objectives.
    Borrowed from CDL (Section 2.3). It assumes the autoencoder's bottleneck captures semantics relevant to ratings.
  • domain assumption Textual descriptions are available and usable for all items in both datasets.
    The authors crawled movie plot summaries from IMDb and book descriptions from lookupbyisbn.com, which may be partial or noisy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Boosting the Rating Prediction with Click Data and Textual Contents." pith.science (2026). https://pith.science/paper/X5TX72ED

@misc{pith2026190807749,
  author       = {Pith},
  title        = {Pith review of: Boosting the Rating Prediction with Click Data and Textual Contents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X5TX72ED}},
  note         = {Machine review of arXiv:1908.07749}
}
read the original abstract

Matrix factorization (MF) is one of the most efficient methods for rating predictions. MF learns user and item representations by factorizing the user-item rating matrix. Further, textual contents are integrated to conventional MF to address the cold-start problem. However, the textual contents do not reflect all aspects of the items. In this paper, we propose a model that leverages the information hidden in the item co-click (i.e., items that are often clicked together by a user) into learning item representations. We develop TCMF (Textual Co Matrix Factorization) that learns the user and item representations jointly from the user-item matrix, textual contents and item co-click matrix built from click data. Item co-click information captures the relationships between items which are not captured via textual contents. The experiments on two real-world datasets MovieTweetings, and Bookcrossing) demonstrate that our method outperforms competing methods in terms of rating prediction. Further, we show that the proposed model can learn effective item representations by comparing with state-of-the-art methods in classification task which uses the item representations as input vectors.

Figures

Figures reproduced from arXiv: 1908.07749 by the authors.

Figure 2
Figure 2. A 2-layer SDAE with L =4. SDAE solves the following optimization problem: min {Wl},{bl} kXc ￾ XLk2 F + ￾ X l kWlk2 F , where ￾ is a regularization parameter and k · kF denotes the Frobenius norm. 3.2 Generalized Bayesian SDAE If we assume that both the clean input Xc and the cor￾rupted input X0 are observed, similar to [4, 19, 3, 7], we can define the following generative process: 1. For each layer l of the SDAE net… view at source ↗
Figure 2
Figure 2. The RMSE for the data of different levels of sparsity [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Test RMSE of in-matrix prediction task on different subsets of MovieTweetings dataset corresponding [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 30 canonical work pages

  1. [1]

    Sarwar, George Karypis, Joseph A

    Badrul M. Sarwar, George Karypis, Joseph A. Konstan, and John Reidl. Item-based collaborative filtering recommenda- tion algorithms. In World Wide Web, pages 285–295, 2001

  2. [2]

    A comprehensive survey of neighborhood-based recommendation methods

    Xia Ning, Christian Desrosiers, and George Karypis. A comprehensive survey of neighborhood-based recommendation methods. In Recommender Systems Handbook, pages 37–76. Springer, 2015

  3. [3]

    Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 426–434. ACM, 2008

  4. [4]

    Probabilistic matrix factorization

    Ruslan Salakhutdinov and Andriy Mnih. Probabilistic matrix factorization. InAdvances in Neural Information Processing Systems, volume 20, 2008

  5. [5]

    Collaborative filtering with temporal dynamics

    Yehuda Koren. Collaborative filtering with temporal dynamics. Commun. ACM, 53(4):89–97, 2010

  6. [6]

    Chong Wang and David M. Blei. Collaborative topic modeling for recommending scientific articles. In KDD, pages 448–456. ACM, 2011

  7. [7]

    Collaborative deep learning for recommender systems

    Hao Wang, Naiyan Wang, and Dit-Yan Yeung. Collaborative deep learning for recommender systems. In KDD, pages 1235–1244. ACM, 2015

  8. [8]

    Prem Gopalan, Laurent Charlin, and David M. Blei. Content-based recommendations with poisson factorization. In NIPS, pages 3176–3184, 2014

Show all 32 references
  1. [9]

    Collaborative variational autoencoder for recommender systems

    Xiaopeng Li and James She. Collaborative variational autoencoder for recommender systems. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD ’17, pages 305–314, New York, NY, USA, 2017. ACM

  2. [10]

    Corrado, and Jeffrey Dean

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111–3119, 2013

  3. [11]

    Glove: Global vectors for word representation

    Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532–1543, 2014

  4. [12]

    A probabilistic model for the cold-start problem in rating prediction using click data

    ThaiBinh Nguyen and Atsuhiro Takasu. A probabilistic model for the cold-start problem in rating prediction using click data. In International Conference on Neural Information Processing , pages 196–205. Springer, Cham, 2017. 14 ThaiBinh Nguyen and Atsuhiro Takasu

  5. [13]

    Efficient estimation of word representations in vector space

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013

  6. [14]

    Distributed representations of sentences and documents

    Quoc V Le and Tomas Mikolov. Distributed representations of sentences and documents. In ICML, volume 14, pages 1188–1196, 2014

  7. [15]

    A generative word embedding model and its low rank positive semidefinite solution

    Shaohua Li, Jun Zhu, and Chunyan Miao. A generative word embedding model and its low rank positive semidefinite solution. In EMNLP, pages 1599–1609. The Association for Computational Linguistics, 2015

  8. [16]

    Neural word embedding as implicit matrix factorization

    Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In Advances in Neural Information Processing Systems 27 , pages 2177–2185. Curran Associates, Inc., 2014

  9. [17]

    K. W. Church and P. Hanks. Word association norms, mutual information, and lexicography.Computational Linguistics, 1(16):22–29, 1990

  10. [18]

    Extracting semantic representations from word co-occurrence statistics: A computa- tional study

    John Bullinaria and Joseph Levy. Extracting semantic representations from word co-occurrence statistics: A computa- tional study. Behavior Research Methods, 39(3):510–526, 2007

  11. [19]

    Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on , pages 263–272. IEEE, 2008

  12. [20]

    Bell and Yehuda Koren

    Robert M. Bell and Yehuda Koren. Scalable collaborative filtering with jointly derived neighborhood interpolation weights. In ICDM, pages 43–52. IEEE Computer Society, 2007

  13. [21]

    Npe: Neural personalized embedding for collaborative filtering

    ThaiBinh Nguyen and Atsuhiro Takasu. Npe: Neural personalized embedding for collaborative filtering. In IJCAI 2018, pages 1583–1589, 2018

  14. [22]

    City recommender system based on a latent topic model

    Thai-Binh NGUYEN, Kenro AIHARA, and Atsuhiro TAKASU. City recommender system based on a latent topic model. IEICE technical report, 115(381):95–99, 2015

  15. [23]

    Collaborative item embedding model for implicit feedback data

    ThaiBinh Nguyen, Kenro Aihara, and Atsuhiro Takasu. Collaborative item embedding model for implicit feedback data. In International Conference on Web Engineering , pages 336–348. Springer, Cham, 2017

  16. [24]

    A hierarchical bayesian factorization model for implicit and explicit feedback data

    ThaiBinh Nguyen and Atsuhiro Takasu. A hierarchical bayesian factorization model for implicit and explicit feedback data. In International Conference on Advanced Data Mining and Applications , pages 104–118. Springer, 2017

  17. [25]

    Kanjirathinkal, and Mohan Kankanhalli

    Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose C. Kanjirathinkal, and Mohan Kankanhalli. Mmalfm: Explainable recommendation by leveraging reviews and images. ACM Transaction on Information Systems (TOIS) , 37(2):16:1–16:28, January 2019

  18. [26]

    Learning representations from product titles for modeling shopping transac- tions

    ThaiBinh Nguyen and Atsuhiro Takasu. Learning representations from product titles for modeling shopping transac- tions. In The AAAI-19 Workshop on Recommender Systems and Natural Language Processing (RecNLP) , 2019

  19. [27]

    Vbpr: Visual bayesian personalized ranking from implicit feedback

    Ruining He and Julian McAuley. Vbpr: Visual bayesian personalized ranking from implicit feedback. In Dale Schuurmans and Michael P. Wellman, editors, AAAI, pages 144–150. AAAI Press, 2016

  20. [28]

    Unifying explicit and implicit feedback for collaborative filtering

    Nathan Nan Liu, Evan Wei Xiang, Min Zhao, and Qiang Yang 0001. Unifying explicit and implicit feedback for collaborative filtering. In CIKM, pages 1445–1448. ACM, 2010

  21. [29]

    Expectation-maximization collaborative filtering with explicit and implicit feedback

    Bin Wang, Mohammadreza Rahimi, Dequan Zhou, and Xin Wang. Expectation-maximization collaborative filtering with explicit and implicit feedback. In PAKDD (1), volume 7301 of Lecture Notes in Computer Science , pages 604–616. Springer, 2012

  22. [30]

    Transfer to rank for heterogeneous one-class collaborative filtering

    Weike Pan, Qiang Yang, Wanling Cai, Yaofeng Chen, Qing Zhang, Xiaogang Peng, and Zhong Ming. Transfer to rank for heterogeneous one-class collaborative filtering. ACM Transaction on Information Systems (TOIS) , 37(1):10:1–10:20, January 2019

  23. [31]

    Item2vec: Neural item embedding for collaborative filtering

    Oren Barkan and Noam Koenigstein. Item2vec: Neural item embedding for collaborative filtering. In RecSys Posters, volume 1688 of CEUR Workshop Proceedings. CEUR-WS.org, 2016

  24. [32]

    Dawen Liang, Jaan Altosaar, Laurent Charlin, and David M. Blei. Factorization meets the item embedding: Regularizing matrix factorization with item co-occurrence. In RecSys, pages 59–66. ACM, 2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.