Modeling Embedding Dimension Correlations via Convolutional Neural Collaborative Filtering
Pith reviewed 2026-05-25 15:02 UTC · model grok-4.3
The pith
ConvNCF models pairwise and high-order correlations among embedding dimensions by applying outer product to user and item embeddings followed by a convolutional network.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ConvNCF is a neural collaborative filtering framework that applies the outer product operation to user and item embeddings to explicitly model pairwise correlations between embedding dimensions and then employs a convolutional neural network to learn high-order correlations among those dimensions, resulting in improved modeling of user-item affinities.
What carries the argument
Outer product of user and item embeddings followed by a convolutional neural network to extract correlations.
If this is right
- The model outperforms several competitive CF methods on two real-world datasets.
- Modeling embedding dimension correlations improves effectiveness in collaborative filtering.
- Three different instantiations using varied user inputs all benefit from the outer-product-plus-CNN design.
- The framework provides a general way to capture dimension-wise interactions in neural recommender models.
Where Pith is reading between the lines
- The same outer-product-plus-CNN pattern could be tested in other embedding-based tasks such as knowledge-graph link prediction.
- It raises whether MLP-based interaction functions in neural CF are limited precisely because they do not treat dimensions as having explicit pairwise structure.
- Performance differences might vary with interaction sparsity or with the way embeddings are initialized.
Load-bearing premise
That explicitly modeling pairwise and high-order correlations among embedding dimensions via outer product and CNN will produce meaningfully better user-item affinity predictions than existing neural CF architectures on real-world data.
What would settle it
An experiment in which a standard neural CF baseline without the outer-product or CNN layers matches or exceeds ConvNCF accuracy on the same two datasets while controlling for embedding size and training procedure would falsify the claim.
Figures
read the original abstract
As the core of recommender system, collaborative filtering (CF) models the affinity between a user and an item from historical user-item interactions, such as clicks, purchases, and so on. Benefited from the strong representation power, neural networks have recently revolutionized the recommendation research, setting up a new standard for CF. However, existing neural recommender models do not explicitly consider the correlations among embedding dimensions, making them less effective in modeling the interaction function between users and items. In this work, we emphasize on modeling the correlations among embedding dimensions in neural networks to pursue higher effectiveness for CF. We propose a novel and general neural collaborative filtering framework, namely ConvNCF, which is featured with two designs: 1) applying outer product on user embedding and item embedding to explicitly model the pairwise correlations between embedding dimensions, and 2) employing convolutional neural network above the outer product to learn the high-order correlations among embedding dimensions. To justify our proposal, we present three instantiations of ConvNCF by using different inputs to represent a user and conduct experiments on two real-world datasets. Extensive results verify the utility of modeling embedding dimension correlations with ConvNCF, which outperforms several competitive CF methods.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes ConvNCF, a neural collaborative filtering framework that applies an outer product to user and item embeddings to explicitly capture pairwise correlations among embedding dimensions and stacks a CNN on the resulting interaction map to learn high-order correlations. Three instantiations are presented (differing in user representation) and evaluated on two real-world datasets, where ConvNCF is reported to outperform several competitive neural and non-neural CF baselines.
Significance. If the performance gains are shown to arise specifically from the explicit dimension-correlation mechanism rather than from increased model capacity or the spatial inductive bias of the CNN, the work would supply a concrete architectural alternative to MLP-based interaction functions in neural CF and could encourage further study of dimension-wise interaction maps.
major comments (2)
- [Experiments] The central experimental claim (outperformance via explicit pairwise and high-order dimension correlation modeling) rests on comparisons to baselines, yet the manuscript provides no ablation that holds parameter count fixed while replacing the outer-product + CNN structure with an equivalent-capacity MLP or a non-convolutional aggregator on the same 2-D map. Without this control, the reported gains cannot be attributed to the claimed correlation mechanism rather than expressivity.
- [Introduction / §3] The abstract and motivation assert that existing neural CF models “do not explicitly consider the correlations among embedding dimensions,” but the manuscript does not supply a formal argument or controlled comparison demonstrating that standard MLP or factorization-machine interaction functions are incapable of learning such correlations when given sufficient capacity.
minor comments (2)
- [Experiments] Table captions and axis labels should explicitly state whether reported metrics are averaged over multiple random seeds and whether hyper-parameter search was performed with the same budget for all methods.
- [§3] The three instantiations of ConvNCF are described at a high level; a single diagram or pseudocode block showing the precise tensor shapes after the outer product and the CNN filter configuration would improve reproducibility.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback. We address the two major comments point-by-point below, indicating where revisions will be made to strengthen the manuscript.
read point-by-point responses
-
Referee: [Experiments] The central experimental claim (outperformance via explicit pairwise and high-order dimension correlation modeling) rests on comparisons to baselines, yet the manuscript provides no ablation that holds parameter count fixed while replacing the outer-product + CNN structure with an equivalent-capacity MLP or a non-convolutional aggregator on the same 2-D map. Without this control, the reported gains cannot be attributed to the claimed correlation mechanism rather than expressivity.
Authors: We agree this is a valid concern. The current experiments compare against published baselines but do not include capacity-controlled ablations against an MLP or non-convolutional aggregator on the interaction map. In the revision we will add such experiments (with parameter counts matched via hidden-layer sizing) to better isolate the contribution of the outer-product + CNN design. revision: yes
-
Referee: [Introduction / §3] The abstract and motivation assert that existing neural CF models “do not explicitly consider the correlations among embedding dimensions,” but the manuscript does not supply a formal argument or controlled comparison demonstrating that standard MLP or factorization-machine interaction functions are incapable of learning such correlations when given sufficient capacity.
Authors: The manuscript's wording emphasizes the lack of explicit modeling (via outer product) rather than claiming that MLPs or FMs are theoretically incapable of capturing dimension correlations implicitly. Universal approximation results imply that sufficiently large MLPs can represent such functions. Our contribution is the explicit, structured construction. We will revise the abstract and introduction to replace “do not explicitly consider” with clearer language distinguishing explicit versus implicit modeling and will not add a formal impossibility proof. revision: partial
Circularity Check
No circularity: new architecture defined independently and validated on external data
full rationale
The paper proposes ConvNCF as a new neural CF architecture using outer product on embeddings followed by CNN layers. This is an explicit design choice, not derived from prior equations that reduce the claimed benefit to a fitted parameter or self-citation. The justification rests on empirical results against baselines on two datasets, which are independent external benchmarks rather than internal fits. No self-definitional loops, no predictions that are statistically forced by construction, and no load-bearing self-citations appear in the provided text. The derivation chain is self-contained as an architectural proposal.
Axiom & Free-Parameter Ledger
free parameters (2)
- embedding dimension
- CNN filter sizes and depths
axioms (2)
- standard math Outer product of two vectors produces a matrix whose entries capture all pairwise products of dimensions.
- domain assumption Convolutional layers can extract higher-order patterns from the outer-product matrix.
Reference graph
Works this paper leans on
-
[1]
Ting Bai, Ji-Rong Wen, Jun Zhang, and Wayne Xin Zhao. 2017. A Neural Collaborative Filtering Model with Interaction- based Neighborhood. In CIKM. 1979–1982
work page 2017
-
[2]
Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. 2017. A Generic Coordinate Descent Framework for Learning from Implicit Feedback. In WWW. 1341–1350
work page 2017
-
[3]
James Bennett, Stan Lanning, et al. 2007. The netflix prize. In Proceedings of KDD cup and workshop , Vol. 2007. New York, NY, USA, 35
work page 2007
-
[4]
Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, and Ed H. Chi. 2018. Latent Cross: Making Use of Context in Recurrent Recommender Systems. In WSDM. 46–54
work page 2018
-
[5]
Da Cao, Xiangnan He, Liqiang Nie, Xiaochi Wei, Xia Hu, Shunxiang Wu, and Tat-Seng Chua. 2017. Cross-Platform App Recommendation by Jointly Modeling Ratings and Texts. ACM Transactions on Information Systems 35, 4 (July 2017), 37:1–37:27
work page 2017
-
[6]
Da Cao, Liqiang Nie, Xiangnan He, Xiaochi Wei, Shunzhi Zhu, and Tat-Seng Chua. 2017. Embedding factorization models for jointly recommending items and user generated lists. In SIGIR. ACM, 585–594
work page 2017
-
[7]
Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention. In SIGIR. 335–344
work page 2017
-
[8]
Tianqi Chen, Weinan Zhang, Qiuxia Lu, Kailong Chen, Zhao Zheng, and Yong Yu. 2012. SVDFeature: a toolkit for feature-based collaborative filtering. Journal of Machine Learning Research 13, Dec (2012), 3619–3622
work page 2012
-
[9]
Xu Chen, Yongfeng Zhang, Hongteng Xu, Zheng Qin, and Hongyuan Zha. 2018. Adversarial Distillation for Efficient Recommendation with External Knowledge. ACM Transactions on Information Systems (TOIS) 37, 1 (2018), 12
work page 2018
-
[10]
Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose C Kanjirathinkal, and Mohan Kankanhalli. 2019. MMALFM: Explainable recommendation by leveraging reviews and images. ACM Transactions on Information Systems (TOIS) 37, 2 (2019), 16
work page 2019
-
[11]
Zhiyong Cheng, Ying Ding, Xiangnan He, Lei Zhu, Xuemeng Song, and Mohan S Kankanhalli. 2018. Aˆ 3NCF: An Adaptive Aspect Attention Model for Rating Prediction.. In IJCAI. 3748–3754
work page 2018
-
[12]
Zhiyong Cheng, Jialie Shen, Lei Zhu, Mohan S Kankanhalli, and Liqiang Nie. 2017. Exploiting Music Play Sequence for Music Recommendation.. In IJCAI. 3654–3660
work page 2017
-
[13]
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. In RecSys. 191–198
work page 2016
-
[14]
George Cybenko. 1989. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems 2, 4 (1989), 303–314
work page 1989
-
[15]
Shuiguang Deng, Longtao Huang, Guandong Xu, Xindong Wu, and Zhaohui Wu. 2017. On deep learning for trust- aware recommendations in social networks. IEEE transactions on neural networks and learning systems 28, 5 (2017), 1164–1177
work page 2017
-
[16]
Jingtao Ding, Fuli Feng, Xiangnan He, Guanghui Yu, Yong Li, and Depeng Jin. 2018. An Improved Sampler for Bayesian Personalized Ranking by Leveraging View Data. In WWW. 13–14
work page 2018
-
[17]
Xiangnan He Xiang Wang Cheng Luo Yiqun Liu Feng, Fuli and Tat-Seng Chua. 2019. Temporal Relational Ranking for Stock Prediction. ACM Transactions on Information Systems (TOIS) 37, 2 (2019), 27
work page 2019
-
[18]
Matt W Gardner and SR Dorling. 1998. Artificial neural networks (the multilayer perceptron)-a review of applications in the atmospheric sciences. Atmospheric environment 32, 14-15 (1998), 2627–2636
work page 1998
-
[19]
Carlos A Gomez-Uribe and Neil Hunt. 2016. The netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems (TMIS) 6, 4 (2016), 13
work page 2016
-
[20]
Xinyu Guan, Zhiyong Cheng, Xiangnan He, Yongfeng Zhang, Zhibo Zhu, Qinke Peng, and Tat-Seng Chua. 2019. Attentive Aspect Modeling for Review-aware Recommendation. ACM Transactions on Information Systems (TOIS) 37, 3 (2019), 28
work page 2019
-
[21]
Yangyang Guo, Zhiyong Cheng, Liqiang Nie, Yinglong Wang, Jun Ma, and Mohan Kankanhalli. 2019. Attentive Long Short-Term Preference Modeling for Personalized Product Search. ACM Transactions on Information Systems (TOIS) 37, 2 (2019), 19
work page 2019
-
[22]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InCVPR. 770–778
work page 2016
-
[23]
Ruining He and Julian McAuley. 2016. VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback.. In AAAI. 144–150
work page 2016
-
[24]
Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. InSIGIR. 355–364
work page 2017
-
[25]
Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat-Seng Chua. 2018. Outer Product-based Neural Collaborative Filtering. In IJCAI
work page 2018
-
[26]
Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial Personalized Ranking for Item Recom- mendation. In SIGIR. ACM Transactions on Information Systems, Vol. 0, No. 0, Article 111. Publication date: 2019. 111:22 Xiaoyu Du, et al
work page 2018
-
[27]
Xiangnan He, Zhenkui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018. NAIS: Neural Attentive Item Similarity Model for Recommendation. IEEE Transactions on Knowledge and Data Engineering (2018)
work page 2018
-
[28]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW. 173–182
work page 2017
-
[29]
Xiangnan He, Jinhui Tang, Xiaoyu Du, Richang Hong, Tongwei Ren, and Tat-Seng Chua. 2019. Fast Matrix Factorization with Non-Uniform Weights on Missing Data. IEEE transactions on neural networks and learning systems (2019)
work page 2019
-
[30]
Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recom- mendation with implicit feedback. In SIGIR. 549–558
work page 2016
-
[31]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780
work page 1997
-
[32]
Kurt Hornik. 1991. Approximation capabilities of multilayer feedforward networks. Neural networks 4, 2 (1991), 251–257
work page 1991
-
[33]
Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. 2017. Collaborative metric learning. In WWW. 193–201
work page 2017
-
[34]
Gao Huang, Zhuang Liu, Kilian Q Weinberger, and Laurens van der Maaten. 2017. Densely connected convolutional networks. In CVPR. 4700–4708
work page 2017
-
[35]
Santosh Kabbur, Xia Ning, and George Karypis. 2013. Fism: factored item similarity models for top-n recommender systems. In SIGKDD. ACM, 659–667
work page 2013
-
[36]
Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, and Hwanjo Yu. 2016. Convolutional matrix factorization for document context-aware recommendation. In RecSys. ACM, 233–240
work page 2016
-
[37]
Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In SIGKDD. ACM, 426–434
work page 2008
-
[38]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009)
work page 2009
-
[39]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In NIPS. 1097–1105
work page 2012
-
[40]
Dawen Liang, Laurent Charlin, James McInerney, and David M Blei. 2016. Modeling user exposure in recommendation. In WWW. 951–961
work page 2016
-
[41]
Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In ICLR
work page 2017
-
[42]
Zhongqi Lu, Zhicheng Dou, Jianxun Lian, Xing Xie, and Qiang Yang. 2015. Content-Based Collaborative Filtering for News Topic Recommendation.. In AAAI. 217–223
work page 2015
-
[43]
Xin Luo, MengChu Zhou, Shuai Li, Zhuhong You, Yunni Xia, and Qingsheng Zhu. 2016. A nonnegative latent factor model for large-scale sparse matrices in recommender systems via alternating direction method. IEEE transactions on neural networks and learning systems 27, 3 (2016), 579–592
work page 2016
-
[44]
Zhanyu Ma, Yuping Lai, W Bastiaan Kleijn, Yi-Zhe Song, Liang Wang, and Jun Guo. 2018. Variational Bayesian learning for Dirichlet process mixture of inverted Dirichlet distributions in non-Gaussian image feature modeling. IEEE transactions on neural networks and learning systems 99 (2018), 1–15
work page 2018
-
[45]
Zhanyu Ma, Jing-Hao Xue, Arne Leijon, Zheng-Hua Tan, Zhen Yang, and Jun Guo. 2018. Decorrelation of neutral vector variables: Theory and applications. IEEE transactions on neural networks and learning systems 29, 1 (2018), 129–143
work page 2018
-
[46]
Weike Pan, Qiang Yang, Wanling Cai, Yaofeng Chen, Qing Zhang, Xiaogang Peng, and Zhong Ming. 2019. Transfer to rank for heterogeneous one-class collaborative filtering. ACM Transactions on Information Systems (TOIS) 37, 1 (2019), 10
work page 2019
-
[47]
Tieyun Qian, Bei Liu, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2019. Spatiotemporal representation learning for translation-based poi recommendation. ACM Transactions on Information Systems (TOIS) 37, 2 (2019), 18
work page 2019
-
[48]
Bohui Fang Weinan Zhang Ruiming Tang Minzhe Niu Huifeng Guo Yong Yu Qu, Yanru and Xiuqiang He. 2018. Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data. ACM Transactions on Information Systems (TOIS) 37, 1 (2018), 5
work page 2018
-
[49]
Steffen Rendle. 2010. Factorization machines. In ICDM. 995–1000
work page 2010
-
[50]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In UAI. 452–461
work page 2009
-
[51]
Amit Sharma, Jake M Hofman, and Duncan J Watts. 2015. Estimating the causal impact of recommendation systems from observational data. In Proceedings of the Sixteenth ACM Conference on Economics and Computation. ACM, 453–470
work page 2015
-
[52]
Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking. In WWW. 729–739
work page 2018
-
[53]
Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. DKN: Deep Knowledge-Aware Network for News Recommendation. In WWW. 1835–1844. ACM Transactions on Information Systems, Vol. 0, No. 0, Article 111. Publication date: 2019. Convolutional NCF 111:23
work page 2018
-
[54]
Suhang Wang, Jiliang Tang, Yilin Wang, and Huan Liu. 2015. Exploring Implicit Hierarchical Structures for Recom- mender Systems. In IJCAI. 1813–1819
work page 2015
-
[55]
Xiang Wang, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2017. Item silk road: Recommending items from information domains to social users. In SIGIR. 185–194
work page 2017
-
[56]
Libing Wu, Cong Quan, Chenliang Li, Qian Wang, Bolong Zheng, and Xiangyang Luo. 2019. A context-aware user-item representation learning for item recommendation. ACM Transactions on Information Systems (TOIS) 37, 2 (2019), 22
work page 2019
-
[57]
Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for top-n recommender systems. In WSDM. ACM, 153–162
work page 2016
-
[58]
Hong-Jian Xue, Xinyu Dai, Jianbing Zhang, Shujian Huang, and Jiajun Chen. 2017. Deep Matrix Factorization Models for Recommender Systems. In IJCAI. 3203–3209
work page 2017
-
[59]
Fisher Yu and Vladlen Koltun. 2015. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)
work page internal anchor Pith review Pith/arXiv arXiv 2015
-
[60]
Wenhui Yu, Huidi Zhang, Xiangnan He, Xu Chen, Li Xiong, and Zheng Qin. 2018. Aesthetic-based Clothing Recom- mendation. In WWW. 649–658
work page 2018
-
[61]
Fajie Yuan, Guibing Guo, Joemon M Jose, Long Chen, Haitao Yu, and Weinan Zhang. 2016. Lambdafm: learning optimal ranking with factorization machines using lambda surrogates. In CIKM. ACM, 227–236
work page 2016
-
[62]
Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2019. A Simple Convolutional Generative Network for Next Item Recommendation. In WSDM
work page 2019
-
[63]
Fajie Yuan, Xin Xin, Xiangnan He, Guibing Guo, Weinan Zhang, Chua Tat-Seng, and Joemon M Jose. 2018. fbgd: Learning embeddings from positive unlabeled data with bgd. (2018)
work page 2018
-
[64]
Yongfeng Zhang, Qingyao Ai, Xu Chen, and W Bruce Croft. 2017. Joint representation learning for top-n recommen- dation with heterogeneous information sources. In CIKM. 1449–1458
work page 2017
-
[65]
Yongfeng Zhang, Min Zhang, Yiqun Liu, Shaoping Ma, and Shi Feng. 2013. Localized matrix factorization for recommendation based on matrix block diagonal forms. In Proceedings of the 22nd international conference on World Wide Web. ACM, 1511–1520
work page 2013
-
[66]
Wayne Xin Zhao, Wenhui Zhang, Yulan He, Xing Xie, and Ji-Rong Wen. 2018. Automatically learning topics and difficulty levels of problems in online judge systems. ACM Transactions on Information Systems (TOIS) 36, 3 (2018), 27
work page 2018
-
[67]
Zhou Zhao, Hanqing Lu, Deng Cai, Xiaofei He, and Yueting Zhuang. 2016. User preference learning for online social recommendation. IEEE Transactions on Knowledge and Data Engineering 28, 9 (2016), 2522–2534. ACM Transactions on Information Systems, Vol. 0, No. 0, Article 111. Publication date: 2019
work page 2016
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.