Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives

The recent work by Rendle et al. (2020), based on empirical observations,\nargues that matrix-factorization collaborative filtering (MCF) compares\nfavorably to neural collaborative filtering (NCF), and conjectures the dot\nproduct's superiority over the feed-forward neural network as similarity\nfunction. In this paper, we address the comparison rigorously by answering the\nfollowing questions: 1. what is the limiting expressivity of each model; 2.\nunder the practical gradient descent, to which solution does each optimization\npath converge; 3. how would the models generalize under the inductive and\ntransductive learning setting. Our results highlight the similar expressivity\nfor the overparameterized NCF and MCF as kernelized predictors, and reveal the\nrelation between their optimization paths. We further show their different\ngeneralization behaviors, where MCF and NCF experience specific tradeoff and\ncomparison in the transductive and inductive collaborative filtering setting.\nLastly, by showing a novel generalization result, we reveal the critical role\nof correcting exposure bias for model evaluation in the inductive setting. Our\nresults explain some of the previously observed conflicts, and we provide\nsynthetic and real-data experiments to shed further insights to this topic.\n