A High-Performance Implementation of Bayesian Matrix Factorization with Limited Communication

Matrix factorization is a very common machine learning technique in\nrecommender systems. Bayesian Matrix Factorization (BMF) algorithms would be\nattractive because of their ability to quantify uncertainty in their\npredictions and avoid over-fitting, combined with high prediction accuracy.\nHowever, they have not been widely used on large-scale data because of their\nprohibitive computational cost. In recent work, efforts have been made to\nreduce the cost, both by improving the scalability of the BMF algorithm as well\nas its implementation, but so far mainly separately. In this paper we show that\nthe state-of-the-art of both approaches to scalability can be combined. We\ncombine the recent highly-scalable Posterior Propagation algorithm for BMF,\nwhich parallelizes computation of blocks of the matrix, with a distributed BMF\nimplementation that users asynchronous communication within each block. We show\nthat the combination of the two methods gives substantial improvements in the\nscalability of BMF on web-scale datasets, when the goal is to reduce the\nwall-clock time.\n