BanditMF: Multi-Armed Bandit Based Matrix Factorization Recommender System

Multi-armed bandits (MAB) provide a principled online learning approach to attain the balance between exploration and exploitation. Generally speaking, in a multi-armed bandit problem, to obtain a higher reward, the agent must choose the optimal action in various states based on previous experience (exploit) known actions to obtain a higher score; to discover these actions, the necessary discovery is required (exploration). Due to the superior performance and low feedback learning without the learning to act in multiple situations, multi-armed bandits are drawing widespread attention in applications ranging from recommender systems. Likewise, within the recommender system, collaborative filtering (CF) is arguably the earliest and most influential method in the recommender system. The meaning of collaboration is to filter the information through the relationship between the users and the feedback of the user’s rating of the items together to find the target users’ preferences. Crucially, new users and an ever-changing pool of recommended items are the challenges that recommender systems need to address. For collaborative filtering, the classical method is to train the model offline, then perform the online testing, but this approach can no longer handle the dynamic changes in user preferences, which is the so-called cold start. So, how to effectively recommend items to users in the absence of effective information? To address the aforementioned problems, a multi-armed bandit based collaborative filtering recommender system has been proposed, named BanditMF. BanditMF is designed to address two challenges in the multi-armed bandits algorithm and collaborative filtering: (1) how to solve the cold start problem for collaborative filtering under the condition of scarcity of valid information, (2) how to solve the sub-optimal problem of bandit algorithms in strong social relations domains caused by independently estimating unknown parameters associated with each user and ignoring correlations between users.