Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning

Since the inception of Recommender Systems (RS), the accuracy of the\nrecommendations in terms of relevance has been the golden criterion for\nevaluating the quality of RS algorithms. However, by focusing on item\nrelevance, one pays a significant price in terms of other important metrics:\nusers get stuck in a "filter bubble" and their array of options is\nsignificantly reduced, hence degrading the quality of the user experience and\nleading to churn. Recommendation, and in particular session-based/sequential\nrecommendation, is a complex task with multiple - and often conflicting\nobjectives - that existing state-of-the-art approaches fail to address.\n In this work, we take on the aforementioned challenge and introduce\nScalarized Multi-Objective Reinforcement Learning (SMORL) for the RS setting, a\nnovel Reinforcement Learning (RL) framework that can effectively address\nmulti-objective recommendation tasks. The proposed SMORL agent augments\nstandard recommendation models with additional RL layers that enforce it to\nsimultaneously satisfy three principal objectives: accuracy, diversity, and\nnovelty of recommendations. We integrate this framework with four\nstate-of-the-art session-based recommendation models and compare it with a\nsingle-objective RL agent that only focuses on accuracy. Our experimental\nresults on two real-world datasets reveal a substantial increase in aggregate\ndiversity, a moderate increase in accuracy, reduced repetitiveness of\nrecommendations, and demonstrate the importance of reinforcing diversity and\nnovelty as complementary objectives.\n