MLSA4Rec: Mamba Combined with Low-Rank Decomposed Self-Attention for Sequential Recommendation

Sequential recommender systems are critical in applications such as e-commerce, online education, and streaming services. Despite the excellent performance of self-attention-based sequential recommendation methods in capturing dependencies between items in user interaction histories, their quadratic complexity and lack of structural bias limit their applicability. Recently, Mamba has been introduced to replace the self-attention module in these methods due to its linear complexity, though the potential complementarity between these approaches has not been fully explored. To address this gap, we propose MLSA4Rec, a hybrid framework that combines Mamba with low-rank decomposed self-attention (LSA) for sequential recommendation, achieving linear complexity with respect to the length of the user’s historical interaction sequence. MLSA4Rec integrates an efficient Mamba-LSA interaction component that leverages LSA’s linear complexity and incorporates structural bias from the Mamba module. Specifically, the LSA module analyzes latent user preferences and dynamically guides the Mamba module to prioritize key information in the user’s interaction history through a gated information transmission mechanism. After completing the information interaction between the Mamba and LSA modules, the outputs from both will contain various types of information, such as local information, global information, and order information. These will be effectively integrated to form the final output of MLSA4Rec. The Mamba-LSA interaction component enables the integration of the specialized knowledge from both Mamba and LSA, thereby harnessing their complementary strengths. To the best of our knowledge, this is the first study to combine Mamba and self-attention in sequential recommendation. Experimental results on three real-world datasets demonstrate that MLSA4Rec outperforms existing methods, showcasing the potential of this hybrid approach.