Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding

State space models (SSMs) have demonstrated remarkable capabilities in natural language processing. Yet, their applicability to video understanding remains underexplored. We present the Video Mamba Suite, a suite of Mamba-based model families together with a unified evaluation protocol covering four roles in video understanding: temporal modeling, cross-modal interaction, temporal adapters, and spatiotemporal modeling. To address limitations of existing bidirectional scanning mechanisms, we propose the Decomposed Bidirectional Mamba (DBM) block, which enhances temporal modeling through parameter-sharing bidirectional pathways with separated input projections. Through extensive experiments spanning 14 SSM-based models across 12 video understanding tasks—including temporal localization, action segmentation, video captioning, and multimodal retrieval—we demonstrate that Mamba consistently achieves competitive or superior performance compared to Transformer baselines while maintaining linear computational complexity. Our findings establish SSMs as viable, versatile alternatives to Transformers for video understanding, offering significant efficiency advantages for long-sequence processing.

Paper

Similar papers

© 2026 NYSGPT2525 LLC