Iterative Event-based Motion Segmentation by Variational Contrast Maximization

Event cameras provide rich signals that are suitable for motion estimation since they respond to changes in the scene. As any visual changes in the scene produce event data, it is paramount to classify the data into different motions (i.e., motion segmentation), which is useful for various tasks such as object detection and visual servoing. We pro-pose an iterative motion segmentation method, by classifying events into background (e.g., dominant motion hypoth-esis) and foreground (independent motion residuals), thus extending the Contrast Maximization framework. Experi-mental results demonstrate that the proposed method suc-cessfully classifies event clusters both for public and self-recorded datasets, producing sharp, motion-compensated edge-like images. The proposed method achieves state-of-the-art accuracy on moving object detection benchmarks with an improvement of over 30%, and demonstrates its possibility of applying to more complex and noisy real-world scenes. We hope this work broadens the sensitiv-ity of Contrast Maximization with respect to both motion parameters and input events, thus contributing to theoret-ical advancements in event-based motion segmentation es-timation. https://github.com/aoki-media-lab/event_based_segmentation.vcmax