In this paper, we investigate effective neural network layers for optical flow estimation and in particular for omni-directional optical flow. Optical flow has many applications in computer graphics, augmented reality and in 3D modeling. We create a simple dataset that enables us to efficiently assess the effectiveness of different neural network layers for optical flow. Based on this small-sized diagnostic dataset, FlowCLEVR, we conclude that a deformable convolution layer is highly effective in reducing motion and occlusion boundary blur. Based on these results, we are able to design modifications to various existing network architectures improving their performance. We demonstrate improved performance on FlowCLEVR, on standard datasets for optical flow in planar images and on a novel omni-directional optical flow dataset. We also extend our work to omni-directional stereo.
Paper
Full text
Effective Convolutional Neural Network Layers in Flow Estimation for Omni-Directional Images
Semantic Scholar · Computer Science · 2019
Abstract
In this paper, we investigate effective neural network layers for optical flow estimation and in particular for omni-directional optical flow. Optical flow has many applications in computer graphics, augmented reality and in 3D modeling. We create a simple dataset that enables us to efficiently assess the effectiveness of different neural network layers for optical flow. Based on this small-sized diagnostic dataset, FlowCLEVR, we conclude that a deformable convolution layer is highly effective in reducing motion and occlusion boundary blur. Based on these results, we are able to design modifications to various existing network architectures improving their performance. We demonstrate improved performance on FlowCLEVR, on standard datasets for optical flow in planar images and on a novel omni-directional optical flow dataset. We also extend our work to omni-directional stereo.