Dissected 3D CNNs: Temporal Skip Connections for Efficient Online Video Processing

Convolutional Neural Networks with 3D kernels (3D-CNNs) currently achieve\nstate-of-the-art results in video recognition tasks due to their supremacy in\nextracting spatiotemporal features within video frames. There have been many\nsuccessful 3D-CNN architectures surpassing the state-of-the-art results\nsuccessively. However, nearly all of them are designed to operate offline\ncreating several serious handicaps during online operation. Firstly,\nconventional 3D-CNNs are not dynamic since their output features represent the\ncomplete input clip instead of the most recent frame in the clip. Secondly,\nthey are not temporal resolution-preserving due to their inherent temporal\ndownsampling. Lastly, 3D-CNNs are constrained to be used with fixed temporal\ninput size limiting their flexibility. In order to address these drawbacks, we\npropose dissected 3D-CNNs, where the intermediate volumes of the network are\ndissected and propagated over depth (time) dimension for future calculations,\nsubstantially reducing the number of computations at online operation. For\naction classification, the dissected version of ResNet models performs 77-90%\nfewer computations at online operation while achieving ~5% better\nclassification accuracy on the Kinetics-600 dataset than conventional 3D-ResNet\nmodels. Moreover, the advantages of dissected 3D-CNNs are demonstrated by\ndeploying our approach onto several vision tasks, which consistently improved\nthe performance.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC