Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition under Occlusion

Computer vision systems in real-world applications need to be robust to\npartial occlusion while also being explainable. In this work, we show that\nblack-box deep convolutional neural networks (DCNNs) have only limited\nrobustness to partial occlusion. We overcome these limitations by unifying\nDCNNs with part-based models into Compositional Convolutional Neural Networks\n(CompositionalNets) - an interpretable deep architecture with innate robustness\nto partial occlusion. Specifically, we propose to replace the fully connected\nclassification head of DCNNs with a differentiable compositional model that can\nbe trained end-to-end. The structure of the compositional model enables\nCompositionalNets to decompose images into objects and context, as well as to\nfurther decompose object representations in terms of individual parts and the\nobjects' pose. The generative nature of our compositional model enables it to\nlocalize occluders and to recognize objects based on their non-occluded parts.\nWe conduct extensive experiments in terms of image classification and object\ndetection on images of artificially occluded objects from the PASCAL3D+ and\nImageNet dataset, and real images of partially occluded vehicles from the\nMS-COCO dataset. Our experiments show that CompositionalNets made from several\npopular DCNN backbones (VGG-16, ResNet50, ResNext) improve by a large margin\nover their non-compositional counterparts at classifying and detecting\npartially occluded objects. Furthermore, they can localize occluders accurately\ndespite being trained with class-level supervision only. Finally, we\ndemonstrate that CompositionalNets provide human interpretable predictions as\ntheir individual components can be understood as detecting parts and estimating\nan objects' viewpoint.\n