DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks

In this paper, we propose <italic>DeepCut</italic>, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known <italic>GrabCut</italic> <xref ref-type="bibr" rid="ref1">[1]</xref> method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the <italic>DeepCut</italic> method and compare those to a naïve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.