f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation

Deep neural networks have become a mainstream approach to interactive\nsegmentation. As we show in our experiments, while for some images a trained\nnetwork provides accurate segmentation result with just a few clicks, for some\nunknown objects it cannot achieve satisfactory result even with a large amount\nof user input. Recently proposed backpropagating refinement (BRS) scheme\nintroduces an optimization problem for interactive segmentation that results in\nsignificantly better performance for the hard cases. At the same time, BRS\nrequires running forward and backward pass through a deep network several times\nthat leads to significantly increased computational budget per click compared\nto other methods. We propose f-BRS (feature backpropagating refinement scheme)\nthat solves an optimization problem with respect to auxiliary variables instead\nof the network inputs, and requires running forward and backward pass just for\na small part of a network. Experiments on GrabCut, Berkeley, DAVIS and SBD\ndatasets set new state-of-the-art at an order of magnitude lower time per click\ncompared to original BRS. The code and trained models are available at\nhttps://github.com/saic-vul/fbrs_interactive_segmentation .\n