Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications

We extend the work of Narasimhan and Bilmes [30] for minimizing set functions\nrepresentable as a dierence between submodular functions. Similar to [30], our\nnew algorithms are guaranteed to monotonically reduce the objective function at\nevery step. We empirically and theoretically show that the per-iteration cost\nof our algorithms is much less than [30], and our algorithms can be used to\nefficiently minimize a dierence between submodular functions under various\ncombinatorial constraints, a problem not previously addressed. We provide\ncomputational bounds and a hardness result on the multiplicative\ninapproximability of minimizing the dierence between submodular functions. We\nshow, however, that it is possible to give worst-case additive bounds by\nproviding a polynomial time computable lower-bound on the minima. Finally we\nshow how a number of machine learning problems can be modeled as minimizing the\ndierence between submodular functions. We experimentally show the validity of\nour algorithms by testing them on the problem of feature selection with\nsubmodular cost features.\n