Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Non-Convex Optimization

Backtracking line-search is an old yet powerful strategy for finding a better\nstep sizes to be used in proximal gradient algorithms. The main principle is to\nlocally find a simple convex upper bound of the objective function, which in\nturn controls the step size that is used. In case of inertial proximal gradient\nalgorithms, the situation becomes much more difficult and usually leads to very\nrestrictive rules on the extrapolation parameter. In this paper, we show that\nthe extrapolation parameter can be controlled by locally finding also a simple\nconcave lower bound of the objective function. This gives rise to a double\nconvex-concave backtracking procedure which allows for an adaptive choice of\nboth the step size and extrapolation parameters. We apply this procedure to the\nclass of inertial Bregman proximal gradient methods, and prove that any\nsequence generated by these algorithms converges globally to a critical point\nof the function at hand. Numerical experiments on a number of challenging\nnon-convex problems in image processing and machine learning were conducted and\nshow the power of combining inertial step and double backtracking strategy in\nachieving improved performances.\n