Solving a class of non-convex min-max games using adaptive momentum methods

Adaptive momentum methods have recently attracted a lot of attention for\ntraining of deep neural networks. They use an exponential moving average of\npast gradients of the objective function to update both search directions and\nlearning rates. However, these methods are not suited for solving min-max\noptimization problems that arise in training generative adversarial networks.\nIn this paper, we propose an adaptive momentum min-max algorithm that\ngeneralizes adaptive momentum methods to the non-convex min-max regime.\nFurther, we establish non-asymptotic rates of convergence for the proposed\nalgorithm when used in a reasonably broad class of non-convex min-max\noptimization problems. Experimental results illustrate its superior performance\nvis-a-vis benchmark methods for solving such problems.\n