Train simultaneously, generalize better: Stability of gradient-based minimax learners

The success of minimax learning problems of generative adversarial networks\n(GANs) has been observed to depend on the minimax optimization algorithm used\nfor their training. This dependence is commonly attributed to the convergence\nspeed and robustness properties of the underlying optimization algorithm. In\nthis paper, we show that the optimization algorithm also plays a key role in\nthe generalization performance of the trained minimax model. To this end, we\nanalyze the generalization properties of standard gradient descent ascent (GDA)\nand proximal point method (PPM) algorithms through the lens of algorithmic\nstability under both convex concave and non-convex non-concave minimax\nsettings. While the GDA algorithm is not guaranteed to have a vanishing excess\nrisk in convex concave problems, we show the PPM algorithm enjoys a bounded\nexcess risk in the same setup. For non-convex non-concave problems, we compare\nthe generalization performance of stochastic GDA and GDmax algorithms where the\nlatter fully solves the maximization subproblem at every iteration. Our\ngeneralization analysis suggests the superiority of GDA provided that the\nminimization and maximization subproblems are solved simultaneously with\nsimilar learning rates. We discuss several numerical results indicating the\nrole of optimization algorithms in the generalization of the learned minimax\nmodels.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC