A Comparison of First-order Algorithms for Machine Learning

Using an optimization algorithm to solve a machine learning problem is one of mainstreams in the field of science. In this work, we demonstrate a comprehensive comparison of some state-of-the-art first- order optimization algorithms for convex optimization problems in ma- chine learning. We concentrate on several smooth and non-smooth ma- chine learning problems with a loss function plus a regularizer. The over- all experimental results show the superiority of primal-dual algorithms in solving a machine learning problem from the perspectives of the ease to construct, running time and accuracy.