In this paper, we study the problem of early stopping for iterative learning\nalgorithms in a reproducing kernel Hilbert space (RKHS) in the nonparametric\nregression framework. In particular, we work with the gradient descent and\n(iterative) kernel ridge regression algorithms. We present a data-driven rule\nto perform early stopping without a validation set that is based on the\nso-called minimum discrepancy principle. This method enjoys only one assumption\non the regression function: it belongs to a reproducing kernel Hilbert space\n(RKHS). The proposed rule is proved to be minimax-optimal over different types\nof kernel spaces, including finite-rank and Sobolev smoothness classes. The\nproof is derived from the fixed-point analysis of the localized Rademacher\ncomplexities, which is a standard technique for obtaining optimal rates in the\nnonparametric regression literature. In addition to that, we present simulation\nresults on artificial datasets that show the comparable performance of the\ndesigned rule with respect to other stopping rules such as the one determined\nby V-fold cross-validation.\n