The Likelihood Ratio Test in High-Dimensional Logistic Regression Is Asymptotically a Rescaled Chi-Square

Logistic regression is used thousands of times a day to fit data, predict\nfuture outcomes, and assess the statistical significance of explanatory\nvariables. When used for the purpose of statistical inference, logistic models\nproduce p-values for the regression coefficients by using an approximation to\nthe distribution of the likelihood-ratio test. Indeed, Wilks' theorem asserts\nthat whenever we have a fixed number $p$ of variables, twice the log-likelihood\nratio (LLR) $2\\Lambda$ is distributed as a $\\chi^2_k$ variable in the limit of\nlarge sample sizes $n$; here, $k$ is the number of variables being tested. In\nthis paper, we prove that when $p$ is not negligible compared to $n$, Wilks'\ntheorem does not hold and that the chi-square approximation is grossly\nincorrect; in fact, this approximation produces p-values that are far too small\n(under the null hypothesis). Assume that $n$ and $p$ grow large in such a way\nthat $p/n\\rightarrow\\kappa$ for some constant $\\kappa < 1/2$. We prove that for\na class of logistic models, the LLR converges to a rescaled chi-square, namely,\n$2\\Lambda~\\stackrel{\\mathrm{d}}{\\rightarrow}~\\alpha(\\kappa)\\chi_k^2$, where the\nscaling factor $\\alpha(\\kappa)$ is greater than one as soon as the\ndimensionality ratio $\\kappa$ is positive. Hence, the LLR is larger than\nclassically assumed. For instance, when $\\kappa=0.3$,\n$\\alpha(\\kappa)\\approx1.5$. In general, we show how to compute the scaling\nfactor by solving a nonlinear system of two equations with two unknowns. Our\nmathematical arguments are involved and use techniques from approximate message\npassing theory, non-asymptotic random matrix theory and convex geometry. We\nalso complement our mathematical study by showing that the new limiting\ndistribution is accurate for finite sample sizes. Finally, all the results from\nthis paper extend to some other regression models such as the probit regression\nmodel.\n