Is interpolation benign for random forest regression?

Statistical wisdom suggests that very complex models, interpolating training\ndata, will be poor at predicting unseen examples.Yet, this aphorism has been\nrecently challenged by the identification of benign overfitting regimes,\nspecially studied in the case of parametric models: generalization capabilities\nmay be preserved despite model high complexity.While it is widely known that\nfully-grown decision trees interpolate and, in turn, have bad predictive\nperformances, the same behavior is yet to be analyzed for Random Forests\n(RF).In this paper, we study the trade-off between interpolation and\nconsistency for several types of RF algorithms. Theoretically, we prove that\ninterpolation regimes and consistency cannot be achieved simultaneously for\nseveral non-adaptive RF.Since adaptivity seems to be the cornerstone to bring\ntogether interpolation and consistency, we study interpolating Median RF which\nare proved to be consistent in the interpolating regime. This is the first\nresult conciliating interpolation and consistency for RF, highlighting that the\naveraging effect introduced by feature randomization is a key mechanism,\nsufficient to ensure the consistency in the interpolation regime and\nbeyond.Numerical experiments show that Breiman's RF are consistent while\nexactly interpolating, when no bootstrap step is involved.We theoretically\ncontrol the size of the interpolation area, which converges fast enough to\nzero, giving a necessary condition for exact interpolation and consistency to\noccur in conjunction.\n