Dear Reviewer UAYX,
It's really great to hear your feedback! We will explain why we didn't make *statistical consistency* (we will use consistency for short in this reply) a second main theorem about GIW.
The consistency of training with IW relies on two parts: the consistency of *weight estimation* and the consistency of *weighted classification*. Consider the second part by assuming we already have the first part. As long as the importance weight is upper bounded by a finite constant, the consistency of weighted classification will be almost the same as that of the standard ERM, where the *estimation error bound* of training with IW would be the est err bound of ERM times the above upper bound of importance weight times 2 (here, the bound is further doubled since the empirical risk estimator is only consistent but not unbiased, and *symmetrization* is not as tight as before). Thus, the convergence rate would be the same.
The difficulty is the first part. Here, it is truly an estimation problem rather than an optimization problem, and the target is not defined by optimization like in the second part. The target must be the ratio of test density over training density, namely $p_{te}(x,y)/p_{tr}(x,y)$. As a consequence, when talking about the consistency of weight estimation or estimated weights, we have to consider *not only estimation error but also approximation error*. In the literature, we have upper bounds of the difference between estimated importance weights and true importance weights (for example, MSE between them). However, the upper bounds don't really converge to zero when the number of data approach infinity, due to the existence of approximation error. When deep models are used and *representation learning* is involved, there is another key issue: the *feature extractor* $\pi:(x,y)\mapsto z$ is obtained from representation learning, and $p_{te}(x,y)/p_{tr}(x,y)=p_{te}(z)/p_{tr}(z)$ only if $\pi$ is *invertible* (there are two other trivial assumptions, and "only if" becomes "if and only if" with all three assumptions). This assumption is really non-trivial but it is only needed for theoretical purposes. We tried to avoid introducing this assumption, because the practical algorithms don't require to use invertible deep networks or so-called *deep invertible networks*.
Fortunately, IW methods have an amazing property: *weighted classification works as long as we have roughly good estimated importance weights*. For example, we know the analytic solution of weights under class-prior shift, and if we assume that the true label is deterministic (which is approximately true for MNIST and CIFAR-10) we also know the analytic solution of weights under class-posterior shift. Intuitively, under class-prior shift, all instances belonging to majority classes should have a very small weight and all instances belonging to minority classes should have a very large weight; under class-posterior shift, all mislabeled instances should have a zero weight and all correctly labeled instances should have a clearly non-zero weight. We can train fairly good classifiers as long as the estimated weights can tell the difference between majority and minority classes or between mislabeled and correctly labeled instances. In fact, in optimization for machine learning, a true (global or local) minimizer with zero gradient may not necessarily be better than a reasonably good solution with near-zero gradient near the true minimizer. The situation of weight estimation is the same, where the true importance weights may not necessarily be better than some reasonably good estimated weights near the true weights, since we have finite data for weighted classification and *what we care most is weighted classification but not weight estimation*. That was also why researchers here don't care too much the consistency of weight estimation.
We didn't have enough space to include the above discussion in our original submission. We omitted it because we thought our readers should mainly be deep learning researchers who may not care about this issue. However, we will try our best to include the discussion into our final submission, to avoid any potential confusion.
Hope our explanation can clarify your concern and also hope you can take care of yourself during your extremely busy days! When you have some time, as you have promised, hope you can remember to "raise the rating to weak accept".
Authors