Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification

We address the overlooked unbiasedness in existing long-tailed classification\nmethods: we find that their overall improvement is mostly attributed to the\nbiased preference of tail over head, as the test distribution is assumed to be\nbalanced; however, when the test is as imbalanced as the long-tailed training\ndata -- let the test respect Zipf's law of nature -- the tail bias is no longer\nbeneficial overall because it hurts the head majorities. In this paper, we\npropose Cross-Domain Empirical Risk Minimization (xERM) for training an\nunbiased model to achieve strong performances on both test distributions, which\nempirically demonstrates that xERM fundamentally improves the classification by\nlearning better feature representation rather than the head vs. tail game.\nBased on causality, we further theoretically explain why xERM achieves\nunbiasedness: the bias caused by the domain selection is removed by adjusting\nthe empirical risks on the imbalanced domain and the balanced but unseen\ndomain. Codes are available at https://github.com/BeierZhu/xERM.\n