Training fair machine learning models, aiming for their interpretability and\nsolving the problem of domain shift has gained a lot of interest in the last\nyears. There is a vast amount of work addressing these topics, mostly in\nseparation. In this work we show that they can be seen as a common framework of\nlearning invariant representations. The representations should allow to predict\nthe target while at the same time being invariant to sensitive attributes which\nsplit the dataset into subgroups. Our approach is based on the simple\nobservation that it is impossible for any learning algorithm to differentiate\nsamples if they have the same feature representation. This is formulated as an\nadditional loss (regularizer) enforcing a common feature representation across\nsubgroups. We apply it to learn fair models and interpret the influence of the\nsensitive attribute. Furthermore it can be used for domain adaptation,\ntransferring knowledge and learning effectively from very few examples. In all\napplications it is essential not only to learn to predict the target, but also\nto learn what to ignore.\n