On Learning Domain-Invariant Representations for Transfer Learning with Multiple Sources

Domain adaptation (DA) benefits from the rigorous theoretical works that\nstudy its insightful characteristics and various aspects, e.g., learning\ndomain-invariant representations and its trade-off. However, it seems not the\ncase for the multiple source DA and domain generalization (DG) settings which\nare remarkably more complicated and sophisticated due to the involvement of\nmultiple source domains and potential unavailability of target domain during\ntraining. In this paper, we develop novel upper-bounds for the target general\nloss which appeal to us to define two kinds of domain-invariant\nrepresentations. We further study the pros and cons as well as the trade-offs\nof enforcing learning each domain-invariant representation. Finally, we conduct\nexperiments to inspect the trade-off of these representations for offering\npractical hints regarding how to use them in practice and explore other\ninteresting properties of our developed theory.\n