Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation

Unsupervised domain adaptation (UDA) seeks to alleviate the problem of domain\nshift between the distribution of unlabeled data from the target domain w.r.t.\nlabeled data from the source domain. While the single-target UDA scenario is\nwell studied in the literature, Multi-Target Domain Adaptation (MTDA) remains\nlargely unexplored despite its practical importance, e.g., in multi-camera\nvideo-surveillance applications. The MTDA problem can be addressed by adapting\none specialized model per target domain, although this solution is too costly\nin many real-world applications. Blending multiple targets for MTDA has been\nproposed, yet this solution may lead to a reduction in model specificity and\naccuracy. In this paper, we propose a novel unsupervised MTDA approach to train\na CNN that can generalize well across multiple target domains. Our\nMulti-Teacher MTDA (MT-MTDA) method relies on multi-teacher knowledge\ndistillation (KD) to iteratively distill target domain knowledge from multiple\nteachers to a common student. The KD process is performed in a progressive\nmanner, where the student is trained by each teacher on how to perform UDA for\na specific target, instead of directly learning domain adapted features.\nFinally, instead of combining the knowledge from each teacher, MT-MTDA\nalternates between teachers that distill knowledge, thereby preserving the\nspecificity of each target (teacher) when learning to adapt to the student.\nMT-MTDA is compared against state-of-the-art methods on several challenging UDA\nbenchmarks, and empirical results show that our proposed model can provide a\nconsiderably higher level of accuracy across multiple target domains. Our code\nis available at: https://github.com/LIVIAETS/MT-MTDA\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC