Currently, the divergence in distributions of design and operational data,\nand large computational complexity are limiting factors in the adoption of CNNs\nin real-world applications. For instance, person re-identification systems\ntypically rely on a distributed set of cameras, where each camera has different\ncapture conditions. This can translate to a considerable shift between source\n(e.g. lab setting) and target (e.g. operational camera) domains. Given the cost\nof annotating image data captured for fine-tuning in each target domain,\nunsupervised domain adaptation (UDA) has become a popular approach to adapt\nCNNs. Moreover, state-of-the-art deep learning models that provide a high level\nof accuracy often rely on architectures that are too complex for real-time\napplications. Although several compression and UDA approaches have recently\nbeen proposed to overcome these limitations, they do not allow optimizing a CNN\nto simultaneously address both. In this paper, we propose an unexplored\ndirection -- the joint optimization of CNNs to provide a compressed model that\nis adapted to perform well for a given target domain. In particular, the\nproposed approach performs unsupervised knowledge distillation (KD) from a\ncomplex teacher model to a compact student model, by leveraging both source and\ntarget data. It also improves upon existing UDA techniques by progressively\nteaching the student about domain-invariant features, instead of directly\nadapting a compact model on target domain data. Our method is compared against\nstate-of-the-art compression and UDA techniques, using two popular\nclassification datasets for UDA -- Office31 and ImageClef-DA. In both datasets,\nresults indicate that our method can achieve the highest level of accuracy\nwhile requiring a comparable or lower time complexity.\n