Domain adaptation performance of a learning algorithm on a target domain is a\nfunction of its source domain error and a divergence measure between the data\ndistribution of these two domains. We present a study of various distance-based\nmeasures in the context of NLP tasks, that characterize the dissimilarity\nbetween domains based on sample estimates. We first conduct analysis\nexperiments to show which of these distance measures can best differentiate\nsamples from same versus different domains, and are correlated with empirical\nresults. Next, we develop a DistanceNet model which uses these distance\nmeasures, or a mixture of these distance measures, as an additional loss\nfunction to be minimized jointly with the task's loss function, so as to\nachieve better unsupervised domain adaptation. Finally, we extend this model to\na novel DistanceNet-Bandit model, which employs a multi-armed bandit controller\nto dynamically switch between multiple source domains and allow the model to\nlearn an optimal trajectory and mixture of domains for transfer to the\nlow-resource target domain. We conduct experiments on popular sentiment\nanalysis datasets with several diverse domains and show that our DistanceNet\nmodel, as well as its dynamic bandit variant, can outperform competitive\nbaselines in the context of unsupervised domain adaptation.\n