Gastrointestinal (GI) cancer precursors require frequent monitoring for risk\nstratification of patients. Automated segmentation methods can help to assess\nrisk areas more accurately, and assist in therapeutic procedures or even\nremoval. In clinical practice, addition to the conventional white-light imaging\n(WLI), complimentary modalities such as narrow-band imaging (NBI) and\nfluorescence imaging are used. While, today most segmentation approaches are\nsupervised and only concentrated on a single modality dataset, this work\nexploits to use a target-independent unsupervised domain adaptation (UDA)\ntechnique that is capable to generalize to an unseen target modality. In this\ncontext, we propose a novel UDA-based segmentation method that couples the\nvariational autoencoder and U-Net with a common EfficientNet-B4 backbone, and\nuses a joint loss for latent-space optimization for target samples. We show\nthat our model can generalize to unseen target NBI (target) modality when\ntrained using only WLI (source) modality. Our experiments on both upper and\nlower GI endoscopy data show the effectiveness of our approach compared to\nnaive supervised approach and state-of-the-art UDA segmentation methods.\n