Segmentation-Driven Infrared and Visible Image Fusion Via Transformer-Enhanced Architecture Searching

Abstract

A series of infrared and visible image fusion (IVIF) methods have emerged to improve the performance of segmentation task. However, existing perception-focused IVIF methods take visual effects and semantic information as a unified goal for training, ignoring the task conflicts. Moreover, these methods often involve manually designed modules, which are laborious and suboptimal. To solve the problems, we propose a collaborative feature learning framework based on neural architecture search (NAS). Specifically, we extract shared features of fusion and segmentation tasks into a unified space and separately process task objectives through a dual decoder. In light of the essential role that semantic information plays in the segmentation task, we construct a hybrid search space with transformers incorporated to enhance context dependence handling. Our method undergoes extensive experiments, showcasing exceptional visual effects and significant enhancements in segmentation tasks compared to other state-of-the-art methods.