A Mutual Information Constrained Multitask Learning Method for Very High-Resolution Building Segmentation

Abstract

Accurately extracting building footprints from remote sensing imagery is essential for urban management. Multitask learning (MTL) has shown its potential on improving segmentation accuracy with shared network weights to simultaneously capture various building-related features. However, due to the lack of adaptive context information balancing and effective knowledge transfer mechanisms between tasks, current MTL methods often obtain incomplete and irregular delineations of buildings. To address these issues, we propose a mutual information constrained multitask learning network (MIMNet) for precise building segmentation from remote sensing imagery. The MIMNet introduces a contextual information parallel perception structure to capture both global and local building features. In addition, it incorporates a fourier mutual information balancing module to promote the interaction and fusion of multiscale contextual information. The MIMNet simultaneously segments building mask and boundary utilizing the MTL strategy, and employs a mutual information loss function to enhance information exchange between the two tasks. Experimental evaluations demonstrate that the proposed MIMNet framework achieves state-of-the-art performance across three benchmark datasets, obtaining intersection over union scores of 91.49% on the WHU aerial dataset, 76.43% on the Massachusetts building dataset, and 83.54% on the Inria aerial dataset.