SEINE: Structure Encoding and Interaction Network for Nuclei Instance Segmentation

Nuclei instance segmentation in histopathological images is crucial for biological analysis and cancer diagnosis. However, it faces two significant challenges: (1) poorly stained nuclei can lead to under-segmentation, as the background may be mistakenly identified as the foreground; and (2) deep textures within nuclei often result in fragmented instance predictions, as these textures can be misinterpreted as contours. To address these problems, this paper proposes a Structure Encoding and Interaction NEtwork, termed SEINE, which develops the nuclei structure modeling scheme and takes advantage of the similarity between nuclei structure to improve the integrality of instance segmentation. Specifically, SEINE introduces a contour-based structure encoding mechanism that integrates the correlation between nuclear structure and semantics, enabling a more accurate structural representation. Building on this encoding, we propose a structure-guided attention module, which uses clear nuclei as prototypes to guide the structural learning of unclear nuclei, thereby addressing the under-segmentation problem. Additionally, a position enhancement strategy applies a centroid distance constraint to reduce contour prediction errors, effectively mitigating fragmented instance segmentation. Extensive experiments demonstrate the effectiveness of SEINE, achieving state-of-the-art performance across four benchmark datasets.