HookNet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images

We propose HookNet, a semantic segmentation model for histopathology\nwhole-slide images, which combines context and details via multiple branches of\nencoder-decoder convolutional neural networks. Concentricpatches at multiple\nresolutions with different fields of view are used to feed different branches\nof HookNet, and intermediate representations are combined via a hooking\nmechanism. We describe a framework to design and train HookNet for achieving\nhigh-resolution semantic segmentation and introduce constraints to guarantee\npixel-wise alignment in feature maps during hooking. We show the advantages of\nusing HookNet in two histopathology image segmentation tasks where tissue type\nprediction accuracy strongly depends on contextual information, namely (1)\nmulti-class tissue segmentation in breast cancer and, (2) segmentation of\ntertiary lymphoid structures and germinal centers in lung cancer. Weshow the\nsuperiority of HookNet when compared with single-resolution U-Net models\nworking at different resolutions as well as with a recently published\nmulti-resolution model for histopathology image segmentation\n