Fiber bundle image restoration using conditional generative adversarial network

Abstract

Fiber bundle have been widely used in biomedical endoscopy areas because of its flexibility and high spatial resolution. However, due to the irregular layout of the core and surrounding cladding, the captured fiber bundle images are affected by repeated honeycomb-like patterns. In this paper, we investigate the conditional generative adversarial network to reconstruct the optical fiber endoscope image. In order to improve the operation efficiency and reduce the burden of generative network, we use traditional method to obtain the fiber location map, as an additional condition to generate the network together with the input image. By injecting the position information of fibers, the generative network will pay more attention to the fiber regions and the surrounding structures, and the discriminative network will be able to assess the local consistency of the restored regions. In order to obtain more extensive contextual information, we apply multi-scale loss on the decoder side of the autoencoder. Each of these losses compares the difference between the output of the convolutional layer and the ground truth value that has been downscaled accordingly. Aside from the multi-scale losses, which are based on a pixel-by-pixel operation, we also add a perceptual loss that measures the global discrepancy between the features of the autoencoder’s output and those of the corresponding ground-truth clean image. Our experiments show the effectiveness of our approach which can effectively remove the honeycomb-like patterns and retain the original image features.