Task Based Semantic Segmentation of Soft X-ray CT Images Using 3D Convolutional Neural Networks

Abstract

Semantic segmentation refers to the process of linking each pixel in an image to a class label, for example, cell phenotype, membrane, nucleus, or mitochondria. In practice, semantic segmentation simplifies a tomographic reconstruction and enables quantifiable analysis of the organelles in the cell, such as density differences, spatial distance metrics such as distances between organelles, and morphometrics. In Soft X-ray Tomography (SXT), segmentation is based on the measured Linear Absorption Coefficient (LAC) and guided and confirmed by structural cues, sub-cellular location, together with data from other imaging modalities. Currently, segmentation is a time-consuming, mostly manual process that often depends on specialist knowledge of the specimen to identify features in the reconstruction. To address this bottleneck, and make segmentation less dependent on user effort and expertise, we aim to incorporate automated, machine learning segmentation algorithms into our existing data processing and analysis pipeline. To date, NCXT staff and users have segmented thousands of individual cells. Since each segmentation represents many hours or even days of effort, the accumulated information and data is an enormously valuable resource for training machine learning algorithms. We have begun taking advantage of this resource by developing a data-driven machine learning segmentation pipeline Fig. 1. Our aim is to refine our segmentation pipeline to improve accuracy and set up a task-based framework, where users can define the required semantic information for their problem, and get a tailor-made algorithm that draws on all the available reference data for that specific task.