Weakly Supervised Volumetric Image Segmentation with Deformed Templates

There are many approaches to weakly-supervised training of networks to segment 2D images. By contrast, existing approaches to segmenting volumetric images rely on full-supervision of a subset of 2D slices of the 3D volume. We propose an approach to volume segmentation that is truly weakly-supervised in the sense that we only need to provide a sparse set of 3D points on the surface of target objects instead of detailed 2D masks. We use the 3D points to deform a 3D template so that it roughly matches the target object outlines and we introduce an architecture that exploits the supervision it provides to train a network to find accurate boundaries. We evaluate our approach on Computed Tomography (CT), Magnetic Resonance Imagery (MRI) and Electron Microscopy (EM) image datasets and show that it substantially reduces the required amount of effort.

Object Priors for Volumetric Image Segmentation

Pamuditha Udaranga Wickramasinghe

Large training datasets have played a vital role in the success of modern deep learning methods in computer vision. But, obtaining sufficient amount of training data is challenging, specially when annotating volumetric images. This is because fully annotating a single image volume require annotating a whole stack of images which is costly. Moreover, majority of volumetric images originate from niche domains that require expert knowledge to annotate and it further increase the associated cost. Therefore, designing models for volumetric image segmentation that can work with small training datasets is of great importance. One way to address this challenge is to use prior knowledge in segmentation model design. There are many forms of prior knowledge and in this thesis, we focus on using object priors which defines prior knowledge related to the properties of the object being segmented. We begin by proposing an end-to-end differentiable architecture that produce surface meshes from input image volumes which enable easy integration of object priors related to topology and surface properties. With that we demonstrate how the priors help achieve better accuracy and quality in predictions compared to state-of-the-art methods that do not use them. Then we focus on surface priors and propose a methodology based on Active Surface models to achieve better trade-offs between the accuracy of the surfaces produced by the segmentation models and their quality. Afterwards, we apply the Active Surface model proposed earlier to develop an interactive annotation tool that significantly reduce the effort necessary to annotate image volumes. Along with that, a volumetric image segmentation model that best utilize annotations produced with the tool is also proposed. Finally we focus on object priors related to overall shape. We use Probabilistic Atlases (PAs) to introduce this prior knowledge into the segmentation model and demonstrate its ability to improve the accuracy in predictions. Overall, we use object prior to improve the accuracy, quality and the speed of annotation in volumetric image segmentation. To demonstrate their effectiveness, we use volumetric images produced by Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and Electron Microscopy (EMs).

EPFL2022

Introducing Geometry in Active Learning for Image Segmentation

Pascal Fua, Ksenia Konyushkova, Raphael Sznitman

We propose an Active Learning approach to training a segmentation classifier that exploits geometric priors to streamline the annotation process in 3D image volumes. To this end, we use these priors not only to select voxels most in need of annotation but to guarantee that they lie on 2D planar patch, which makes it much easier to annotate than if they were randomly distributed in the volume. A simplified version of this approach is effective in natural 2D images. We evaluated our approach on Electron Microscopy and Magnetic Resonance image volumes, as well as on natural images. Comparing our approach against several accepted baselines demonstrates a marked performance increase.

2015

Geometry in active learning for binary and multi-class image segmentation

Pascal Fua, Ksenia Konyushkova, Raphael Sznitman

We propose an active learning approach to image segmentation that exploits geometric priors to speed up and streamline the annotation process. It can be applied for both background foreground and multi-class segmentation tasks in 2D images and 3D image volumes. Our approach combines geometric smoothness priors in the image space with more traditional uncertainty measures to estimate which pixels or voxels are the most informative, and thus should to be annotated next. For multi-class settings, we additionally introduce two novel criteria for uncertainty. In the 3D case, we use the resulting uncertainty measure to select voxels lying on a planar patch, which makes batch annotation much more convenient for the end user compared to the setting where voxels are randomly distributed in a volume. The planar patch is found using a branch-and-bound algorithm that looks for a 2D patch in a 3D volume where the most informative instances are located. We evaluate our approach on Electron Microscopy and Magnetic Resonance image volumes, as well as on regular images of horses and faces. We demonstrate a substantial performance increase over other approaches thanks to the use of geometric priors.

2019

Object Priors for Volumetric Image Segmentation

Pamuditha Udaranga Wickramasinghe

EPFL2022