Learning Discriminative Features and Structured Models for Segmentation in Microscopy and Natural Images

Aurélien Lucchi
EPFL, 2013
EPFL thesis

Abstract

Segmenting images is a significant challenge that has drawn a lot of attention from different fields of artificial intelligence and has many practical applications. One such challenge addressed in this thesis is the segmentation of electron microscope (EM) imaging of neural tissue. EM microscopy is one of the key tools used to analyze neural tissue and understand the brain, but the huge amounts of data it produces make automated analysis necessary. In addition to the challenges specific to EM data, the common problems encountered in image segmentation must also be addressed. These problems include extracting discriminative features from the data and constructing a statistical model using ground-truth data. Although complex models appear to be more attractive because they allow for more expressiveness, they also lead to a higher computational complexity. On the other hand, simple models come with a lower complexity but less faithfully express the real world. Therefore, one of the most challenging tasks in image segmentation is in constructing models that are expressive enough while remaining tractable. In this work, we propose several automated graph partitioning approaches that address these issues. These methods reduce the computational complexity by operating on supervoxels instead of voxels, incorporating features capable of describing the 3D shape of the target objects and using structured models to account for correlation in output variables. One of the non-trivial issues with such models is that their parameters must be carefully chosen for optimal performance. A popular approach to learning model parameters is a maximum-margin approach called Structured SVM (SSVM) that provides optimality guarantees but also suffers from two main drawbacks. First, SSVM-based approaches are usually limited to linear kernels, since more powerful nonlinear kernels cause the learning to become prohibitively expensive. In this thesis, we introduce an approach to “kernelize” the features so that a linear SSVM framework can leverage the power of nonlinear kernels without incurring their high computational cost. Second, the optimality guarentees are violated for complex models with strong inter-relations between the output variables. We propose a new subgradient-based method that is more robust and leads to improved convergence properties and increased reliability. The different approaches presented in this thesis are applicable to both natural and medical images. They are able to segment mitochondria at a performance level close to that of a human annotator, and outperform state-of-the-art segmentation techniques while still benefiting from a low learning time.

Official source

https://infoscience.epfl.ch/record/188278?ln=en

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts

Related publications

Aurélien Lucchi
EPFL, 2013
EPFL thesis

Abstract

Official source

https://infoscience.epfl.ch/record/188278?ln=en

About this result

Related concepts (32)

Related concepts

Related publications (102)

Related publications

, , ,

Most state-of-the-art approaches to image segmentation formulate the problem using Conditional Random Fields. These models typically include a unary term and a pairwise term, whose parameters must be carefully chosen for optimal performance. Recently, structured learning approaches such as Structured SVMs (SSVM) have made it possible to jointly learn these model parameters. However, they have been limited to linear kernels, since more powerful non-linear kernels cause the learning to become prohibitively expensive. In this paper, we introduce an approach to ``kernelize'' the features so that a linear SSVM framework can leverage the power of non-linear kernels without incurring the high computational cost. We demonstrate the advantages of this approach in a series of image segmentation experiments on the MSRC data set as well as 2D and 3D datasets containing imagery of neural tissue acquired with electron microscopes.

2012

Towards Scalable Segmentation of 3D Image Stacks

Carlos Joaquin Becker

Imaging modalities such as Electron Microscopy (EM) and Light Microscopy (LM) can now deliver high-quality, high-resolution image stacks of neural structures. Though these imaging modalities can be used to analyze a variety of components that are critical to understanding brain function, the amount of human annotation effort required to analyze them remains a major bottleneck. This has triggered great interest in automating the annotation process, with most state-of-the-art algorithms nowadays relying on machine learning. However, such methods still require significant amounts of labeled examples for training, which can be highly time consuming and arduous, stressing the need for new approaches that require less amount of human effort. In light of this, we present here two efficient machine learning algorithms that incorporate expert knowledge to maximize prediction performance and simultaneously speed up analysis by reducing the required amount of labeled data. First, we present a new approach for the automated segmentation of synapses in image stacks acquired in EM that relies on image features specifically designed to take spatial context into account. These features are used to train a classifier that can effectively learn cues such as the presence of a nearby post-synaptic region. Our algorithm successfully distinguishes synapses from the numerous other organelles that appear within an EM volume, including those whose local textural properties are relatively similar. We evaluate our approach on three different datasets and demonstrate our ability to reliably collect shape, density, and orientation statistics over hundreds of synapses. Second, we focus on reducing the required amount of annotation effort. Due to changing experimental conditions in the image acquisition process, successive stacks often exhibit differences that are severe enough to make it difficult to use a classifier trained for a specific volume on another one. This means that the tedious annotation process has to be repeated for each new stack, resulting in a major bottleneck. We present a domain adaptation algorithm that addresses this issue by effectively leveraging labeled examples across different acquisitions and significantly reducing the annotation requirements. Our approach can handle complex, non-linear image feature transformations and scales to large microscopy datasets and high-dimensional feature spaces. We evaluate our approach on four EM and LM applications where annotation is very costly. We achieve a significant improvement over the state-of-the-art methods and demonstrate our ability to greatly reduce human annotation effort. Third, we apply our synapse segmentation approach to analyze and compare the structure and shape of synaptic densities in adult and aged mice, such as their area and number of perforations. This detailed analysis requires labeling each voxel within every synapse, making manual annotation unfeasible for large volumes. We show that we can bridge this gap with our approach and demonstrate its effectiveness on six large FIB/SEM brain stacks. Our approach generates segmentations that agree with expert annotations, while requiring very little annotation effort. To our knowledge, we are the first ones to analyze synapse shape in such detail on large stacks, as previous work has strongly relied on manual annotations, restricting analysis to small volumes.

EPFL2016

Learning Approach to Delineation of Curvilinear Structures in 2D and 3D Images

Agata Justyna Mosinska

Detection of curvilinear structures has long been of interest due to its wide range of applications. Large amounts of imaging data could be readily used in many fields, but it is practically not possible to analyze them manually. Hence, the need for automated delineation approaches. In the recent years Computer Vision witnessed a paradigm shift from mathematical modelling to data-driven methods based on Machine Learning. This led to improvements in performance and robustness of the detection algorithms. Nonetheless, most Machine Learning methods are general-purpose and they do not exploit the specificity of the delineation problem. In this thesis, we present learning methods suited for this task and we apply them to various kinds of microscopic and natural images, proving the general applicability of the presented solutions. First, we introduce a topology loss - a new training loss term, which captures higher-level features of curvilinear networks such as smoothness, connectivity and continuity. This is in contrast to most Deep Learning segmentation methods that do not take into account the geometry of the resulting prediction. In order to compute the new loss term, we extract topology features of prediction and ground-truth using a pre-trained network, whose filters are activated by structures at different scales and orientations. We show that this approach yields better results in terms of conventional segmentation metrics and overall topology of the resulting delineation. Although segmentation of curvilinear structures provides useful information, it is not always sufficient. In many cases, such as neuroscience and cartography, it is crucial to estimate the network connectivity. In order to find the graph representation of the structure depicted in the image, we propose an approach for joint segmentation and connection classification. Apart from pixel probabilities, this approach also returns the likelihood of a proposed path being a part of the reconstructed network. We show that segmentation and path classification are closely related tasks and can benefit from the synergy. The aforementioned methods rely on Machine Learning, which requires significant amounts of annotated ground-truth data to train models. The labelling process often requires expertise, it is costly and tiresome. To alleviate this problem, we introduce an Active Learning method that significantly decreases the time spent on annotating images. It queries the annotator only about the most informative examples, in this case the hypothetical paths belonging to the structure of interest. Contrary to conventional Active Learning methods, our approach exploits local consistency of linear paths to pick the ones that stand out from their neighborhood. Our final contribution is a method suited for both Active Learning and proofreading the result, which often requires more time than the automated delineation itself. It investigates edges of the delineation graph and determines the ones that are especially significant for the global reconstruction by perturbing their weights. Our Active Learning and proofreading strategies are combined with a new efficient formulation of an optimal subgraph computation and reduce the annotation effort by up to 80%.