Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks

This tutorial covers biomedical image reconstruction, from the foundational concepts of system modeling and direct reconstruction to modern sparsity and learning-based approaches. Imaging is a critical tool in biological research and medicine, and most imaging systems necessarily use an image reconstruction algorithm to create an image; the design of these algorithms has been a topic of research since at least the 1960’s. In the last few years, machine learning-based approaches have shown impressive performance on image reconstruction problems, triggering a wave of enthusiasm and creativity around the paradigm of learning. Our goal is to unify this body of research, identifying common principles and reusable building blocks across decades and among diverse imaging modalities. We first describe system modeling, emphasizing how a few building blocks can be used to describe a broad range of imaging modalities. We then discuss reconstruction algorithms, grouping them into three broad generations. The first are the classical direct methods, including Tikhonov regularization; the second are the variational methods based on sparsity and the theory of compressive sensing; and the third are the learning-based (also called data-driven) methods, especially those using deep convolutional neural networks. There are strong links between these generations: classical (first-generation) methods appear as modules inside the latter two, and the former two are used to inspire new designs for learning-based (third-generation) methods. As a result, a solid understanding of all three generations is necessary for the design of state-of-the-art algorithms.

3D high-density localization microscopy using hybrid astigmatic/biplane imaging and sparse image reconstruction

Lina Carlini, Seamus John Holden, Suliana Manley, Michaël Unser

Localization microscopy achieves nanoscale spatial resolution by iterative localization of sparsely activated molecules, which generally leads to a long acquisition time. By implementing advanced algorithms to treat overlapping point spread functions (PSFs), imaging of densely activated molecules can improve the limited temporal resolution, as has been well demonstrated in two-dimensional imaging. However, three-dimensional (3D) localization of high-density data remains challenging since PSFs are far more similar along the axial dimension than the lateral dimensions. Here, we present a new, high-density 3D imaging system and algorithm. The hybrid system is implemented by combining astigmatic and biplane imaging. The proposed 3D reconstruction algorithm is extended from our state-of-the art 2D high-density localization algorithm. Using mutual coherence analysis of model PSFs, we validated that the hybrid system is more suitable than astigmatic or biplane imaging alone for 3D localization of high-density data. The efficacy of the proposed method was confirmed via simulation and real data of microtubules. Furthermore, we also successfully demonstrated fluorescent-protein-based live cell 3D localization microscopy with a temporal resolution of just 3 seconds, capturing fast dynamics of the endoplasmic recticulum. (C) 2014 Optical Society of America

Optical Soc Amer2014

Deep Convolutional Neural Network for Inverse Problems in Imaging

Emmanuel Augustin Julien Froustey, Kyong Hwan Jin, Michael Thompson McCann, Michaël Unser

In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyperparameter selection. The starting point of this paper is the observation that unrolled iterative methods have the form of a CNN (filtering followed by pointwise nonlinearity) when the normal operator

(H ^{ * }

H, where

H ^{ * }

is the adjoint of the forward imaging operator, H) of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal-convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse-view reconstruction (down to 50 views) on parallel beam X-ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a 512 × 512 image on the GPU.

2017

SUBDIVIDE AND CONQUER Active Contours and Surfaces for Biomedical Image Segmentation

Anaïs Laure Marie-Thérèse Badoual

Ongoing advances in imaging techniques create new demands regarding the analysis of images in medicine and biology. Image segmentation is a key step of many image analysis pipelines and its proper execution is a particularly challenging task. This thesis is dedicated to the development of segmentation algorithms for biomedical structures in 2D and 3D images. In this work, we aim to improve upon classical parametric active contours/surfaces, whose limitations we address. This thesis is organized in three parts. First, we introduce two representation models. They adapt their resolution to the level of detail of the object to be segmented. We then focus on the formulation of cost functions, called energies, that guide the curve/surface toward the boundary of the target in the image. Among others, we present novel energies based on ridge and texture information. Finally, with these two ingredients in hand, we design new semi-automated active contours/surfaces, also called snakes, for various applications. Our methods are generic enough to be used with a broad variety of data. In particular, we illustrate their performance in segmenting real biomedical images. In addition to those three parts, we provide mathematical tools for signal processing that we designed to efficiently process periodic functions. To find the optimal curve/surface that best fits a given target, we adopt throughout the thesis a subdivide and conquer strategy. First, we look at several smoothed versions of the original image and, for each, adapt the resolution of the snake curve/surface to the level of detail of the target (i.e., subdivide). Then, we segment the object of interest at each resolution recursively, from the coarsest to the finest image (i.e., conquer). This robust subdivide and conquer strategy exploits the multiresolution property of our curve/surface representations, as well as characteristics of smoothed images (few details and low noise). Finally, we give a special attention to the conversion of our algorithms into usable software that respect the open-source, user-friendly and reproducibility criteria.

EPFL2019