Fast and automated wavelet-regularized image restoration in fluorescence microscopy

The subject of this thesis is image restoration, that is, deconvolution and denoising. Our work is motivated by applications in the rapidly expanding field of biological imaging and in particular fluorescence microscopy. Inverse problems in this area typically involve regularly structured but high-dimensional data sets, thus requiring a specific tradeoff between restoration quality, computational complexity and user interaction. Our interest lies in novel algorithmic solutions for addressing these challenges. The common point of our developments is that they rely centrally on wavelets and multilevel concepts. The thesis is organized in three main parts. In the first part, we provide a general overview of fluorescence microscopy in the context of biological research. We then review the current state of the art in deconvolution microscopy, within a unifying variational framework. The second part is concerned with deconvolution using a non-quadratic (typically ℓ1) wavelet-domain regularization. In the context of deconvolution microscopy, this approach is novel. It amounts to imposing a sparsity constraint on the object to be recovered, a concept that has recently attracted considerable interest in signal and image processing. This type of regularization was shown to belong to the most effective restoration methods, but it involves a computationally challenging optimization problem. We propose a multilevel version of the standard "thresholded Landweber" algorithm that yields a substantial acceleration. This allows us to apply wavelet regularization to large-scale deconvolution problems encountered in deconvolution microscopy. In the third part, we present some applications of risk estimation techniques for automating the process of image restoration. We first propose a denoising method that is specifically designed for very low light intensities. The algorithm is designed from the beginning for complete self-adjustment of its parameters and for low computational complexity. It also performs competitively with most existing approaches for Poisson intensity estimation. Our second contribution is a general method for estimating the signal-to-noise ratio during the execution of iterative image-restoration procedures; it is applicable to most algorithms emanating from variational formulations, in particular those involving wavelet regularization.

Resolution and Complex Deconvolution in Holographic Microscopy

Yann Jérôme Michel Pascal Cotte

Interest in high resolution imaging techniques has recently multiplied due to their importance in bio-medical research. Quantitative phase measurements by holographic microscopy is an extraordinary tool to gain new understanding of transparent biological samples. The extremely high demand on resolution well beyond the diffraction limit, however, sets new benchmarks for imaging techniques. Digital Holographic Microscopy is an interferometric method providing access to the complex wave front. Its capacity to image amplitude and quantitative phase simultaneously makes it an attractive research tool in many fields of biological research. It is on the verge of becoming an appealing alternative to classical fluorescence microscopy. For intensity-based microscopy, however, super-resolution methods are well established. In this thesis, approaches to unleash resolution for phase imaging techniques are explored. Based on truncated inverse filtering, a theory for deconvolution of complex fields is developed. It is a post-processing method that does not require any additional optics in the holographic microscopy setup. Gain in resolution arises by accessing the object's complex field – containing the information encoded in the phase – and deconvolving it with the characteristic coherent transfer function (CTF). By efficient harvest of a diffraction limited bandpass, complex deconvolution is demonstrated to exceed the coherent resolution limit. A novel method, called a complex point source, serves to characterize the holographic microscopy system. It consists of a coherently illuminated nano-metric hole, located on a conventional microscope slide, most common in bio-microscopy. A thin opaque film is directly evaporated on the slide and the circular sub-wavelength structures are drilled with a focused ion beam. Theoretical as well as practical work on transmission properties of apertures confirms the advantageous signal-to-noise yield of proposed method. More abstractly, the thus gained experimental amplitude point spread function is demonstrated to be suitable for experimental CTF reconstruction. Herefrom, experimental parameters are extracted and introduced into scalar and vectorial Debye theory, called synthetic CTF, adaptable to realistic imaging conditions. Also, it allows to study the role of noise in the context of complex field deconvolution. The resolution power of holographic microscopy systems is systematically examined with pairs of nano-metric apertures separated by sub-resolution pitches. Theory of information content is elaborated depending on different experimental configurations. Resolution extends beyond classical Abbe's limit by introducing spatially a varying phase into the illumination beam of a phase imaging system. In a last step, the novel deconvolution method is generalized to three-dimensional image processing of complex fields. By combining scattering theory and signal processing, the method is demonstrated to yield the reconstruction of the scattering object field under non-design microscope objective imaging conditions. The suggested technique is best suited for an implementation in high-resolution diffraction tomography based on sample and/or illumination rotation.

EPFL2012

The Colored Revolution of Bioimaging

François Aguet, Michaël Unser, Cédric René Jean Vonesch

With the recent development of fluorescent probes and new high-resolution microscopes, biological imaging has entered a new era and is presently having a profound impact on the way research is being conducted in the life sciences. Biologists have come to depend more and more on imaging. They can now visualize subcellular components and processes in vivo, both structurally and functionally. Observations can be made in two or three dimensions, at different wavelengths (spectroscopy), possibly with time-lapse imaging to investigate cellular dynamics. The observation of many biological processes relies on the ability to identify and locate specific proteins within their cellular environment. Cells are mostly transparent in their natural state and the immense number of molecules that constitute them are optically indistinguishable from one another. This makes the identification of a particular protein a very complex task—akin to finding a needle in a haystack. However, if a bright marker were attached to the protein of interest, it could very precisely indicate its position. Much effort has gone into finding suitable markers for this purpose, but it is only over the course of the past decade, with the advent of fluorescent proteins, that this concept has been revolutionized. These biological markers have the crucial properties necessary for dynamic observations of living cells: they are essentially harmless to the organism, and can be attached to other proteins without impacting their function. Fluorescence microscopy was invented almost a century ago, when microscopists were experimenting with ultraviolet light to achieve higher resolutions. In the very beginning, observations were limited to specimens that naturally fluoresce. Rapidly, fluorescent dyes for staining tissues and cells were investigated. But it was not until the 1940s that fluorescence microscopy became popular, when Coons and Kaplan introduced a technique to label antibodies with a fluorescent dye to study antibody-antigen interactions, profoundly changing the field of immunohistochemistry. The discovery that really brought fluorescence microscopy to the forefront came in 1994, when M. Chalfie et al. succeeded in expressing a naturally fluorescent protein, the now-famous green fluorescent protein (GFP), in living organisms. This was a landmark evolution in the field, fostering a whole new class of tagging methods. While genetic engineering is at the origin of this new methodology, a number of innovations from the fields of physics, optics, and mechanical and electrical engineering have been combined to provide the necessary instrumentation. Impressive enhancements in classical microscopy have been achieved, and new imaging systems are actively being developed. A key element for the evolution of microscopy in general was the shift to digital imaging in the 1990s, with the availability of affordable high-sensitivity acquisition devices and powerful computer hardware. The capabilities of today's systems often lead to enormous data sets that, in most cases, require postprocessing for their interpretation. Signal processing methods for biological research are only at their prelude; the needs are considerable and most probably not even clearly formulated yet. It is thus predictable that signal processing will be one of the main challenges of fluorescence microscopy in the forthcoming years. The goal of this article is to provide an overview of the main aspects of modern fluorescence microscopy. We first cover the principles of fluorescence and highlight the key discoveries in the history of fluorescence microscopy. In subsequent sections, we present the optics of fluorescence microscopes and examine various types of detectors. Finally, we discuss the signal and image processing challenges in fluorescence microscopy and highlight some of the present developments and future trends in the field.

IEEE2006

Super-resolution fluorescence microscopy based on physical models

François Aguet

This thesis introduces a collection of physics-based methods for super-resolution in optical microscopy. The core of these methods constitutes a framework for 3-D localization of single fluorescent molecules. Localization is formulated as a parameter estimation problem relying on a physically accurate model of the system's point spread function (PSF). In a similar approach, methods for fitting PSF models to experimental observations and for extended-depth-of-field imaging are proposed. Imaging of individual fluorophores within densely labeled samples has become possible with the discovery of dyes that can be photo-activated or switched between fluorescent and dark states. A fluorophore can be localized from its image with nanometer-scale accuracy, through fitting with an appropriate image function. This concept forms the basis of fluorescence localization microscopy (FLM) techniques such as photo-activated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM), which rely on Gaussian fitting to perform the localization. Whereas the image generated by a single fluorophore corresponds to a section of the microscope's point spread function, only the in-focus section of the latter is well approximated by a Gaussian. Consequently, applications of FLM have for the most part been limited to 2-D imaging of thin specimen layers. In the first section of the thesis, it is shown that localization can be extended to 3-D without loss in accuracy by relying on a physically accurate image formation model in place of a Gaussian approximation. A key aspect of physically realistic models lies in their incorporation of aberrations that arise either as a consequence of mismatched refractive indices between the layers of the sample setup, or as an effect of experimental settings that deviate from the design conditions of the system. Under typical experimental conditions, these aberrations change as a function of sample depth, inducing axial shift-variance in the PSF. This property is exploited in a maximum-likelihood framework for 3-D localization of single fluorophores. Due to the shift-variance of the PSF, the axial position of a fluorophore is uniquely encoded in its diffraction pattern, and can be estimated from a single acquisition with nanometer-scale accuracy. Fluroescent molecules that remain fixed during image acquisition produce a diffraction pattern that is highly characteristic of the orientation of the fluorophore's underlying electromagnetic dipole. The localization is thus extended to incorporate the estimation of this 3-D orientation. It is shown that image formation for dipoles can be represented through a combination of six functions, based on which a 3-D steerable filter for orientation estimation and localization is derived. Experimental results demonstrate the feasibility of joint position and orientation estimation with accuracies of the order of one nanometer and one degree, respectively. Theoretical limits on localization accuracy for these methods are established in a statistical analysis based on Cramér-Rao bounds. A noise model for fluorescence microscopy based on a shifted Poisson distribution is proposed. In these localization methods, the aberration parameters of the PSF are assumed to be known. However, state-of-the-art PSF models depend on a large number of parameters that are generally difficult to determine experimentally with sufficient accuracy, which may limit their use in localization and deconvolution applications. A fitting algorithm analogous to localization is proposed; it is based on a simplified PSF model and shown to accurately reproduce the behavior of shift-variant experimental PSFs. Finally, it is shown that these algorithms can be adapted to more complex experiments, such as imaging of thick samples in brightfield microscopy. An extended-depth-of-field algorithm for the fusion of in-focus information from frames acquired at different focal positions is presented; it relies on a spline representation of the sample surface and as a result yields a continuous and super-resolved topography of the specimen.

EPFL2009