Application of CMOS image sensors in optical sectioning and polarization microscopy

In this thesis new imaging approaches for optical microscopy are proposed and studied. They are based on the use of dynamic structured illumination in combination with a demodulation detection concept employing CMOS image detectors. Two particular implementations are suggested: real-time optical sectioning microscopy and whole field quantitative polarization microscopy. Optical sectioning, i.e depth resolved imaging, allows observation of thin sections in volumetric samples. In its classical configuration the wide-field microscope does not allow optical sectioning of the investigated objects. In this thesis the optical sectioning in a wide-field microscope is achieved by illuminating the sample with a continuously moving single spatial frequency pattern, which temporarily modulates the signal obtained on the photodetector. Only the object parts that are in the focus have a good contrast and consequently generate stronger signal on the detector. The optically sectioned images are obtained employing a CMOS image detector, which demodulates the time-dependent component of the image. Two different systems for real-time acquisition of optically sectioned images are proposed. The first one is based on a specially designed smart-pixel-detector-array (SPDA), which allows performing the signal processing by an electronic circuit integrated to each pixel of the sensor. The second system utilizes a commercial CMOS image sensor. Here, the images are treated by a digital signal processor (DSP) integrated into the camera (iMVS-155). Both approaches provide real-time acquisition of optically sectioned images. The proof of principle for the new method is demonstrated by imaging artificial three-dimensional reflective samples. The optical sectioning imaging of biological samples in bright-field mode and in fluorescence mode is also demonstrated. The optical sectioning performances of the studied systems are similar to that of the confocal microscope. However the new method has some advantages over the classical confocal system: e.g. the difficulties with the alignment and the post-processing inherent to the confocal system are avoided. The theoretical framework presented in the thesis describes the image formation in structured illumination for both reflective and fluorescent samples. The influence of longitudinal chromatic aberration and spherical aberration caused by specimen induced index mismatch on depth discrimination property are studied. In extension to the work related to optically sectioned microscopy, the quantitative polarization microscopy imaging method is proposed as a new application for CMOS detectors. The polarization state of the illuminating light is dynamically changed and the demodulation detection approach is employed to extract the retardance and azimuth of the birefringent sample in real-time; for a whole field of view. Examples of polarization sensitive measurements for different samples are presented. The theoretical analysis is performed using the Jones formalism. Finally, the potentials and limitations of the CMOS detectors for applications in optical microscopy are discussed. Thanks to constant advancements in CMOS technology, the acquisition speed, resolution and sensitivity of new CMOS detectors generations have been improved. This will allow adapting the methods proposed in this thesis for investigations of fast dynamic process where high temporal and spatial resolution is required.

Imaging tissues and cells beyond the diffraction limit with structured illumination microscopy and Bayesian image reconstruction

Tomas Lukes, Jakub Pospisil

Background: Structured illumination microscopy (SIM) is a family of methods in optical fluorescence microscopy that can achieve both optical sectioning and super-resolution effects. SIM is a valuable method for high-resolution imaging of fixed cells or tissues labeled with conventional fluorophores, as well as for imaging the dynamics of live cells expressing fluorescent protein constructs. In SIM, one acquires a set of images with shifting illumination patterns. This set of images is subsequently treated with image analysis algorithms to produce an image with reduced out-of-focus light (optical sectioning) and/or with improved resolution (super-resolution). Findings: Five complete, freely available SIM datasets are presented including raw and analyzed data. We report methods for image acquisition and analysis using open-source software along with examples of the resulting images when processed with different methods. We processed the data using established optical sectioning SIM and super-resolution SIM methods and with newer Bayesian restoration approaches that we are developing. Conclusions: Various methods for SIM data acquisition and processing are actively being developed, but complete raw data from SIM experiments are not typically published. Publically available, high-quality raw data with examples of processed results will aid researchers when developing new methods in SIM. Biologists will also find interest in the high-resolution images of animal tissues and cells we acquired. All of the data were processed with SIMToolbox, an open-source and freely available software solution for SIM.

Oxford University Press (OUP)2019

Biomedical optical techniques for intracochlear cellular imaging

Marilisa Romito

Imaging the inner ear microanatomy is of great importance for the assessment of the state of the cells responsible for sound detection. Hearing disorders are mainly due to a malfunction of the cochlea, the bone containing the cells inside the inner ear. Under this situation, using suitable imaging techniques would be extremely helpful in detecting disease and diagnosing pathologies of the inner ear. However, cochlear small size and encasement in bone provide challenging obstacles, preventing visualization of intracochlear microanatomy using standard clinical imaging modalities. In this thesis, we explore optical techniques to image inner ear cells and we report an innovative method to observe intracochlear structures through the scattering cochlear bone. Firstly, we report different imaging techniques we used for intracochlear cellular investigation, starting with conventional optical microscopy. Two-photon excitation fluorescence (TPEF) microscopy is the most suitable technique for high quality images of the hair cells within the organ of Corti. Our results from extracted mouse organ of Corti show that we are able to distinguish between healthy and damaged sensory cells and nerve fibers, analyzing TPEF images from different mouse samples: young, old, exposed to noise and not exposed to noise. We then describe different noninvasive methods that have been used to image intact cochleae, such as Optical Coherence Tomography (OCT), X-ray Computed Tomography (X-Ray CT), micro-Computed Tomography (X-Ray µCT) and Optical Diffraction Tomography (ODT). These techniques allow the observation of the internal ear anatomy, despite the bone. We report tomographic volumetric reconstructions of mouse cochlea and we compare them in terms of achievable image resolution. µ-CT is the most appropriate techniques providing both penetration through bone and high-level resolution images down to 2µm. However, the high dose of radiation used in µCT studies prevent translation of these techniques to living humans. Due to the discussed challenges for imaging the hair cells through the highly scattering bone, the aim of my thesis is to develop a new method for intracochlear cellular imaging with minimum trauma. The last chapter represents the core of this PhD project. In this chapter, after considering the advantages and disadvantages of all the possible imaging techniques, we report our innovative technique for visualization of cochlear cells through the overlying scattering bone by combining femtosecond laser bone ablation and TPEF microscopy. The controlled ultrafast laser ablation reduces the optical scattering of the cochlear bone while the TPEF provides a contrast mechanism to resolve individual cells behind the bone. We implemented a simultaneous OCT with the laser ablation to enhance the precision of ablation and prevent inadvertent violation of the delicate cells hidden behind bone. An additional bright-field camera shows real-time images of the sample. We demonstrate that our approach improves the light focusing capability through the cochlear bone, allowing imaging of intracochlear structures in intact cochleae with high resolution. This extremely highly promising approach can be further developed for clinical ablation instruments and endoscopes that can reach the cochlea and produce images of cells within the inner ear to establish precise diagnosis, guide treatment and assess response to therapies, which is not yet possible in the clinic.

EPFL2019

Computational methods for live heart imaging with speed-constrained microscopes

Olivia Mariani

Imaging methods to capture the beating and developing heart inside embryonic animal models such as the zebrafish are a key component for the study of fundamental biological processes such as cardiac birth defects or tissue regeneration. However, live heart imaging is subject to many challenges that limit the achievable spatial and temporal resolutions. While many commonly available microscopes offer the necessary optical sectioning capabilities to produce slices or volumes at the required scale, they often lack the necessary speed to seamlessly resolve the beating heart within a still frame, the entire heartbeat, or over a complete developmental sequence. In this thesis, I aim to assemble dynamic image series of the heart by using a wide range of microscopes, including, in particular, scanning microscopes---which are widely available but slow---and microscopes with a slow framerate. First, I considered the problem of reconstructing an image sequence covering one heartbeat from still images arbitrarily triggered throughout multiple cardiac cycles, when the underlying cardiac motion is temporally asymmetrical. I proposed to sort the images such as to maximize the overall similarity between contiguous frames, which I formulated as a TSP. The performance of the method increases with the number of images. I reconstructed image sequences of the heart at a virtual frame-rate of up to 300 frames per second. Second, to resolve the ill-posedness of the sorting problem when identical object poses appear multiple times in one cycle, I proposed to illuminate alternate frames with a temporally-asymmetric pattern. Sorting issues are mitigated at the cost of a custom setup and additional light exposure, without adversely affecting the reconstructed sequence. Third, I considered the problem of imaging the beating heart on conventional scanning confocal microscopes (LSM) that have a frame scanning rate comparable to the heartbeat rate. My approach takes a set of arbitrarily triggered images that may contain scanning aberrations and assembles it into a sequence that covers one heartbeat. Reconstructing scanning-aberrated frames offers a robust alternative for LSM. The quality of recovered cardiac heartbeat dynamics is comparable to that obtained by fast microscopes. Finally, I considered the problem of building smooth time-lapse sequences in the cardiac phase versus development or imaging depth spaces, based on a set of still images captured at weakly-constrained times within the heartbeat and over the course of development or at various depths. This method determines a smooth path in cardiac phase-development or depth space enforcing neighborhood similarity via a within-stage (or within-depth) sorting followed by a cyclical least-squares phase alignment approach that I formulated as a mixed integer linear problem. Constraining the paths of time-lapses by providing key-points while enforcing local similarity successfully stabilizes the the excursion error from which recursive registration approaches suffer. The results suggest that reliably imaging the heart on a wide range of microscopes is possible, provided the heart motions are stereotypical and that minimal sampling constraints are met. The newly gained possibility of using conventional instruments may be particularly relevant for applications with emerging modalities that do not allow wide-field capture.