3D Reconstruction of Optical Diffraction Tomography Based on a Neural Network Model

Optical tomography has been widely investigated for biomedical imaging applications. In recent years, it has been combined with digital holography and has been employed to produce high quality images of phase objects such as cells. In this Thesis, we look into some of the newest optical Diffraction Tomography (DT) based techniques to solve Three-Dimensional (3D) reconstruction problems and discuss and compare some of the leading ideas and papers. Then we propose a neural-network-based algorithm to solve this problem and apply it on both synthetic and biological samples. Conventional phase tomography with coherent light and off axis recording is performed. The Beam Propagation Method (BPM) is used to model scattering and each x-y plane is modeled by a layer of neurons in the BPM. The network's output (simulated data) is compared to the experimental measurements and the error is used for correcting the weights of the neurons (the refractive indices of the nodes) using standard error back-propagation techniques. The proposed algorithm is detailed and investigated. Then, we look into resolution-conserving regularization and discuss a method for selecting regularizing parameters. In addition, the local minima and phase unwrapping problems are discussed and ways of avoiding them are investigated. It is shown that the proposed learning tomography (LT) achieves better performance than other techniques such as, DT especially when insufficient number or incomplete set of measurements is available. We also explore the role of regularization in obtaining higher fidelity images without losing resolution. It is experimentally shown that due to overcoming multiple scattering, the LT reconstruction greatly outperforms the DT when the sample contains two or more layers of cells or beads. Then, reconstruction using intensity measurements is investigated. 3D reconstruction of a live cell during apoptosis is presented in a time-lapse format. At the end, we present a final comparison with leading papers and commercially available systems. It is shown that -compared to other existing algorithms- the results of the proposed method have better quality. In particular, parasitic granular structures and the missing cone artifact are improved. Overall, the perspectives of our approach are pretty rich for high-resolution tomographic imaging in a range of practical applications.

Advanced Techniques in Optical Diffraction Tomography

Ahmed Bassam Sayed Ayoub Mohamed Emam

In this thesis, we study the 3 challenges described above. First, we study different reconstruction techniques and assess the fidelity of each reconstruction results by means of structured illumination and phase conjugation. By reconstructing the 3D refractive index of the sample using different algorithms (i.e. Born, Rytov, and Radon) and then perform a numerical back-propagation of experimentally measured structured illumination pattern we are able to assess the fidelity of each reconstruction algorithms without prior information about the 3D RI distribution of the sample.The second part of the thesis is concerned with the 3D reconstruction of samples using intensity-only measurements which the need to holographically acquire them. We show that using intensity-only measurements, we could still be able to reconstruct the 3D volume of the sample with edge-enhanced effects which was proven useful for drug delivery applications in which nano-particles were identified on the cell membrane of immune T-cells in a drug delivery studies. Such reconstruction technique would result in more robust imaging system where the commercial imaging microscope systems can be incorporated with LEDs for high-quality speckle noise-free imaging systems. In addition, we show that under certain conditions, we can be able to reconstruct the 3D refractive index distribution of different samples.The third part of the thesis is contributing to high-speed complex wave-front shaping using DMDs. In that part, new modulation technique is demonstrated that can boost the speed of the current time-multiplexing techniques by a factor of 32. The modulation technique is based on amplitude modulation where an amplitude modulator is synchronized withvthe DMD to modulate the intensity of each bit-plane of an 8-bit image and then all the modulated bit-planes are linearly added on the detector. Such modulation technique can be used not only for structured illumination microscopy but also for high-speed 3D printing applications as well as projectors.The last part is concerned with using deep learning approaches to solve the missing cone problem usually accompanied with optical imaging due to the limited numerical aperture of the imaging system. Two techniques are discussed; the first is based on using a physical model to enhance the quality of the 3D RI reconstruction and the second is based on using deep neural network to solve the missing cone problem.

EPFL2022

Shot noise, refractive index tomography and aberrations estimation in digital holographic microscopy

Florian Charrière

The present thesis develops some specific aspects of digital holographic microscopy (DHM), namely the effect of shot noise on the phase image accuracy, the use of DHM in micro-tomography and in aberrations evaluation of a microscope objective (MO). DHM is an imaging technique, allowing to measure quantitatively the wavefront transmitted through or reflected by a specimen seen through a MO. A hologram, composed by the interference of the wave coming from the object with a reference wave, is recorded with a camera and then numerically processed to extract both amplitude and phase information. Thanks to its interferometric nature, DHM provides phase images, corresponding to a nanometric accuracy along the optical axis of the microscope, revealing extremely detailed information about the specimen surface in reflection configuration or its internal structure in transmission configuration. DHM has proven its efficiency on numerous applications fields going from cells biology to MEMS-MOEMS devices. In a first part, the use of DHM as metrological tools in the field of micro-optics testing is demonstrated. DHM measurement principle is compared with techniques employed in Twyman-Green, Mach-Zehnder, and white-light interferometers. Refractive microlenses are characterized with reflection DHM and the data are confronted with data obtained with standard interferometers. Specific features of DHM such as digital focussing, measurement of shape differences with respect to a perfect model, surface roughness measurements, and evaluation of a lens optical performance are discussed. The capability to image nonspherical lenses without modification of the optical setup, a key advantage of DHM against conventional interferometers, is demonstrated on a cylindrical mircrolens and a square lenses array. A second part treats the effect of shot noise in DHM. DHM is a single shot imaging technique, and its short hologram acquisition time (down to microseconds) offers a reduced sensitivity to vibrations. Real time observation is achievable, thanks to present performances of personal computers and digital camera. Fast dynamic imaging at low-light level involves few photons, requiring proper settings of the system (integration time and gain of the camera; power of the light source) to minimize the influence of shot noise on the hologram when the highest phase accuracy is aimed. With simulated and experimental data, a systematic analysis of the fundamental shot noise influence on phase accuracy in DHM is presented. Different configurations of the reference wave and the object wave intensities are also discussed, illustrating the detection limit and the coherent amplification of the object wave. In a third part, DHM has for the first time been applied to perform optical diffraction tomography of biological specimens: a pollen grain and living amoebas. Quantitative 2D phase images are acquired for regularly-spaced angular positions of the specimen covering a total angle of π, allowing to build 3D quantitative refractive index distributions by an inverse Radon transform. A precision of 0.01 for the refractive index estimation and a spatial resolution in the micron range are shown. For the amoebas, morphometric measurements are extracted from the tomographic reconstructions. The fourth part presents a DHM technique to determine the integral refractive index and morphology of cells. As the refractive index is a function of the cell dry mass, depending on the intra-cellular concentration and the organelles arrangement, the optical phase shift induced by the specimen on the transmitted wave can be regarded as a powerful endogenous contrast agent. The dual-wavelengths technique proposed in this thesis exploits the dispersion of the perfusion medium to obtain a set of equations, allowing decoupling the contributions of the refractive index and the cellular thickness to the total phase signal. The two wavelengths are chosen in the vicinity of the absorption peak of a dye added to the perfusion medium, where the absorption is accompanied by a strong variation of the refractive index as a function of the wavelength. The technique is demonstrated on yeasts. The last part exposes two methods capable of measuring the complex 3D amplitude point spread function (APSF) of an optical imaging system. The first approach consists in evaluating in amplitude and phase the image of a single emitting point, a 60 nm diameter tip of a Scanning Near Field Optical Microscopy (SNOM) fiber, with an original digital holographic setup. A single hologram giving access to the transverse APSF, the 3D APSF is obtained by performing an axial scan of the SNOM fiber. The method is demonstrated on an 20x 0.4 NA MO. For a 100x 1.3 NA MO, measurements performed with the new setup are compared with the prediction of an analytical aberrations model. The second method allows measuring the APSF of a MO with a single holographic acquisition of its pupil wavefront. The aberration function is extracted from this pupil measurement and then inserted in a scalar model of diffraction allowing to calculate the distribution of the complex wavefront propagated around the focal point. The results are compared with a direct measurement of the APSF achieved with the first proposed approach.

EPFL2007

Learning approaches to high-fidelity optical diffraction tomography

Joowon Lim

Optical diffraction tomography (ODT) provides us 3D refractive index (RI) distributions of transparent samples. Since RI values differ across different materials, they serve as endogenous contrasts. It, therefore, enables us to image without pre-processing of labeling which can disturb samples during measurement. It has been utilized in various applications to study hematology, morphological parameters, biochemical information, and so on. The fundamental principle of ODT reconstruction is to recover the 3D information from multiple 2D measurements. While we require 2D measurements acquired by fully scanning a sample, there exist missing measurements that we are not able to access due to the limited numerical apertures (NAs) in the optical system. This is called the missing cone problem since the parts which are not covered by the NAs form cone shapes. The missing cone problem degrades the final reconstruction by underestimating RI values and more severely elongating images along the optical axis. Another challenge in ODT reconstruction is to model the nonlinear relationship between a sample and the measurements. The first order of scattering is commonly considered while neglecting the other higher orders to linearize the relationship, however, this results in degradation of the final reconstruction as the higher orders of scattering become more pronounced. In this thesis, we aim at solving the challenges in ODT reconstruction to provide more accurate quantitative information, namely, RI distributions. The first approach is based on model-based iterative reconstruction schemes. We choose the beam propagation method (BPM) for the forward model in order to capture the high orders of scattering. Due to the similarity of the multi-layer structure of the BPM with that of neural networks used in deep learning, we call this scheme learning tomography (LT). We rigorously investigate the performance of LT over the conventional linear model-based reconstruction scheme. Furthermore, by applying a more advanced BPM for the forward model, we even improve the LT and demonstrate the dramatically improved performance by both simulations and experiments. The second approach is based on statistically learning artifacts present in final reconstructions using a deep neural network (DNN) from a large dataset. Unlike the previous approaches which require iterations, the DNN approach instantly reconstructs RI distributions. We demonstrate the use of DNN using red blood cells which are highly distorted by the missing cone problem. In order to overcome the lack of ground truth in 3D ODT reconstruction, we digitally generate a synthetic dataset. The reconstruction results from the network present highly accurate results for the synthetic test set. Most importantly, we obtain high-fidelity reconstructions of experimental data using the network trained only on the synthetic data. Unlike other imaging modalities, ODT provides 3D quantitative information without labeling. To fully benefit from the capacity of quantitative imaging, it is critical to solve the existing challenges in ODT reconstruction to produce high-fidelity reconstructions. In this contribution, we aim to resolve the major challenges in ODT reconstruction using various learning approaches, and we believe that it can further improve ODT as a powerful tool for various applications.

EPFL2020