Imaging complex objects using learning tomography

Optical diffraction tomography (ODT) can be described using the scattering process through an inhomogeneous media. An inherent nonlinearity exists relating the scattering medium and the scattered field due to multiple scattering. Multiple scattering is often assumed to be negligible in weakly scattering media. This assumption becomes invalid as the sample gets more complex resulting in distorted image reconstructions. This issue becomes very critical when we image a complex sample. Multiple scattering can be simulated using the beam propagation method (BPM) as the forward model of ODT combined with an iterative reconstruction scheme. The iterative error reduction scheme and the multi-layer structure of BPM are similar to neural networks. Therefore we refer to our imaging method as learning tomography (LT). To fairly assess the performance of LT in imaging complex samples, we compared LT with the conventional iterative linear scheme using Mie theory which provides the ground truth. We also demonstrate the capacity of LT to image complex samples using experimental data of a biological cell.

Learning approaches to high-fidelity optical diffraction tomography

Joowon Lim

Optical diffraction tomography (ODT) provides us 3D refractive index (RI) distributions of transparent samples. Since RI values differ across different materials, they serve as endogenous contrasts. It, therefore, enables us to image without pre-processing of labeling which can disturb samples during measurement. It has been utilized in various applications to study hematology, morphological parameters, biochemical information, and so on. The fundamental principle of ODT reconstruction is to recover the 3D information from multiple 2D measurements. While we require 2D measurements acquired by fully scanning a sample, there exist missing measurements that we are not able to access due to the limited numerical apertures (NAs) in the optical system. This is called the missing cone problem since the parts which are not covered by the NAs form cone shapes. The missing cone problem degrades the final reconstruction by underestimating RI values and more severely elongating images along the optical axis. Another challenge in ODT reconstruction is to model the nonlinear relationship between a sample and the measurements. The first order of scattering is commonly considered while neglecting the other higher orders to linearize the relationship, however, this results in degradation of the final reconstruction as the higher orders of scattering become more pronounced. In this thesis, we aim at solving the challenges in ODT reconstruction to provide more accurate quantitative information, namely, RI distributions. The first approach is based on model-based iterative reconstruction schemes. We choose the beam propagation method (BPM) for the forward model in order to capture the high orders of scattering. Due to the similarity of the multi-layer structure of the BPM with that of neural networks used in deep learning, we call this scheme learning tomography (LT). We rigorously investigate the performance of LT over the conventional linear model-based reconstruction scheme. Furthermore, by applying a more advanced BPM for the forward model, we even improve the LT and demonstrate the dramatically improved performance by both simulations and experiments. The second approach is based on statistically learning artifacts present in final reconstructions using a deep neural network (DNN) from a large dataset. Unlike the previous approaches which require iterations, the DNN approach instantly reconstructs RI distributions. We demonstrate the use of DNN using red blood cells which are highly distorted by the missing cone problem. In order to overcome the lack of ground truth in 3D ODT reconstruction, we digitally generate a synthetic dataset. The reconstruction results from the network present highly accurate results for the synthetic test set. Most importantly, we obtain high-fidelity reconstructions of experimental data using the network trained only on the synthetic data. Unlike other imaging modalities, ODT provides 3D quantitative information without labeling. To fully benefit from the capacity of quantitative imaging, it is critical to solve the existing challenges in ODT reconstruction to produce high-fidelity reconstructions. In this contribution, we aim to resolve the major challenges in ODT reconstruction using various learning approaches, and we believe that it can further improve ODT as a powerful tool for various applications.

EPFL2020

3D Reconstruction of Optical Diffraction Tomography Based on a Neural Network Model

Morteza Hasani Shoreh

Optical tomography has been widely investigated for biomedical imaging applications. In recent years, it has been combined with digital holography and has been employed to produce high quality images of phase objects such as cells. In this Thesis, we look into some of the newest optical Diffraction Tomography (DT) based techniques to solve Three-Dimensional (3D) reconstruction problems and discuss and compare some of the leading ideas and papers. Then we propose a neural-network-based algorithm to solve this problem and apply it on both synthetic and biological samples. Conventional phase tomography with coherent light and off axis recording is performed. The Beam Propagation Method (BPM) is used to model scattering and each x-y plane is modeled by a layer of neurons in the BPM. The network's output (simulated data) is compared to the experimental measurements and the error is used for correcting the weights of the neurons (the refractive indices of the nodes) using standard error back-propagation techniques. The proposed algorithm is detailed and investigated. Then, we look into resolution-conserving regularization and discuss a method for selecting regularizing parameters. In addition, the local minima and phase unwrapping problems are discussed and ways of avoiding them are investigated. It is shown that the proposed learning tomography (LT) achieves better performance than other techniques such as, DT especially when insufficient number or incomplete set of measurements is available. We also explore the role of regularization in obtaining higher fidelity images without losing resolution. It is experimentally shown that due to overcoming multiple scattering, the LT reconstruction greatly outperforms the DT when the sample contains two or more layers of cells or beads. Then, reconstruction using intensity measurements is investigated. 3D reconstruction of a live cell during apoptosis is presented in a time-lapse format. At the end, we present a final comparison with leading papers and commercially available systems. It is shown that -compared to other existing algorithms- the results of the proposed method have better quality. In particular, parasitic granular structures and the missing cone artifact are improved. Overall, the perspectives of our approach are pretty rich for high-resolution tomographic imaging in a range of practical applications.

EPFL2018

Adaptive Mitigation: Identification of the Dynamic Drivers of Effective Policy during the COVID-19 Pandemic

Thierry Bossy

Introduction: The unprecedented speed and scale of the COVID-19 pandemic necessitated the rapid implementation of untested public health measures to mitigate the consequences of viral spread. In the 8 months that have passed since the first recognized case, an enormous amount of research has been published evaluating the efficacy of the various policies implemented by different countries, however the majority of these studies focus on a specific region or time, over-representing high-income countries during periods of extreme transmission or “peak events”. The aim of this study is to provide a more general analysis that considers a global scope of the pandemic and the dynamic drivers that build effective policies to mitigate human interactions and slow the spread of disease. Methods: We collected a range of information regarding epidemic trends, weather, demographics, government response and mobility reports across the globe. We then built an hybrid Neural Network that combined an LSTM layer with a multilayer perceptron to infer the reproduction number from various non-epidemiological factors. The model was designed to predict the reproduction number (R-value) on 93 countries with available data and compare it to our ground truth estimate obtained from officially reported epidemiological data. Finally, we used an alternative model to assess the impact of public health measures on the epidemic. Findings: From the available features, we obtained the best performances using demographics combined with mobility features. The sanitary indices (beds/thousand, diabetes prevalence, ...) did not help the prediction and, more interestingly, the pressure indicator of historical weather forecast improved the prediction of the reproduction number by about 4.5%. This optimized model predicted the reproduction number with a mean absolute error of 0.254 across the 93 countries over the time of the epidemic. For many countries (Switzerland, United Kingdom, South Africa, ...) this error passed below 0.17. An alternative version of the model allowed us to estimate the impact of policies in terms of average reduction in reproduction number, and more importantly, allowed us to compare these trends between countries. For instance, we observe that the model showed that no policy had a positive impact in India as opposed to Switzerland, where most of them are associated to improved epidemic control. Conclusion: Understanding these complex interactions may allow individuals and policy makers to better adapt mitigation strategies to optimize the efficacy of the implemented policies.

2020