3D reconstruction from multiple images

3D reconstruction from multiple images is the creation of three-dimensional models from a set of images. It is the reverse process of obtaining 2D images from 3D scenes. The essence of an image is a projection from a 3D scene onto a 2D plane, during which process the depth is lost. The 3D point corresponding to a specific image point is constrained to be on the line of sight. From a single image, it is impossible to determine which point on this line corresponds to the image point. If two images are available, then the position of a 3D point can be found as the intersection of the two projection rays. This process is referred to as triangulation. The key for this process is the relations between multiple views which convey the information that corresponding sets of points must contain some structure and that this structure is related to the poses and the calibration of the camera. In recent decades, there is an important demand for 3D content for computer graphics, virtual real

Advanced Techniques in Optical Diffraction Tomography

Ahmed Bassam Sayed Ayoub Mohamed Emam

In this thesis, we study the 3 challenges described above. First, we study different reconstruction techniques and assess the fidelity of each reconstruction results by means of structured illumination and phase conjugation. By reconstructing the 3D refractive index of the sample using different algorithms (i.e. Born, Rytov, and Radon) and then perform a numerical back-propagation of experimentally measured structured illumination pattern we are able to assess the fidelity of each reconstruction algorithms without prior information about the 3D RI distribution of the sample.The second part of the thesis is concerned with the 3D reconstruction of samples using intensity-only measurements which the need to holographically acquire them. We show that using intensity-only measurements, we could still be able to reconstruct the 3D volume of the sample with edge-enhanced effects which was proven useful for drug delivery applications in which nano-particles were identified on the cell membrane of immune T-cells in a drug delivery studies. Such reconstruction technique would result in more robust imaging system where the commercial imaging microscope systems can be incorporated with LEDs for high-quality speckle noise-free imaging systems. In addition, we show that under certain conditions, we can be able to reconstruct the 3D refractive index distribution of different samples.The third part of the thesis is contributing to high-speed complex wave-front shaping using DMDs. In that part, new modulation technique is demonstrated that can boost the speed of the current time-multiplexing techniques by a factor of 32. The modulation technique is based on amplitude modulation where an amplitude modulator is synchronized withvthe DMD to modulate the intensity of each bit-plane of an 8-bit image and then all the modulated bit-planes are linearly added on the detector. Such modulation technique can be used not only for structured illumination microscopy but also for high-speed 3D printing applications as well as projectors.The last part is concerned with using deep learning approaches to solve the missing cone problem usually accompanied with optical imaging due to the limited numerical aperture of the imaging system. Two techniques are discussed; the first is based on using a physical model to enhance the quality of the 3D RI reconstruction and the second is based on using deep neural network to solve the missing cone problem.

EPFL2022

3D Reconstruction of Optical Diffraction Tomography Based on a Neural Network Model

Morteza Hasani Shoreh

Optical tomography has been widely investigated for biomedical imaging applications. In recent years, it has been combined with digital holography and has been employed to produce high quality images of phase objects such as cells. In this Thesis, we look into some of the newest optical Diffraction Tomography (DT) based techniques to solve Three-Dimensional (3D) reconstruction problems and discuss and compare some of the leading ideas and papers. Then we propose a neural-network-based algorithm to solve this problem and apply it on both synthetic and biological samples. Conventional phase tomography with coherent light and off axis recording is performed. The Beam Propagation Method (BPM) is used to model scattering and each x-y plane is modeled by a layer of neurons in the BPM. The network's output (simulated data) is compared to the experimental measurements and the error is used for correcting the weights of the neurons (the refractive indices of the nodes) using standard error back-propagation techniques. The proposed algorithm is detailed and investigated. Then, we look into resolution-conserving regularization and discuss a method for selecting regularizing parameters. In addition, the local minima and phase unwrapping problems are discussed and ways of avoiding them are investigated. It is shown that the proposed learning tomography (LT) achieves better performance than other techniques such as, DT especially when insufficient number or incomplete set of measurements is available. We also explore the role of regularization in obtaining higher fidelity images without losing resolution. It is experimentally shown that due to overcoming multiple scattering, the LT reconstruction greatly outperforms the DT when the sample contains two or more layers of cells or beads. Then, reconstruction using intensity measurements is investigated. 3D reconstruction of a live cell during apoptosis is presented in a time-lapse format. At the end, we present a final comparison with leading papers and commercially available systems. It is shown that -compared to other existing algorithms- the results of the proposed method have better quality. In particular, parasitic granular structures and the missing cone artifact are improved. Overall, the perspectives of our approach are pretty rich for high-resolution tomographic imaging in a range of practical applications.

EPFL2018

Transparent and Specular Object Reconstruction with a Plenoptic Camera

Ying Song

Over the past two decades, most techniques that have been developed are focused on opaque objects with Lambertian reflectance. For the transparent and specular objects, they have more complex light transport effect, which pose challenging problems for the current acquisition systems. In this project, we focus on the current transparent and specular objects reconstruction algorithms. A final algorithm is expected to fully use the advantages of the light field camera and achieve reasonable reconstruction results with a simple acquisition framework.

2015