Stereo-vision three-dimensional reconstruction of curvilinear structures imaged with a TEM

Deriving accurate three-dimensional (3-D) structural information of materials at the nanometre level is often crucial for understanding their properties. Tomography in transmission electron microscopy (TEM) is a powerful technique that provides such information. It is however demanding and sometimes inapplicable, as it requires the acquisition of multiple images within a large tilt arc and hence prolonged exposure to electrons. In some cases, prior knowledge about the structure can tremendously simplify the 3-D reconstruction if incorporated adequately. Here, a novel algorithm is presented that is able to produce a full 3-D reconstruction of curvilinear structures from stereo pair of TEM images acquired within a small tilt range that spans from only a few to tens of degrees. Reliability of the algorithm is demonstrated through reconstruction of a model 3-D object from its simulated projections, and is compared with that of conventional tomography. This method is experimentally demonstrated for the 3-D visualization of dislocation arrangements in a deformed metallic micro-pillar. (C) 2017 Published by Elsevier B.V.

Structure-aware Multi-view 3D Reconstruction of Dislocations in TEM with Message Passing Neural Networks

Okan Altingövde, Pascal Fua, Gulnaz Ganeeva, Cécile Hébert, Anastasiia Mishchuk, Emadeddin Oveisi

Efficient analysis of the three-dimensional (3D) shape and distribution of curvilinear crystal defects, namely dislocations, is an open research topic in material science and computer vision. In order to determine the structural and opto-electrical characteristics of the material, accurate 3D reconstructions of dislocations are required. In its current state, an established way to obtain 3D information of dislocations is electron tomography which relies on acquiring dozens of images from tilted sample for wide range of angles. Although tomography yields sufficient results in many cases, it is manually demanding to acquire, register and process dozens of images. In previous works [1, 2], it is shown that a classical stereo reconstruction approach may be applied as an alternative in order to increase reconstruction throughput while providing good performance. A later work [3] employing modern data-driven computer vision models showed that this stereo approach can be largely automatized and be used in various imaging conditions. In this work, we further improve the performance and memory consumption of previous convolutional neural network (CNN) based stereo reconstruction approach and extend it to be applicable in cases where more than two images are available. Our proposed multi-view approach directly incorporates the structural prior of dislocations to the model"s architecture and estimates 3D line segments approximating 3D shape of dislocations. In its core, our method has two stages which are structural connectivity estimation and depth estimation. In first stage, we employ a CNN to estimate affinity of sampled image locations on dislocation cores. This affinity information is used in the later depth estimation stage to eliminate the depth discontinuities on dislocations (Figure1). Unlike previous deep data-driven method, our new approach uses structural knowledge to separate depth estimations of different dislocation segments and yields more accurate reconstructions. In Figure 2, an example affinity estimation is shown for 4 different image locations on the same dislocation. It may be seen that dislocation points that are connected by dislocation segment in 3D results in higher affinity. This information is crucial to connect 3D point estimations and obtain linear structures in 3D.

2021

Three-dimensional electron imaging of dislocations from a single sample tilt

Pascal Fua, Cécile Hébert, Antoine Letouzey, Emadeddin Oveisi

Linear crystal defects called dislocations are one of the most fascinating concepts in materials science that govern mechanical and optoelectronic properties of many materials across a broad range of application. Three-dimensional (3-D) study of dislocation network is in principle accessible by conventional tomographic and stereoscopic techniques in Transmission Electron Microscopy (TEM). In these techniques in general the need to tilt the specimen for acquiring image series over large tilt ranges remain however an intricate problem, in particular when diffraction contrast or sensitivity to electron beam are involved. Here, a novel method in scanning TEM (STEM) is presented that provides a reliable and fast assessment of the 3-D configuration of dislocations using data acquired from just one sample tilt. This technique acquires a stereoscopic pair of images by selecting different ray paths of a convergent illumination in STEM mode. The resulting images are then treated with a dedicated stereovision reconstruction algorithm, yielding a full 3-D reconstruction of dislocations arrangement. The success of this method is demonstrated by measurement of dislocation arrangements in two experimental cases.

Wiley2016

Image Coding of 3D Volume Using Wavelet Transform for Fast Retrieval of 2D Images

Vijayaraghavan Thirumalai

An encoder/decoder system for 3D volumetric medical data is proposed. The system allows fast access to any 2D image by decoding only the relevant information from each subband image and thus provides minimum decoding time. This will be of immense use for the medical community because most of the computerised tomography, magnetic resonance imaging and positron emission tomography modalities produce volumetric data. As a fully fledged 3D wavelet transform is used for compression, the advantage of good compression ratio is preserved. Preprocessing is carried out prior to wavelet transform, to enable easier identification of coefficients from each subband image. Inclusion of special characters in the bit stream (markers) facilitates access to corresponding information from the encoded data. Experiments are carried out by performing Daub4 filter along x (row) and y (column) directions and Haar filter along the z (slice) direction to account for the difference between interslice and intraslice resolution. The performance of the system has been evaluated on four sets of volumetric data and the results are compared to other 3D encoding/2D decoding schemes. Results show that for slice spacing of 3–10 mm, there is substantial improvement in decoding time. The speedup is found to be 2.

2006

Structure-aware Multi-view 3D Reconstruction of Dislocations in TEM with Message Passing Neural Networks

Okan Altingövde, Pascal Fua, Gulnaz Ganeeva, Cécile Hébert, Anastasiia Mishchuk, Emadeddin Oveisi

2021