Spline-Based Image-to-Volume Registration for Three-Dimensional Electron Microscopy

This paper presents an algorithm based on a continuous framework for a posteriori angular and translational assignment in three-dimensional electron microscopy (3DEM) of single particles. Our algorithm can be used advantageously to refine the assignment of standard quantized-parameter methods by registering the images to a reference 3D particle model. We achieve the registration by employing a gradient-based iterative minimization of a least-squares measure of dissimilarity between an image and a projection of the volume in the Fourier transform (FT) domain. We compute the FT of the projection using the central-slice theorem (CST). To compute the gradient accurately, we take advantage of a cubic B-spline model of the data in the frequency domain. To improve the robustness of the algorithm, we weight the cost function in the FT domain and apply a “mixed” strategy for the assignment based on the minimum value of the cost function at registration for several different initializations. We validate our algorithm in a fully controlled simulation environment. We show that the mixed strategy improves the assignment accuracy; on our data, the quality of the angular and translational assignment was better than 2 voxel (i.e., 6.54 Å). We also test the performance of our algorithm on real EM data. We conclude that our algorithm outperforms a standard projection-matching refinement in terms of both consistency of 3D reconstructions and speed.

Spline projection-based volume-to-image registration

Slavica Jonic

This thesis focuses on the rigid-body registration of a three-dimensional model of an object to a set of its two-dimensional projections. The main contribution is the development of two registration algorithms that use a continuous model of the volume based on splines, either in the space domain or in the frequency domain. This allows for a well-defined gradient of the dissimilarity measure, which is a necessary condition for efficient and accurate registration. The first part of the thesis contains a review of the literature on volume-to- image registration. Then, we discuss data interpolation in the space domain and in the frequency domain. The basic concepts of our registration strategy are given in the second part of the thesis. We present a novel one-step approach for fast ray casting to simulate space-based volume projections. We also discuss the use of the central-slice theorem to simulate frequency-based volume projections. Then, we consider the question of the registration robustness. To improve the robustness of the space-based approach, we apply a multiresolution optimization strategy where spline-based data pyramids are processed in coarse-to-fine fashion, which improves speed as well. To improve the robustness of the frequency-based registration, we apply a coarse-to-fine strategy that involves weights in the frequency domain. In the third part, we apply our space-based algorithm to computer-assisted orthopedic surgery while adapting it to the perspective projection model. We show that the registration accuracy achieved using the orthopedic data is consistent with the current standards. Then, we apply our frequency-based registration to three-dimensional electron-microscopy application. We show that our algorithm can be used to obtain a refined solution with respect to currently available algorithms. The novelty of our approach is in dealing with a continuous space of geometric parameters, contrary to the standard methods which deal with quantized parameters. We conclude that our continuous parameter space leads to better registration accuracy. Last, we compare the performance of the frequency-based algorithm with that of the space-based algorithm in the context of electron microscopy. With these data, we observe that frequency-based registration algorithm outperforms the space-based one, which we attribute to the suitability of interpolation in the frequency domain when dealing with strictly space-limited data.

EPFL2003

Superresolution images reconstructed from aliased images

Sabine Süsstrunk, Patrick Vandewalle, Martin Vetterli

In this paper, we present a simple method to almost quadruple the spatial resolution of aliased images. From a set of four low resolution, undersampled and shifted images, a new image is constructed with almost twice the resolution in each dimension. The resulting image is aliasing-free. A small aliasing-free part of the frequency domain of the images is used to compute the exact subpixel shifts. When the relative image positions are known, a higher resolution image can be constructed using the Papoulis-Gerchberg algorithm. The proposed method is tested in a simulation where all simulation parameters are well controlled, and where the resulting image can be compared with its original. The algorithm is also applied to real, noisy images from a digital camera. Both experiments show very good results.

2003

Super-resolution from unregistered aliased images

Patrick Vandewalle

Aliasing in images is often considered as a nuisance. Artificial low frequency patterns and jagged edges appear when an image is sampled at a too low frequency. However, aliasing also conveys useful information about the high frequency content of the image, which is exploited in super-resolution applications. We use a set of input images of the same scene to extract such high frequency information and create a higher resolution aliasing-free image. Typically, there is a small shift or more complex motion between the different images, such that they contain slightly different information about the scene. Super-resolution image reconstruction can be formulated as a multichannel sampling problem with unknown offsets. This results in a set of equations that are linear in the unknown signal coefficients but nonlinear in the offsets. This thesis concentrates on the computation of these offsets, as they are an essential prerequisite for an accurate high resolution reconstruction. If a part of the image spectra is free of aliasing, the planar shift and rotation parameters can be computed using only this low frequency information. In such a case, the images can be registered pairwise to a reference image. Such a method is not applicable if the images are undersampled by a factor of two or larger. A higher number of images needs to be registered jointly. Two subspace methods are discussed for such highly aliased images. The first approach is based on a Fourier description of the aliased signals as a sum of overlapping parts of the spectrum. It uses a rank condition to find the correct offsets. The second one uses a more general expansion in an arbitrary Hilbert space to compute the signal offsets. The sampled signal is represented as a linear combination of sampled basis functions. The offsets are computed by projecting the signal onto varying subspaces. Under certain conditions, in particular for bandlimited signals, the nonlinear super-resolution equations can be written as a set of polynomial equations. Using Buchberger's algorithm, the solution can then be computed as a Gröbner basis for the corresponding polynomial ideal. After a description of a standard algorithm, adaptations are made for the use with noisy measurements. The techniques presented in this thesis are tested in simulations and practical experiments. The experiments are performed on sets of real images taken with a digital camera. The results show the validity of the algorithms: registration parameters are computed with subpixel precision, and aliasing is accurately removed from the resulting high resolution image. This thesis is produced according to the concepts of reproducible research. All the results and examples used in this thesis are reproducible using the code and data available online.