Spline projection-based volume-to-image registration

This thesis focuses on the rigid-body registration of a three-dimensional model of an object to a set of its two-dimensional projections. The main contribution is the development of two registration algorithms that use a continuous model of the volume based on splines, either in the space domain or in the frequency domain. This allows for a well-defined gradient of the dissimilarity measure, which is a necessary condition for efficient and accurate registration. The first part of the thesis contains a review of the literature on volume-to- image registration. Then, we discuss data interpolation in the space domain and in the frequency domain. The basic concepts of our registration strategy are given in the second part of the thesis. We present a novel one-step approach for fast ray casting to simulate space-based volume projections. We also discuss the use of the central-slice theorem to simulate frequency-based volume projections. Then, we consider the question of the registration robustness. To improve the robustness of the space-based approach, we apply a multiresolution optimization strategy where spline-based data pyramids are processed in coarse-to-fine fashion, which improves speed as well. To improve the robustness of the frequency-based registration, we apply a coarse-to-fine strategy that involves weights in the frequency domain. In the third part, we apply our space-based algorithm to computer-assisted orthopedic surgery while adapting it to the perspective projection model. We show that the registration accuracy achieved using the orthopedic data is consistent with the current standards. Then, we apply our frequency-based registration to three-dimensional electron-microscopy application. We show that our algorithm can be used to obtain a refined solution with respect to currently available algorithms. The novelty of our approach is in dealing with a continuous space of geometric parameters, contrary to the standard methods which deal with quantized parameters. We conclude that our continuous parameter space leads to better registration accuracy. Last, we compare the performance of the frequency-based algorithm with that of the space-based algorithm in the context of electron microscopy. With these data, we observe that frequency-based registration algorithm outperforms the space-based one, which we attribute to the suitability of interpolation in the frequency domain when dealing with strictly space-limited data.

Spline-Based Image-to-Volume Registration for Three-Dimensional Electron Microscopy

Philippe Thévenaz, Michaël Unser

This paper presents an algorithm based on a continuous framework for a posteriori angular and translational assignment in three-dimensional electron microscopy (3DEM) of single particles. Our algorithm can be used advantageously to refine the assignment of standard quantized-parameter methods by registering the images to a reference 3D particle model. We achieve the registration by employing a gradient-based iterative minimization of a least-squares measure of dissimilarity between an image and a projection of the volume in the Fourier transform (FT) domain. We compute the FT of the projection using the central-slice theorem (CST). To compute the gradient accurately, we take advantage of a cubic B-spline model of the data in the frequency domain. To improve the robustness of the algorithm, we weight the cost function in the FT domain and apply a “mixed” strategy for the assignment based on the minimum value of the cost function at registration for several different initializations. We validate our algorithm in a fully controlled simulation environment. We show that the mixed strategy improves the assignment accuracy; on our data, the quality of the angular and translational assignment was better than 2 voxel (i.e., 6.54 Å). We also test the performance of our algorithm on real EM data. We conclude that our algorithm outperforms a standard projection-matching refinement in terms of both consistency of 3D reconstructions and speed.

Elsevier2005

Graph Representation Learning with Optimal Transport: Analysis and Applications

Effrosyni Simou

In several machine learning settings, the data of interest are well described by graphs. Examples include data pertaining to transportation networks or social networks. Further, biological data, such as proteins or molecules, lend themselves well to graph-structured descriptions. In such settings, it is desirable to design representation learning algorithms that can take into account the information captured by the underlying structure. This thesis focuses on providing new methods to ingrain geometrical information in end-to-end deep learning architectures for graphs through the use of elements from Optimal Transport theory.First, we consider the setting where the data can be described by a fixed graph. In this setting, each datapoint corresponds to a node of the graph and the edges among nodes signify their pairwise relations. We propose an autoencoder architecture that consists of a linear layer in the encoder and a novel Wasserstein barycentric layer at the decoder. Our proposed barycentric layer takes into account the underlying geometry through the diffusion distance of the graph and provides a way to obtain structure-aware, non-linear interpolations, which lead to directly interpretable node embeddings that are stable to perturbations of the graph structure. Further, the embeddings obtained with our proposed autoencoder achieve competitive or superior results compared to state-of-the-art methods in node classification tasks.Second, we consider the setting where the data consist of multiple graphs. In that setting, the graphs can be of varying size and, therefore, a global pooling operation is needed in order to obtain representations of fixed size and enable end-to-end learning with deep networks. We propose a global pooling operation that optimally preserves the statistical properties of the representations. In order to do so, we take into account the geometry of the representation space and introduce a global pooling layer that minimizes the Wasserstein distance between a graph representation and its pooled counterpart, of fixed size. This is achieved by performing a Wasserstein gradient flow with respect to the pooled representation. Our proposed method demonstrates promising results in the task of graph classification.Overall, in this thesis we provide new methods for incorporating geometrical information in end-to-end deep learning architectures for graph structured data. We believe that our proposals will contribute to the development of algorithms that learn meaningful representations and that take fully into account the geometry of the data under consideration.

EPFL2022

Graph Representation Learning with Optimal Transport: Analysis and Applications

Effrosyni Simou

EPFL2022