Combining Model Driven and Data Driven Approaches for Inverse Problems in Parameter Estimation and Image Reconstruction: From Modelling to Validation

Machine learning has become the state of the art for the solution of the diverse inverse problems arising from computer vision and medical imaging, e.g. denoising, super-resolution, de-blurring, reconstruction from scanner data, quantitative magnetic resonance imaging, etc, largely replacing the variational solutions of regularized optimization problems. However, between the two extremes of purely model-driven solutions, such as the solution of regularized optimization problems, and purely data-driven solutions, such as supervised deep learning, exist hybrid methods which combine aspects of both model-driven and data-driven solutions. Such hybrid methods are as manifold as the number of different inverse problems, as the particular characteristics of the inverse problem, e.g. availability of training data, complexity of the forward model, prior knowledge on solutions, etc., will understandably have a huge impact on the structure as well as the underlying techniques of the hybrid method. Furthermore, the validation of such approaches is also of utmost importance, particularly in medical imaging, where there are stringent requirements on the reliability of methods. In particular, hybrid methods are important when large, realistic training datasets are unavailable, such that one cannot immediately apply standard data-driven algorithms. In this thesis, we examine the solution and validation of four inverse problems derived from Magnetic Resonance Imaging (MRI) and computer vision, where we address the lack of large, realistic training datasets through solutions which try to maximally take advantage of the available model-driven and data-driven resources. We first show that self-supervised learning embedded in traditional model-driven schemes can be used to robustly solve inverse problems without ground-truth data. We conduct a rigorous validation of self-supervised methods for reconstructing MR images from raw measurement data through novel experiments on clinically relevant data, showing the importance of correct formulation of the forward model/conformity to the training data for image reconstruction quality, critically examining commonly used metrics for quantitative evaluation, and the generalization capabilities of self-supervised approaches. Furthermore, we propose embedding a neural network into a Hamiltonian Markov Chain Monte Carlo (HMCMC) sampling scheme with a self-supervised loss which improves the robustness and accuracy of solutions to a joint diffusometry/relaxometry problem, with respect to state of the art methods. Complementarily, we show that embedding realistic modeling into standard supervised learning schemes can be used to accommodate the lack of realistic, ground truth data. We combine realistic models and priors to create an extensive synthetic dataset and train a multi-layer perceptron for reconstructing $T_2$ spectra from MRI data which is more accurate, robust, and orders of magnitude less computationally expensive than the state of the art. Finally, we propose parametrizing an analytically infeasible albeit realistic downsampling model in single image super-resolution through a neural network and integrating it into arbitrary deep learning pipelines which were trained on data with an unrealistic downsampling model, achieving state of the art performance in real-world super-resolution.