Structured sparsity regularization

Structured sparsity regularization is a class of methods, and an area of research in statistical learning theory, that extend and generalize sparsity regularization learning methods. Both sparsity and structured sparsity regularization methods seek to exploit the assumption that the output variable Y (i.e., response, or dependent variable) to be learned can be described by a reduced number of variables in the input space X (i.e., the domain, space of features or explanatory variables). Sparsity regularization methods focus on selecting the input variables that best describe the output. Structured sparsity regularization methods generalize and extend sparsity regularization methods, by allowing for optimal selection over structures like groups or networks of input variables in X . Common motivation for the use of structured sparsity methods are model interpretability, high-dimensional learnin

Sparsity-Inducing Reconstruction Methods for Fluorescence Diffuse Optical Tomography

Jean-Charles Baritaux

The recent advances in fluorescent molecular probes, photon detection instrumentation, and photon propagation models in tissue, have facilitated the emergence of innovative molecular imaging technologies such as Fluorescence Diffuse Optical Tomography (FDOT). FDOT has already successfully been applied on its own to resolve specific molecular targets and pathways in vivo. Recent attempts have tried to integrate FDOT with structural imaging technology into a multi-modal system. This thesis is amongst the first academic-industrial partnerships to develop a hybrid X-Ray Computed Tomography/FDOT platform for small animal imaging. In this study, we focus on image reconstruction algorithms for FDOT, and consider three main research directions. First, we investigate innovative discretization techniques for FDOT, in the context of variational reconstruction methods. Our goal is to treat in a consistent manner the propagation equation, and the regularization, needed in the reconstruction algorithm. This study uses the diffusion approximation for modeling the propagation of photons in tissue. We solve the diffusion equation with a novel Finite Volumes Method that is specifically designed to be compatible with the sparsity-inducing regularization developed in the second part of this work. Our approach results in a computationally efficient implementation of the forward model. Next, we study a novel regularization approach for FDOT which is based on sparsity-promoting regularization. This approach is expected to enhance the reconstruction when the fluorescence distribution presents a sparsity pattern in a suitable representation, or after linear transformation. We propose a variational framework based on ℓp-norm penalization in order to incorporate the sparsity a priori in the inversion algorithm. We demonstrate this approach on experimental data, using computationally efficient algorithms relying on convex optimization principles. Finally, we focus on hybrid imaging systems that combine FDOT with another imaging technique. The latter is employed to acquire a high-resolution image of the anatomy of the specimen under investigation, which provides valuable information for the FDOT reconstruction. We propose a new regularization method that allows the incorporation of the anatomical information in the FDOT reconstruction. Our method is based on the concept of group sparsity, and implemented in practice using ℓ2,1-norm penalization. We extend our sparsity-inducing algorithms to this case, and study the performance of the resulting implementation on experimental data.

EPFL2012

Learning of Continuous and Piecewise-Linear Functions With Hessian Total-Variation Regularization

Shayan Aziznejad, Joaquim Gonçalves Garcia Barreto Campos, Michaël Unser

We develop a novel 2D functional learning framework that employs a sparsity-promoting regularization based on second-order derivatives. Motivated by the nature of the regularizer, we restrict the search space to the span of piecewise-linear box splines shifted on a 2D lattice. Our formulation of the infinite-dimensional problem on this search space allows us to recast it exactly as a finite-dimensional one that can be solved using standard methods in convex optimization. Since our search space is composed of continuous and piecewise-linear functions, our work presents itself as an alternative to training networks that deploy rectified linear units, which also construct models in this family. The advantages of our method are fourfold: the ability to enforce sparsity, favoring models with fewer piecewise-linear regions; the use of a rotation, scale and translation-invariant regularization; a single hyperparameter that controls the complexity of the model; and a clear model interpretability that provides a straightforward relation between the parameters and the overall learned function. We validate our framework in various experimental setups and compare it with neural networks.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Sparsity-Inducing Reconstruction Methods for Fluorescence Diffuse Optical Tomography

Jean-Charles Baritaux

EPFL2012