Multiple sclerosis cortical and WM lesion segmentation at 3T MRI: a deep learning method based on FLAIR and MP2RAGE

The presence of cortical lesions in multiple sclerosis patients has emerged as an important biomarker of the disease. They appear in the earliest stages of the illness and have been shown to correlate with the severity of clinical symptoms. However, cortical lesions are hardly visible in conventional magnetic resonance imaging (MRI) at 3T, and thus their automated detection has been so far little explored. In this study, we propose a fully-convolutional deep learning approach, based on the 3D U-Net, for the automated segmentation of cortical and white matter lesions at 3T. For this purpose, we consider a clinically plausible MRI setting consisting of two MRI contrasts only: one conventional T2-weighted sequence (FLAIR), and one specialized T1-weighted sequence (MP2RAGE). We include 90 patients from two different centers with a total of 728 and 3856 gray and white matter lesions, respectively. We show that two reference methods developed for white matter lesion segmentation are inadequate to detect small cortical lesions, whereas our proposed framework is able to achieve a detection rate of 76% for both cortical and white matter lesions with a false positive rate of 29% in comparison to manual segmentation. Further results suggest that our framework generalizes well for both types of lesion in subjects acquired in two hospitals with different scanners.

Ocular Structures Segmentation from Multi-sequences MRI Using 3D Unet with Fully Connected CRFs

Meritxell Bach Cuadra, Alessia Pica, Raphael Sznitman

The use of 3D Magnetic Resonance Imaging (MRI) has attracted growing attention for the purpose of diagnosis and treatment planning of intraocular ocular cancers. Precise segmentation of such tumors are highly important to characterize tumors, their progression and to define a treatment plan. Along this line, automatic and effective segmentation of tumors and healthy eye anatomy would be of great value. The major challenge to this end however lies in the disease variability encountered over different populations, often imaged under different acquisition conditions and high heterogeneity of tumor characterization in location, size and appearance. In this work, we consider the Retinoblastoma disease, the most common eye cancer in children. To provide automated segmentations of relevant structures, a multi-sequences MRI dataset of 72 subjects is introduced, collected across different clinical sites with different magnetic fields (3T and 1.5T), with healthy and pathological subjects (children and adults). Using this data, we present a framework to segment both healthy and pathological eye structures. In particular, we make use of a 3D U-net CNN whereby using four encoder and decoder layers to produce conditional probabilities of different eye structures. These are further refined using a Conditional Random Field with Gaussian kernels to maximize label agreement between similar voxels in multi-sequence MRIs. We show experimentally that our approach brings state-of-the-art performances for several relevant eye structures and that these results are promising for use in clinical practice.

SPRINGER INTERNATIONAL PUBLISHING AG2018

Understanding Deep Neural Networks using Adversarial Attacks

Krishna Kanth Nakka

Deep Neural Networks (DNNs) have achieved great success in a wide range of applications, such as image recognition, object detection, and semantic segmentation. Even thoughthe discriminative power of DNNs is nowadays unquestionable, serious concerns have arised ever since DNNs have shown to be vulnerable to adversarial examples craftedby adding imperceptible perturbations to clean images. The implications of these malicious attacks are even more significant for DNNs deployed in real-world systems, e.g.,autonomous driving and biometric authentication. Consequently, an intriguing question that we aim to understand is the underlying behavior of DNNs to adversarial attacks.This thesis contributes to a better understanding of the mechanism of adversarial attacks on DNNs. Our main contributions are broadly in two directions: (1) we proposeinterpretable architectures first to understand the reasons for the success of adversarial attacks and then to improve the robustness of DNNs; (2) we design intuitive adversarialattacks to both mislead and use as a tool to expand our present understanding of DNNs' internal workings and their limitations. In the first direction, we introduce deep architectures that allow humans to interpret the reasoning process of DNNs prediction. Specifically, we incorporate Bag-of-visual-wordsrepresentations from the pre-deep learning era into DNNs using an attention scheme. We find key reasons for adversarial attack success and use these insights to propose anadversarial defense by maximally separating the latent features of discriminative regions while minimizing the contribution of non-discriminative regions in the final prediction.The second direction deals with the design of adversarial attacks to understand DNNs' limitations in a real-world environment. To begin with, we show that existing state-of-the-art semantic segmentation networks that achieve superior performance by exploiting the context are highly susceptible to indirect local attacks. Furthermore, we demonstrate the existence of universal directional perturbations that are quasi-independent of the input template but still successfully fool unknown siamese-based visual object trackers. We then identify that the mid-level filter banks across different backbones bear strong similarities and thus can be potential common ground for attack. We, therefore, learn a generator that disrupts mid-level features with high transferability across different target architectures, datasets, and tasks. In short, our attacks highlight critical vulnerabilities of DNNs, which make their deployment challenging in the real-world environment, even in the extreme case when the attacker is unaware of the target architecture or the targetdata used to train it.Furthermore, we go beyond fooling networks and demonstrate the usefulness of adversarial attacks for studying the internal disentangled representations in self-supervised 3D pose estimation networks. We observe that adversarial manipulation of appearance information in the input image alters the pose output, indicating that the pose code contains appearance information and disentanglement is far from complete. Besides the above contributions, an underlying theme that arises multiple times in this thesis is counteracting the adversarial attacks by detecting them.

EPFL2022

Object Priors for Volumetric Image Segmentation

Pamuditha Udaranga Wickramasinghe

Large training datasets have played a vital role in the success of modern deep learning methods in computer vision. But, obtaining sufficient amount of training data is challenging, specially when annotating volumetric images. This is because fully annotating a single image volume require annotating a whole stack of images which is costly. Moreover, majority of volumetric images originate from niche domains that require expert knowledge to annotate and it further increase the associated cost. Therefore, designing models for volumetric image segmentation that can work with small training datasets is of great importance. One way to address this challenge is to use prior knowledge in segmentation model design. There are many forms of prior knowledge and in this thesis, we focus on using object priors which defines prior knowledge related to the properties of the object being segmented. We begin by proposing an end-to-end differentiable architecture that produce surface meshes from input image volumes which enable easy integration of object priors related to topology and surface properties. With that we demonstrate how the priors help achieve better accuracy and quality in predictions compared to state-of-the-art methods that do not use them. Then we focus on surface priors and propose a methodology based on Active Surface models to achieve better trade-offs between the accuracy of the surfaces produced by the segmentation models and their quality. Afterwards, we apply the Active Surface model proposed earlier to develop an interactive annotation tool that significantly reduce the effort necessary to annotate image volumes. Along with that, a volumetric image segmentation model that best utilize annotations produced with the tool is also proposed. Finally we focus on object priors related to overall shape. We use Probabilistic Atlases (PAs) to introduce this prior knowledge into the segmentation model and demonstrate its ability to improve the accuracy in predictions. Overall, we use object prior to improve the accuracy, quality and the speed of annotation in volumetric image segmentation. To demonstrate their effectiveness, we use volumetric images produced by Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and Electron Microscopy (EMs).

EPFL2022

Understanding Deep Neural Networks using Adversarial Attacks

Krishna Kanth Nakka

EPFL2022

Object Priors for Volumetric Image Segmentation

Pamuditha Udaranga Wickramasinghe

EPFL2022