Gradient-based Methods for Deep Model Interpretability

In this dissertation, we propose gradient-based methods for characterizing model behaviour for the purposes of knowledge transfer and post-hoc model interpretation. Broadly, gradients capture the variation of some output feature of the model upon unit variation of an input feature, and thus encodes the local model behaviour while being agnostic to the underlying model architectural choices.Our first contribution is to propose a sample-efficient method to mimic the behaviour of a pre-trained teacher model with an untrained student model using gradient information. We interpret our approach as an efficient alternative to data augmentation used with canonical knowledge transfer approaches, where noise is added to the inputs. We apply this to distillation and a transfer learning task, where we show improved performance for small datasets.Our second contribution is to propose a novel saliency method to visualize the input features that are most relevant for predictions made by a given model. We first propose the full-gradient representation, which satisfies a property called completeness which provably cannot be satisfied by gradient-based saliency methods. Based on this, we propose an approximate saliency map representation called FullGrad which naturally captures the information within a model across feature hierarchies. Our experimental results show that FullGrad captures model behaviour better than other saliency methods.Our final contribution is to take a step back and ask why input-gradients are informative for standard neural network models in the first place, especially when their structure may as well be arbitrary. Our analysis here reveals that for a subset of gradient-based saliency maps, the map relies not on the underlying discriminative model p(y | x) but on a hidden density model p(x | y) implicit within softmax-based disciminative models. Thus we find that the reason input-gradients are informative is due to the alignment of the implicit density model with that of the ground truth density, which we verify experimentally.

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

Our brain continuously self-organizes to construct and maintain an internal representation of the world based on the information arriving through sensory stimuli. Remarkably, cortical areas related to different sensory modalities appear to share the same functional unit, the neuron, and develop through the same learning mechanism, synaptic plasticity. It motivates the conjecture of a unifying theory to explain cortical representational learning across sensory modalities. In this thesis we present theories and computational models of learning and optimization in neural networks, postulating functional properties of synaptic plasticity that support the apparent universal learning capacity of cortical networks. In the past decades, a variety of theories and models have been proposed to describe receptive field formation in sensory areas. They include normative models such as sparse coding, and bottom-up models such as spike-timing dependent plasticity. We bring together candidate explanations by demonstrating that in fact a single principle is sufficient to explain receptive field development. First, we show that many representative models of sensory development are in fact implementing variations of a common principle: nonlinear Hebbian learning. Second, we reveal that nonlinear Hebbian learning is sufficient for receptive field formation through sensory inputs. A surprising result is that our findings are independent of specific details, and allow for robust predictions of the learned receptive fields. Thus nonlinear Hebbian learning and natural statistics can account for many aspects of receptive field formation across models and sensory modalities. The Hebbian learning theory substantiates that synaptic plasticity can be interpreted as an optimization procedure, implementing stochastic gradient descent. In stochastic gradient descent inputs arrive sequentially, as in sensory streams. However, individual data samples have very little information about the correct learning signal, and it becomes a fundamental problem to know how many samples are required for reliable synaptic changes. Through estimation theory, we develop a novel adaptive learning rate model, that adapts the magnitude of synaptic changes based on the statistics of the learning signal, enabling an optimal use of data samples. Our model has a simple implementation and demonstrates improved learning speed, making this a promising candidate for large artificial neural network applications. The model also makes predictions on how cortical plasticity may modulate synaptic plasticity for optimal learning. The optimal sampling size for reliable learning allows us to estimate optimal learning times for a given model. We apply this theory to derive analytical bounds on times for the optimization of synaptic connections. First, we show this optimization problem to have exponentially many saddle-nodes, which lead to small gradients and slow learning. Second, we show that the number of input synapses to a neuron modulates the magnitude of the initial gradient, determining the duration of learning. Our final result reveals that the learning duration increases supra-linearly with the number of synapses, suggesting an effective limit on synaptic connections and receptive field sizes in developing neural networks.

EPFL2016

Text Detection and Recognition in Images and Videos

Hervé Bourlard, Datong Chen, Jean-Marc Odobez

Text embedded in images and videos represents a rich source of information for content-based indexing and retrieval applications. In this paper, we present a new method for localizing and recognizing text in complex images and videos. Text localization is performed in a two step approach that combines the speed of a focusing step with the strength of a machine learning based text verification step. The experiments conducted show that the support vector machine is more appropriate for the verification task than the more commonly used neural networks. To perform text recognition on the localized regions, we propose a new multi-hypotheses method. Assuming different models of the text image, several segmentation hypotheses are produced. They are processed by an optical character recognition (OCR) system, and the result is selected from the generated strings according to a confidence value computed using language modeling and OCR statistics. Experiments show that this approach leads to much better results than the conventional method that tries to improve the individual segmentation algorithm. The whole system has been tested on several hours of videos and showed good performance when integrated in a sports video annotation system and a video indexing system within the framework of two European projects.

IDIAP2002

Methods for acute and long-term imaging of the ventral nerve cord in behaving adult Drosophila

Laura Joan Hermans

Understanding how neural circuits remodel and adapt to animal behavior is a central theme in the field of Neuroscience. One strategy to reach this goal is to repeatedly record the same animal's neural circuits and observe how they adapt when facing different challenges. Research has primarily focused on identifying neural circuits and connectivity maps within the brains of model organisms. Nevertheless, it has become evident that functional recordings are essential to have a complete understanding of how neural circuits generate behaviors. Numerous techniques have been introduced to record activity from brain circuits in model animals spanning from monkeys to insects. However, imaging the spinal cord in mammals or the ventral nerve cord in insects remains an open challenge due to the complexity of obtaining optical access without impacting the animal's behavior and damaging neural tissues. As a result, we have very limited information on how neural circuits in the spinal cord generate behaviors and how they adapt to different experiences such as injury or aging. Drosophila is a perfect model organism for carrying out this line of research thanks to its small size, genetically tractable nervous system, and behavioral diversity. The results are relevant to human physiology and behavior as many mechanisms have homologs in mammals. What is missing is a technological advancement that allows recording of behavior, structural changes in the circuits, and spatiotemporal dynamics of neural activity on the same animal.This thesis addresses this gap and introduces two novel methods for in vivo functional recordings of the fly's ventral nerve cord (VNC). The first one gains short-term access to the fly's VNC using a novel dissection preparation. The second one enables long-term functional recordings of the fly's VNC thanks to a toolkit combining four microengineered devices: a surgical arm, a V-shape implant, an optically transparent and numbered window and a remounting stage. The toolkit is suitable for both male and female flies and provides optical access to the VNC for at least one month. Additionally, the long-term toolkit does not impact flies' behaviors compared to intact flies and can be used to image both sparse and large populations of neural activity in flies' cervical connective and VNC. The toolkit was tested in two proof-of-concept (POC) applications. In the first POC, the toolkit monitored the degradation of sensory neurons in the VNC over weeks following limb amputation. In the second POC, the toolkit monitored global changes in neural dynamics following ingestion of a high-concentration caffeine solution. In conclusion, this thesis introduces new techniques for functional recordings of the fly's cervical connective and VNC over short and long time scales. They enable optical access to the fly's cervical connective that can be leveraged to study the content of information transmitted between the brain and the VNC while the fly is behaving. The two POCs demonstrate that the novel technology is ideally suited for long-term experiments and could be applied to a wide range of studies. For instance, the toolkit could be used to investigate how behaviors are encoded in the fly's VNC and how they adapt over time to experiences such as disease, aging, drug intake and learning.