Élagage synaptique | EPFL Graph Search

vignette|droite|upright=1.2|alt=Schéma représentant une modélisation des différents types d'expression neuro-synaptique.|Les différents types de connexions où le processus d'élagage synaptique peut intervenir. L'élagage synaptique est le processus d'élimination des synapses qui se produit entre la petite enfance et le début de la puberté chez les humains. Le même phénomène se produit chez de nombreux mammifères. Ce mécanisme comprend d'une part l'élagage des axones et d'autre part celui des dendrites. L'élagage des connexions synaptiques, étroitement associé à la phase d'apprentissage et au développement neurocognitif de l'enfant, permet d'affiner les faisceaux neuronaux, mais également d'éliminer les structures neuronales devenues obsolètes ou déficientes. À cet effet, ce processus présente un caractère sélectif et joue un rôle de régulateur synaptique. Contexte neurocognitif et développemental vignette|droite|upright=0.9|Modélisation d'une liaison synaptique. L'élagage s

Genetic dissection of the role of BMP signaling for the development of the auditory brainstem nuclei and the large calyx of Held synapse

Elin Kristina Kronander

Synapses are key elements for communication within the complex network of neurons that make up our brain. Interestingly, synaptic connections can differ greatly in form and function, depending on the specific circuit into which they are embedded. In this PhD thesis, I use a large excitatory synaptic connection in the auditory brainstem, the calyx of Held, as a model to study the molecular mechanisms of synapse development. We are especially interested in the role of bone morphogenetic protein (BMP) signaling in this process. BMPs are extracellular morphogens with signaling roles early in the development of multicellular animals, and in brain development. In the first part of this PhD thesis, I have established an organotypic culture model of the auditory brainstem, in which large calyx-like synapses form in vitro. I could demonstrate the usefulness of this culture system for studying molecular mechanisms of nerve terminal growth, by showing that the BMP-receptor inhibitor, LDN-193189, leads to a reduction in the size of nerve terminals forming in vitro. In the second part, I have used mouse genetics, especially floxed alleles of BMP-receptors and the SMAD4 protein, coupled with morphological analysis and electrophysiological measurements, to investigate the role of BMP signaling in the development of large calyx of Held nerve terminals. Genetic inactivation of BMP type I receptors early-on (~ embryonic day 9 (E9) led to a significantly smaller ventral cochlear nucleus (VCN) and medial nucleus of the trapezoid body (MNTB), indicating that BMP signaling is involved in the histogenesis of these auditory brainstem nuclei. These deficits were in part caused by the intracellular SMAD - signaling pathway, as revealed by conditional inactivation of the SMAD4 gene. To circumvent morphological changes observed upon the early inactivation of BMP receptors, I investigated the onset of Cre expression in a Parvalbumin (PV) Cre mouse line where recombination happens in presynaptic VCN neurons at ~ E15, and in postsynaptic MNTB neurons at postnatal day (P) 6. Conditional inactivation of BMP-receptors using the PVCre mouse line did not lead to morphological alterations. However, in these conditional KO mice, the functional development of the calyx of Held synapse was not significantly altered when measured at P6 (only the presynaptic receptors should be removed), nor at P14 or at P21, at which both pre- and postsynaptic receptors should be removed. Thus, presynaptic BMP-receptor activation alone seems unlikely to be necessary for calyx of Held growth. Also, ongoing BMP-signaling in both pre- and postsynaptic compartments seems dispensable for later maturation processes and the maintenance of the calyx of Held synapse. Finally, I removed BMP receptors specifically in the presynaptic neuron pool via virus-mediated expression of Cre-recombinase in newborn mice. I then observed subtle changes in the branching pattern of presynaptic axons and a slight increase in double-innervations. Taken together, this study shows that BMP signaling plays a role in the histogenesis of auditory brainstem structures and partially in the growth of calyx of Held synapses and elimination of competing synaptic connections. This work also offers a deeper understanding into the use of mouse genetic tools for studying the development of specific synaptic connections and establishes a novel culture model for studying such connections in vitro.

EPFL2016

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

Our brain continuously self-organizes to construct and maintain an internal representation of the world based on the information arriving through sensory stimuli. Remarkably, cortical areas related to different sensory modalities appear to share the same functional unit, the neuron, and develop through the same learning mechanism, synaptic plasticity. It motivates the conjecture of a unifying theory to explain cortical representational learning across sensory modalities. In this thesis we present theories and computational models of learning and optimization in neural networks, postulating functional properties of synaptic plasticity that support the apparent universal learning capacity of cortical networks. In the past decades, a variety of theories and models have been proposed to describe receptive field formation in sensory areas. They include normative models such as sparse coding, and bottom-up models such as spike-timing dependent plasticity. We bring together candidate explanations by demonstrating that in fact a single principle is sufficient to explain receptive field development. First, we show that many representative models of sensory development are in fact implementing variations of a common principle: nonlinear Hebbian learning. Second, we reveal that nonlinear Hebbian learning is sufficient for receptive field formation through sensory inputs. A surprising result is that our findings are independent of specific details, and allow for robust predictions of the learned receptive fields. Thus nonlinear Hebbian learning and natural statistics can account for many aspects of receptive field formation across models and sensory modalities. The Hebbian learning theory substantiates that synaptic plasticity can be interpreted as an optimization procedure, implementing stochastic gradient descent. In stochastic gradient descent inputs arrive sequentially, as in sensory streams. However, individual data samples have very little information about the correct learning signal, and it becomes a fundamental problem to know how many samples are required for reliable synaptic changes. Through estimation theory, we develop a novel adaptive learning rate model, that adapts the magnitude of synaptic changes based on the statistics of the learning signal, enabling an optimal use of data samples. Our model has a simple implementation and demonstrates improved learning speed, making this a promising candidate for large artificial neural network applications. The model also makes predictions on how cortical plasticity may modulate synaptic plasticity for optimal learning. The optimal sampling size for reliable learning allows us to estimate optimal learning times for a given model. We apply this theory to derive analytical bounds on times for the optimization of synaptic connections. First, we show this optimization problem to have exponentially many saddle-nodes, which lead to small gradients and slow learning. Second, we show that the number of input synapses to a neuron modulates the magnitude of the initial gradient, determining the duration of learning. Our final result reveals that the learning duration increases supra-linearly with the number of synapses, suggesting an effective limit on synaptic connections and receptive field sizes in developing neural networks.

EPFL2016

Dynamic Probabilistic Pruning: A General Framework for Hardware-Constrained Pruning at Different Granularities

Lizeth Gonzalez Carabarin, Alexandre Schmid

Unstructured neural network pruning algorithms have achieved impressive compression ratios. However, the resulting-typically irregular-sparse matrices hamper efficient hardware implementations, leading to additional memory usage and complex control logic that diminishes the benefits of unstructured pruning. This has spurred structured coarse-grained pruning solutions that prune entire feature maps or even layers, enabling efficient implementation at the expense of reduced flexibility. Here, we propose a flexible new pruning mechanism that facilitates pruning at different granularities (weights, kernels, and feature maps) while retaining efficient memory organization (e.g., pruning exactly k-out-of- n weights for every output neuron or pruning exactly k-out-of-n kernels for every feature map). We refer to this algorithm as dynamic probabilistic pruning (DPP). DPP leverages the Gumbel-softmax relaxation for differentiable k-out-of-n sampling, facilitating end-to-end optimization. We show that DPP achieves competitive compression ratios and classification accuracy when pruning common deep learning models trained on different benchmark datasets for image classification. Relevantly, the dynamic masking of DPP facilitates for joint optimization of pruning and weight quantization in order to even further compress the network, which we show as well. Finally, we propose novel information-theoretic metrics that show the confidence and pruning diversity of pruning masks within a layer.

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

EPFL2016

Genetic dissection of the role of BMP signaling for the development of the auditory brainstem nuclei and the large calyx of Held synapse

Elin Kristina Kronander

EPFL2016