Publications related to Weight Initialization for High Order and Multilayer Perceptrons

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

Privacy-preserving federated neural network training and inference

Sinem Sav

Training accurate and robust machine learning models requires a large amount of data that is usually scattered across data silos. Sharing, transferring, and centralizing the data from silos, however, is difficult due to current privacy regulations (e.g., H ...

EPFL2023

Sharp asymptotics on the compression of two-layer neural networks

Marco Mondelli

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M < N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimiz ...

IEEE2022

Deep Learning Detection of GPS Spoofing

Mirjana Stojilovic, Olivia Jullian Parra

Unmanned aerial vehicles (UAVs) are widely deployed in air navigation, where numerous applications use them for safety-of-life and positioning, navigation, and timing tasks. Consequently, GPS spoofing attacks are more and more frequent. The aim of this wor ...

Springer, Cham2022

Exploring quantum perceptron and quantum neural network structures with a teacher-student scheme

Near-term quantum devices can be used to build quantum machine learning models, such as quantum kernel methods and quantum neural networks (QNN), to perform classification tasks. There have been many proposals on how to use variational quantum circuits as ...

2022

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Nikolaos Dimitriadis

In this paper, we study an emerging class of neural networks, the Morphological Neural networks, from some modern perspectives. Our approach utilizes ideas from tropical geometry and mathematical morphology. First, we state the training of a binary morphol ...

IEEE2021

Crowding and the importance of grouping and segmentation processes in human vision

Alban Bornet

Human vision has evolved to make sense of a world in which elements almost never appear in isolation. Surprisingly, the recognition of an element in a visual scene is strongly limited by the presence of other nearby elements, a phenomenon known as visual c ...

EPFL2021

Disentangling feature and lazy training in deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Arthur Jacot

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

Mixed-Precision Deep Learning Based on Computational Memory

Irem Boybat Kara, Evangelos Eleftheriou, Abu Sebastian

Deep neural networks (DNNs) have revolutionized the field of artificial intelligence and have achieved unprecedented success in cognitive tasks such as image and speech recognition. Training of large DNNs, however, is computationally intensive and this has ...

FRONTIERS MEDIA SA2020

Weight Initialization for High Order and Multilayer Perceptrons

Graph Chatbot

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Benign Overfitting in Deep Neural Networks under Lazy Training

Privacy-preserving federated neural network training and inference

Sharp asymptotics on the compression of two-layer neural networks

Deep Learning Detection of GPS Spoofing

Exploring quantum perceptron and quantum neural network structures with a teacher-student scheme

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Crowding and the importance of grouping and segmentation processes in human vision

Disentangling feature and lazy training in deep neural networks

Mixed-Precision Deep Learning Based on Computational Memory

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Benign Overfitting in Deep Neural Networks under Lazy Training

Sharp asymptotics on the compression of two-layer neural networks

Crowding and the importance of grouping and segmentation processes in human vision

Mixed-Precision Deep Learning Based on Computational Memory

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Deep Learning Detection of GPS Spoofing

Privacy-preserving federated neural network training and inference

Disentangling feature and lazy training in deep neural networks

Exploring quantum perceptron and quantum neural network structures with a teacher-student scheme