Publications related to Analysis of MLP Based Hierarchical Phoneme Posterior Probability Estimator

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

Biologically plausible unsupervised learning in shallow and deep neural networks

Bernd Albert Illing

The way our brain learns to disentangle complex signals into unambiguous concepts is fascinating but remains largely unknown. There is evidence, however, that hierarchical neural representations play a key role in the cortex. This thesis investigates biolo ...

EPFL2021

Learning multivariate new physics

Raffaele Tito D'Agnolo, Andrea Wulzer

We discuss a method that employs a multilayer perceptron to detect deviations from a reference model in large multivariate datasets. Our data analysis strategy does not rely on any prior assumption on the nature of the deviation. It is designed to be sensi ...

SPRINGER2021

Mixed-Precision Deep Learning Based on Computational Memory

Irem Boybat Kara, Evangelos Eleftheriou, Abu Sebastian

Deep neural networks (DNNs) have revolutionized the field of artificial intelligence and have achieved unprecedented success in cognitive tasks such as image and speech recognition. Training of large DNNs, however, is computationally intensive and this has ...

FRONTIERS MEDIA SA2020

A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers

Philip Neil Garner

Neural Network (NN) classifiers can assign extreme probabilities to samples that have not appeared during training (out-of-distribution samples) resulting in erroneous and unreliable predictions. One of the causes for this unwanted behaviour lies in the us ...

IEEE2020

Disentangling feature and lazy training in deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Arthur Jacot

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

Biologically plausible deep learning – but how far can we go with shallow networks?

Wulfram Gerstner, Johanni Michael Brea, Bernd Albert Illing

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining s ...

2019

Leveraging Convolutional Pose Machines for Fast and Accurate Head Pose Estimation

Jean-Marc Odobez, Olivier Canévet

We propose a head pose estimation framework that leverages on a recent keypoint detection model. More specifically, we apply the convolutional pose machines (CPMs) to input images, extract different types of facial keypoint features capturing appearance in ...

IEEE2018

Analysis of MLP Based Hierarchical Phoneme Posterior Probability Estimator

Graph Chatbot

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

An exact mapping from ReLU networks to spiking neural networks

Benign Overfitting in Deep Neural Networks under Lazy Training

Biologically plausible unsupervised learning in shallow and deep neural networks

Learning multivariate new physics

Mixed-Precision Deep Learning Based on Computational Memory

A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers

Disentangling feature and lazy training in deep neural networks

Biologically plausible deep learning – but how far can we go with shallow networks?

Leveraging Convolutional Pose Machines for Fast and Accurate Head Pose Estimation

A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers

Deep Learning Theory Through the Lens of Diagonal Linear Networks

An exact mapping from ReLU networks to spiking neural networks

Learning multivariate new physics

Biologically plausible unsupervised learning in shallow and deep neural networks

Benign Overfitting in Deep Neural Networks under Lazy Training

Mixed-Precision Deep Learning Based on Computational Memory

Disentangling feature and lazy training in deep neural networks

Biologically plausible deep learning – but how far can we go with shallow networks?

Leveraging Convolutional Pose Machines for Fast and Accurate Head Pose Estimation