Publications related to Reservoir optimization in recurrent neural networks using properties of Kronecker product

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti, Liang Xu

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins.All these successful methods depend on a gradient-based learning algorithm to train a model on massive a ...

EPFL2023

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Michaël Unser

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

New York2023

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

Probabilistic methods for neural combinatorial optimization

Nikolaos Karalias

The monumental progress in the development of machine learning models has led to a plethora of applications with transformative effects in engineering and science. This has also turned the attention of the research community towards the pursuit of construc ...

EPFL2023

Time Series Analysis of Urban Liveability

Devis Tuia, Diego Marcos Gonzalez, Alex Hubertus Levering

In this paper we explore deep learning models to monitor longitudinal liveability changes in Dutch cities at the neighbourhood level. Our liveability reference data is defined by a country-wise yearly survey based on a set of indicators combined into a liv ...

IEEE2023

Learned Compressive Representations for Single-Photon 3D Imaging

Edoardo Charbon, Claudio Bruschini, Andrei Ardelean, Mohit Gupta

Single-photon 3D cameras can record the time-of-arrival of billions of photons per second with picosecond accuracy. One common approach to summarize the photon data stream is to build a per-pixel timestamp histogram, resulting in a 3D histogram tensor that ...

Ieee Computer Soc2023

Reservoir optimization in recurrent neural networks using properties of Kronecker product

Graph Chatbot

Chat with Graph Search

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Deep Learning Generalization with Limited and Noisy Labels

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Transformer Models for Vision

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

An exact mapping from ReLU networks to spiking neural networks

Probabilistic methods for neural combinatorial optimization

Time Series Analysis of Urban Liveability

Learned Compressive Representations for Single-Photon 3D Imaging

Learned Compressive Representations for Single-Photon 3D Imaging

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Probabilistic methods for neural combinatorial optimization

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Deep Learning Generalization with Limited and Noisy Labels

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Transformer Models for Vision

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

An exact mapping from ReLU networks to spiking neural networks

Time Series Analysis of Urban Liveability