Publications related to The committee machine: computational to statistical gaps in learning a two-layers neural network

Random matrix methods for high-dimensional machine learning models

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Elisabetta Cornacchia

This thesis focuses on two selected learning problems: 1) statistical inference on graphs models, and, 2) gradient descent on neural networks, with the common objective of defining and analysing the measures that characterize the fundamental limits.In the ...

EPFL2023

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Michaël Unser

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

New York2023

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Leonardo Petrini

Artificial intelligence, particularly the subfield of machine learning, has seen a paradigm shift towards data-driven models that learn from and adapt to data. This has resulted in unprecedented advancements in various domains such as natural language proc ...

EPFL2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins.All these successful methods depend on a gradient-based learning algorithm to train a model on massive a ...

EPFL2023

Time Series Analysis of Urban Liveability

Devis Tuia, Diego Marcos Gonzalez, Alex Hubertus Levering

In this paper we explore deep learning models to monitor longitudinal liveability changes in Dutch cities at the neighbourhood level. Our liveability reference data is defined by a country-wise yearly survey based on a set of indicators combined into a liv ...

IEEE2023

Penalising the biases in norm regularisation enforces sparsity

Nicolas Henri Bernard Flammarion, Etienne Patrice Boursier

Controlling the parameters' norm often yields good generalisation when training neural networks. Beyond simple intuitions, the relation between parameters' norm and obtained estimators theoretically remains misunderstood. For one hidden ReLU layer networks ...

2023

Spatially adaptive machine learning models for predicting water quality in Hong Kong

Rongrong Li, Qiaoli Wang, Yu Xu

Water quality prediction in the spatially heterogeneous environment is challenging as the importance of water quality parameters (WQPs) and the performance of prediction models may vary across space. Thus, this study proposed spatially adaptive machine lea ...

ELSEVIER2023

The committee machine: computational to statistical gaps in learning a two-layers neural network

Graph Chatbot

Chat with Graph Search

Random matrix methods for high-dimensional machine learning models

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Deep Learning Generalization with Limited and Noisy Labels

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Transformer Models for Vision

Time Series Analysis of Urban Liveability

Penalising the biases in norm regularisation enforces sparsity

Spatially adaptive machine learning models for predicting water quality in Hong Kong

Random matrix methods for high-dimensional machine learning models

Deep Learning Generalization with Limited and Noisy Labels

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Transformer Models for Vision

Time Series Analysis of Urban Liveability

Penalising the biases in norm regularisation enforces sparsity

Spatially adaptive machine learning models for predicting water quality in Hong Kong