Publications related to Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Machine Learning for Modeling Stock Returns

Teng Andrea Xu

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

EPFL2024

M2SKD: Multi-to-Single Knowledge Distillation of Real-Time Epileptic Seizure Detection for Low-Power Wearable Systems

David Atienza Alonso, Amir Aminifar, Tomas Teijeiro Campo, Alireza Amirshahi, Farnaz Forooghifar, Saleh Baghersalimi

Integrating low-power wearable systems into routine health monitoring is an ongoing challenge. Recent advances in the computation capabilities of wearables make it possible to target complex scenarios by exploiting multiple biosignals and using high-perfor ...

2024

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti, Liang Xu

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Elisabetta Cornacchia

This thesis focuses on two selected learning problems: 1) statistical inference on graphs models, and, 2) gradient descent on neural networks, with the common objective of defining and analysing the measures that characterize the fundamental limits.In the ...

EPFL2023

Privacy-preserving federated neural network training and inference

Sinem Sav

Training accurate and robust machine learning models requires a large amount of data that is usually scattered across data silos. Sharing, transferring, and centralizing the data from silos, however, is difficult due to current privacy regulations (e.g., H ...

EPFL2023

Predicting nonlinear optical scattering with physics-driven neural networks

Demetri Psaltis, Carlo Gigli, Ahmed Ayoub

Deep neural networks trained on physical losses are emerging as promising surrogates for nonlinear numerical solvers. These tools can predict solutions to Maxwell's equations and compute gradients of output fields with respect to the material and geometric ...

AIP Publishing2023

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Graph Chatbot

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Machine Learning for Modeling Stock Returns

M2SKD: Multi-to-Single Knowledge Distillation of Real-Time Epileptic Seizure Detection for Low-Power Wearable Systems

Deep Learning Generalization with Limited and Noisy Labels

Benign Overfitting in Deep Neural Networks under Lazy Training

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Privacy-preserving federated neural network training and inference

Predicting nonlinear optical scattering with physics-driven neural networks

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Machine Learning for Modeling Stock Returns

Deep Learning Generalization with Limited and Noisy Labels

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Privacy-preserving federated neural network training and inference

M2SKD: Multi-to-Single Knowledge Distillation of Real-Time Epileptic Seizure Detection for Low-Power Wearable Systems

Benign Overfitting in Deep Neural Networks under Lazy Training

Predicting nonlinear optical scattering with physics-driven neural networks