Publications related to Constructive Training Methods for Feedforward Neural Networks with Binary Weights

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

Mattia Atzeni

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

EPFL2024

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

Expectation consistency for calibration of neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Lucas Andry Clarte, Bruno Loureiro

Despite their incredible performance, it is well reported that deep neural networks tend to be overoptimistic about their prediction confidence. Finding effective and efficient calibration methods for neural networks is therefore an important endeavour tow ...

2023

Privacy-preserving federated neural network training and inference

Sinem Sav

Training accurate and robust machine learning models requires a large amount of data that is usually scattered across data silos. Sharing, transferring, and centralizing the data from silos, however, is difficult due to current privacy regulations (e.g., H ...

EPFL2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

A new variable shape parameter strategy for RBF approximation using neural networks

Jan Sickmann Hesthaven

The choice of the shape parameter highly effects the behaviour of radial basis function (RBF) approximations, as it needs to be selected to balance between the ill-conditioning of the interpolation matrix and high accuracy. In this paper, we demonstrate ho ...

PERGAMON-ELSEVIER SCIENCE LTD2023

The influence of splitter blades and meridional profiles on the performance of small-scale turbopumps for ORC applications; analysis, neural network modeling and optimization

Jürg Alexander Schiffmann, Sajjad Zakeralhoseini

This work presents the influence of splitters on the performance of small-scale turbopumps for organic Rankine cycle applications numerically. Different design parameters, such as impeller diameter at the outlet, number of blades, blade angle at the traili ...

2023

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu, Yongtao Wu

Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds. The study on NTK has been devoted to typical neural network architectures, but it is incomplete for neural networks with Hadamar ...

2022

Constructive Training Methods for Feedforward Neural Networks with Binary Weights

Graph Chatbot

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

An exact mapping from ReLU networks to spiking neural networks

Expectation consistency for calibration of neural networks

Privacy-preserving federated neural network training and inference

Deep Learning Generalization with Limited and Noisy Labels

Benign Overfitting in Deep Neural Networks under Lazy Training

A new variable shape parameter strategy for RBF approximation using neural networks

The influence of splitter blades and meridional profiles on the performance of small-scale turbopumps for ORC applications; analysis, neural network modeling and optimization

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

An exact mapping from ReLU networks to spiking neural networks

Expectation consistency for calibration of neural networks

Privacy-preserving federated neural network training and inference

Deep Learning Generalization with Limited and Noisy Labels

Benign Overfitting in Deep Neural Networks under Lazy Training

The influence of splitter blades and meridional profiles on the performance of small-scale turbopumps for ORC applications; analysis, neural network modeling and optimization

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Deep Learning Theory Through the Lens of Diagonal Linear Networks

A new variable shape parameter strategy for RBF approximation using neural networks