Publications related to Highway network

Deep Learning Theory Through the Lens of Diagonal Linear Networks

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Efficient and Accurate Peer-to-Peer Training of Machine Learning Based Home Thermal Models

Rachid Guerraoui, Alexandre David Olivier Maurer, Omid Ardakanian

The integration of smart thermostats in home automation systems has created an opportunity to optimize space heating and cooling through the use of machine learning, for example for thermal model identification. Nonetheless, its full potential remains unta ...

New York2023

Universal Approximation Property of Hamiltonian Deep Neural Networks

Giancarlo Ferrari Trecate, Muhammad Zakwan

This letter investigates the universal approximation capabilities of Hamiltonian Deep Neural Networks (HDNNs) that arise from the discretization of Hamiltonian Neural Ordinary Differential Equations. Recently, it has been shown that HDNNs enjoy, by design, ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2023

Bayesian reconstruction of memories stored in neural networks from their connectivity

Florent Gérard Krzakala, Lenka Zdeborová

The advent of comprehensive synaptic wiring diagrams of large neural circuits has created the field of connectomics and given rise to a number of open research questions. One such question is whether it is possible to reconstruct the information stored in ...

2023

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Michaël Unser

The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L\documentclass[12pt]{ ...

New York2023

The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

Indaco Biazzo

Autoregressive Neural Networks (ARNNs) have shown exceptional results in generation tasks across image, language, and scientific domains. Despite their success, ARNN architectures often operate as black boxes without a clear connection to underlying physic ...

Berlin2023

Generalization Properties of NAS under Activation and Skip Connection Search

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

Neural Architecture Search (NAS) has fostered the automatic discovery of stateof- the-art neural architectures. Despite the progress achieved with NAS, so far there is little attention to theoretical guarantees on NAS. In this work, we study the generaliza ...

2022

Global information processing in feedforward deep networks

Michael Herzog, Ben Henrik Lönnqvist, Adrien Christophe Doerig, Alban Bornet

While deep neural networks are state-of-the-art models of many parts of the human visual system, here we show that they fail to process global information in a humanlike manner. First, using visual crowding as a probe into global visual information process ...

2022

Distributed neural network control with dependability guarantees: a compositional port-Hamiltonian approach

Giancarlo Ferrari Trecate, Luca Furieri, Muhammad Zakwan, Clara Lucía Galimberti

Large-scale cyber-physical systems require that control policies are distributed, that is, that they only rely on local real-time measurements and communication with neighboring agents. Optimal Distributed Control (ODC) problems are, however, highly intrac ...

2022

Population receptive fields isolate or combine target and flankers in (un)crowding

Michael Herzog, Bogdan Draganski, Ayberk Ozkirli, Maya Anna Jastrzebowska

Crowding by neighboring elements leads to the deterioration of target discrimination. This phenomenon is traditionally explained with feedforward, hierarchical models. In these models, a “bottleneck” at the earliest stages of visual processing causes an ir ...

2022

Highway network

Graph Chatbot

Chat with Graph Search

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Efficient and Accurate Peer-to-Peer Training of Machine Learning Based Home Thermal Models

Universal Approximation Property of Hamiltonian Deep Neural Networks

Bayesian reconstruction of memories stored in neural networks from their connectivity

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

Generalization Properties of NAS under Activation and Skip Connection Search

Global information processing in feedforward deep networks

Distributed neural network control with dependability guarantees: a compositional port-Hamiltonian approach

Population receptive fields isolate or combine target and flankers in (un)crowding

From Kernel Methods to Neural Networks: A Unifying Variational Formulation

Universal Approximation Property of Hamiltonian Deep Neural Networks

Generalization Properties of NAS under Activation and Skip Connection Search

Global information processing in feedforward deep networks

The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

Population receptive fields isolate or combine target and flankers in (un)crowding

Efficient and Accurate Peer-to-Peer Training of Machine Learning Based Home Thermal Models

Distributed neural network control with dependability guarantees: a compositional port-Hamiltonian approach

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Bayesian reconstruction of memories stored in neural networks from their connectivity