Network architecture | EPFL Graph Search

Network architecture is the design of a computer network. It is a framework for the specification of a network's physical components and their functional organization and configuration, its operational principles and procedures, as well as communication protocols used. In telecommunication, the specification of a network architecture may also include a detailed description of products and services delivered via a communications network, as well as detailed rate and billing structures under which services are compensated. The network architecture of the Internet is predominantly expressed by its use of the Internet protocol suite, rather than a specific model for interconnecting networks or nodes in the network, or the usage of specific types of hardware links. OSI model The Open Systems Interconnection model (OSI model) defines and codifies the concept of layered network architecture. Abstraction layers are used to subdivide a communications system further into smaller

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing games and many more. In spite of these major achievements, our understanding of these models, in particular of what happens during their training, remains very limited. This PhD started with the introduction of the Neural Tangent Kernel (NTK) to describe the evolution of the function represented by the network during training. In the infinite-width limit, i.e. when the number of neurons in the layers of the network grows to infinity, the NTK converges to a deterministic and time-independent limit, leading to a simple yet complete description of the dynamics of infinitely-wide DNNs. This allowed one to give the first general proof of convergence of DNNs to a global minimum, and yielded the first description of the limiting spectrum of the Hessian of the loss surface of DNNs throughout training.More importantly, the NTK plays a crucial role in describing the generalization abilities of DNNs, i.e. the performance of the trained network on unseen data. The NTK analysis uncovered a direct link between the function learned by infinitely wide DNNs and Kernel Ridge Regression predictors, whose generalization properties are studied in this thesis using tools of random matrix theory. Our analysis of KRR reveals the importance of the eigendecomposition of the NTK, which is affected by a number of architectural choices. In very deep networks, an ordered regime and a chaotic regime appear, determined by the choice of non-linearity and the balance between the weights and bias parameters; these two phases are characterized by different speeds of decay of the eigenvalues of the NTK, leading to a tradeoff between convergence speed and generalization. In practical contexts such as Generative Adversarial Networks or Topology Optimization, the network architecture can be chosen to guarantee certain properties of the NTK and its spectrum.These results give an almost complete description DNNs in this infinite-width limit. It is then natural to wonder how it extends to finite-width networks used in practice. In the so-called NTK regime, the discrepancy between finite- and infinite-widths DNNs is mainly a result of the variance w.r.t. to the sampling of the parameters, as shown empirically and mathematically relying on the similarity between DNNs and random feature models.In contrast to the NTK regime, where the NTK remains constant during training, there exist so-called active regimes, where the evolution of the NTK is significant, which appear in a number of settings. One such regime appears in Deep Linear Networks with a very small initialization, where the training dynamics approach a sequence of saddle-points, representing linear maps of increasing rank, leading to a low-rank bias which is absent in the NTK regime.

EPFL2022

PDE-Aware Deep Learning for Inverse Problems in Cardiac Electrophysiology

Simone Deparis, Stefano Pagani, Alfio Quarteroni, Riccardo Tenderini

In this work, we present a PDE-aware deep learning model for the numerical solution to the inverse problem of electrocardiography. The model both leverages data availability and exploits the knowledge of a physically based mathematical model, expressed by means of partial differential equations (PDEs), to carry out the task at hand. The goal is to estimate the epicardial potential field from measurements of the electric potential at a discrete set of points on the body surface. The employment of deep learning techniques in this context is made difficult by the low amount of clinical data at disposal, as measuring cardiac potentials requires invasive procedures. Suitably exploiting the underlying physically based mathematical model allowed circumventing the data availability issue and led to the development of fast-training and low-complexity models. Physical awareness has been pursued by means of two elements: the projection of the epicardial potential onto a space-time reduced subspace, spanned by the numerical solutions of the governing PDEs, and the inclusion of a tensorial reduced basis solver of the forward problem in the network architecture. Numerical tests have been conducted only on synthetic data, obtained via a full order model approximation of the problem at hand, and two variants of the model have been addressed. Both proved to be accurate, up to an average

\ell^1

-norm relative error on epicardial activation maps of 3.5%, and both could be trained in

\approx$$15

min. Nevertheless, some improvements, mostly concerning data generation, are necessary in order to bridge the gap with clinical applications.

2022

SensorScope: Out-of-the-Box Environmental Monitoring

Guillermo Barrenetxea Kobas, Olivier Couach, François Ingelrest, Marc Parlange, Martin Vetterli

An important field of application for wireless sensor networks is environmental monitoring. Given the importance of potential climate changes, environmental impact on cities, and pollution, it is an application domain where sensor networks can have great impact and as such is getting more and more attention. Current data collection techniques are indeed rather limited and make use of very expensive sensing stations, leading to a lack of appropriate observations. In this paper, we present SensorScope, a collaborative project between environmental and network researchers, that aims at providing an efficient and cheap out-of-the-box environmental monitoring system based on a wireless sensor network. We especially focus on data gathering and present the hardware and network architecture of SensorScope. We also describe a real-world deployment, which took place on a rock glacier in Switzerland, as well as the results we obtained.

2008