Relative Stability Towards Diffeomorphisms in Deep Nets

This lecture explores the concept of relative stability towards diffeomorphisms in deep neural networks, proposing that image classification is achievable due to classes being invariant to smooth deformations. By studying typical diffeomorphisms of controlled norm, the lecture delves into how stability to smooth input transformations relative to generic transformations impacts performance. Various architectures, from simple CNNs to state-of-the-art methods, are analyzed, showing that low relative sensitivity to diffeomorphisms is crucial for good performance. The correlation between relative stability and performance is highlighted, indicating that parameters like depth or skip connections are less significant. The findings suggest that relative stability to diffeomorphisms is a better predictor of performance than stability to noise alone.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (34)

Related concepts (34)

PHYS-754: Lecture series on scientific machine learning

This lecture presents ongoing work on how scientific questions can be tackled using machine learning. Machine learning enables extracting knowledge from data computationally and in an automatized way.

Feedforward neural network

A feedforward neural network (FNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. Its flow is uni-directional, meaning that the information in the model flows in only one direction—forward—from the input nodes, through the hidden nodes (if any) and to the output nodes, without any cycles or loops, in contrast to recurrent neural networks, which have a bi-directional flow.

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Types of artificial neural networks

There are many types of artificial neural networks (ANN). Artificial neural networks are computational models inspired by biological neural networks, and are used to approximate functions that are generally unknown. Particularly, they are inspired by the behaviour of neurons and the electrical signals they convey between input (such as from the eyes or nerve endings in the hand), processing, and output from the brain (such as reacting to light, touch, or heat). The way neurons semantically communicate is an area of ongoing research.

Artificial neural network

Artificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.

Computer vision

Computer vision tasks include methods for , , and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images (the input to the retina in the human analog) into descriptions of the world that make sense to thought processes and can elicit appropriate action.

Neural Networks: Two Layers Neural Network MATH-212: Analyse numérique et optimisation

Covers the basics of neural networks, focusing on the development from two layers neural networks to deep neural networks.

Numerical analysis MATH-351: Advanced numerical analysis

Covers advanced numerical analysis topics including deep neural networks and optimization methods.

Convolutional Networks: Motivation & Ideas PHYS-467: Machine learning for physicists

Explores the motivation and ideas behind convolutional networks, emphasizing weight sharing and pooling layers.

Neural Networks: Multilayer Learning PHYS-467: Machine learning for physicists

Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.

Deep Learning for Autonomous Vehicles: Learning CIVIL-459: Deep learning for autonomous vehicles

Explores learning in deep learning for autonomous vehicles, covering predictive models, RNN, ImageNet, and transfer learning.