Neural Networks: Multilayer Perceptrons

This lecture introduces Multilayer Perceptrons (MLP) as a flexible function family that can solve complex problems by finding smart features and regression coefficients simultaneously. It covers the concept of artificial neurons, popular activation functions like relu and sigmoid, matrix notation for MLP, and the importance of flexibility and regularization in neural networks. The lecture also discusses regression and classification tasks with MLPs, showcasing the XOR problem and the parametrization of conditional densities. Practical examples include regression with MLPs and classification with MLPs for tasks like fitting weather data and predicting analyte retention time in liquid chromatography.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (1,000)

Related concepts (220)

BIO-322: Introduction to machine learning for bioengineers

Students understand basic concepts and methods of machine learning. They can describe them in mathematical terms and can apply them to data using a high-level programming language (julia/python/R).

Logistic regression

In statistics, the logistic model (or logit model) is a statistical model that models the probability of an event taking place by having the log-odds for the event be a linear combination of one or more independent variables. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (the coefficients in the linear combination).

Artificial neural network

Artificial neural networks (ANNs, also shortened to neural networks (NNs) or neural nets) are a branch of machine learning models that are built using principles of neuronal organization discovered by connectionism in the biological neural networks constituting animal brains. An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit a signal to other neurons.

Deep learning

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.

Residual neural network

A Residual Neural Network (a.k.a. Residual Network, ResNet) is a deep learning model in which the weight layers learn residual functions with reference to the layer inputs. A Residual Network is a network with skip connections that perform identity mappings, merged with the layer outputs by addition. It behaves like a Highway Network whose gates are opened through strongly positive bias weights. This enables deep learning models with tens or hundreds of layers to train easily and approach better accuracy when going deeper.

Kernel Methods: Neural Networks PHYS-467: Machine learning for physicists

Covers the fundamentals of neural networks, focusing on RBF kernels and SVM.

Neural Networks: Basics and Applications

Explores neural networks basics, XOR problem, classification, and practical applications like weather data prediction.

Feature Engineering: Missing Data and Standardization

Covers techniques for handling missing data and standardizing features, as well as transforming input and output data.

Neural Networks: Regression and Classification

Explores neural networks for regression and classification tasks, covering training, regularization, and practical examples.

Neural Networks: Training and Activation CIVIL-226: Introduction to machine learning for engineers

Explores neural networks, activation functions, backpropagation, and PyTorch implementation.