Multi-layered Perceptron: History and Training Algorithm

This lecture covers the historical development of artificial neural networks, starting with the threshold logic unit and the perceptron. The instructor explains the training algorithm for the perceptron, focusing on the gradient descent method. The lecture then introduces the multi-layered perceptron, discussing its architecture, activation functions, and the backpropagation algorithm. The importance of feature design and the limitations of linear models are also addressed. The instructor demonstrates how a multi-layered perceptron can approximate any continuous function and the challenges in interpreting its operations.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (396)

Related concepts (66)

EE-311: Fundamentals of machine learning

Ce cours présente une vue générale des techniques d'apprentissage automatique, passant en revue les algorithmes, le formalisme théorique et les protocoles expérimentaux.

Deep learning

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Feedforward neural network

A feedforward neural network (FNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. Its flow is uni-directional, meaning that the information in the model flows in only one direction—forward—from the input nodes, through the hidden nodes (if any) and to the output nodes, without any cycles or loops, in contrast to recurrent neural networks, which have a bi-directional flow.

Machine learning

Machine learning (ML) is an umbrella term for solving problems for which development of algorithms by human programmers would be cost-prohibitive, and instead the problems are solved by helping machines 'discover' their 'own' algorithms, without needing to be explicitly told what to do by any human-developed algorithms. Recently, generative artificial neural networks have been able to surpass results of many previous approaches.

Extreme learning machine

Extreme learning machines are feedforward neural networks for classification, regression, clustering, sparse approximation, compression and feature learning with a single layer or multiple layers of hidden nodes, where the parameters of hidden nodes (not just the weights connecting inputs to hidden nodes) need to be tuned. These hidden nodes can be randomly assigned and never updated (i.e. they are random projection but with nonlinear transforms), or can be inherited from their ancestors without being changed.

Active learning (machine learning)

Active learning is a special case of machine learning in which a learning algorithm can interactively query a user (or some other information source) to label new data points with the desired outputs. In statistics literature, it is sometimes also called optimal experimental design. The information source is also called teacher or oracle. There are situations in which unlabeled data is abundant but manual labeling is expensive. In such a scenario, learning algorithms can actively query the user/teacher for labels.

Kernel Methods: Neural Networks PHYS-467: Machine learning for physicists

Covers the fundamentals of neural networks, focusing on RBF kernels and SVM.

Neural Networks: Multilayer Perceptron EE-311: Fundamentals of machine learning

Explores the history, models, training, convergence, and limitations of neural networks, including the backpropagation algorithm and universal approximation.

Neural Networks: Training and Activation CIVIL-226: Introduction to machine learning for engineers

Explores neural networks, activation functions, backpropagation, and PyTorch implementation.

Deep Learning Fundamentals ME-390: Foundations of artificial intelligence

Introduces deep learning, from logistic regression to neural networks, emphasizing the need for handling non-linearly separable data.

Neural Networks: Multilayer Learning PHYS-467: Machine learning for physicists

Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.