Stochastic gradient descent

Applied sciences
Information engineering
Machine learning
Topics in machine learning

Related lectures (28)

About
Privacy
Disclaimer

Graph Chatbot

Page 2 of 3

Gradient Descent: Optimization Techniques

Explores gradient descent, loss functions, and optimization techniques in neural network training.

Multilayer Perceptron: Training and Optimization

Explores the multilayer perceptron model, training, optimization, data preprocessing, activation functions, backpropagation, and regularization.

Gradient Descent: Linear Regression

Covers the concept of gradient descent for linear regression, explaining the iterative process of updating parameters.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Structures in Non-Convex Optimization

Explores non-convex optimization in deep learning, covering critical points, SGD convergence, saddle points, and adaptive gradient methods.

Deep Learning: Data Representation and Multilayer Perceptron

Covers data representation, MLP training, activation functions, and gradient-based learning in deep neural networks.

Optimization Methods: RMSprop and ADAM

Explores RMSprop and ADAM optimization methods in Artificial Neural Networks, focusing on error functions, momentum, and signal-to-noise ratio.

Introduction to Learning by Stochastic Gradient Descent: Simple Perceptron

Covers the derivation of the stochastic gradient descent formula for a simple perceptron and explores the geometric interpretation of classification.

Gradient Noise: Smooth and Nonsmooth Risks

Explores the impact of gradient noise on optimization algorithms, focusing on smooth and nonsmooth risk functions and the derivation of gradient noise moments.

Convolutional Neural Networks

Covers Convolutional Neural Networks, including layers, training strategies, standard architectures, tasks like semantic segmentation, and deep learning tricks.