Optimization Methods: RMSprop and ADAM

About
Privacy
Disclaimer

Graph Chatbot

Related lectures (30)

Page 3 of 3

Introduces deep learning fundamentals, covering data representations, neural networks, and convolutional neural networks.

Deep Learning: Convolutional Neural Networks

Covers Convolutional Neural Networks, standard architectures, training techniques, and adversarial examples in deep learning.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Deep Learning III

Delves into deep learning optimization, challenges, SGD variants, critical points, overparametrized networks, and adaptive methods.

Double Descent Curves: Overparametrization

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Neural Networks: Multilayer Learning

Covers the fundamentals of multilayer neural networks and deep learning, including back-propagation and network architectures like LeNet, AlexNet, and VGG-16.

Gradient Descent: Linear Regression

Covers the concept of gradient descent for linear regression, explaining the iterative process of updating parameters.

Structures in Non-Convex Optimization

Covers non-convex optimization, deep learning training problems, stochastic gradient descent, adaptive methods, and neural network architectures.