Lecture
Stochastic Gradient Descent: Theory and Applications
Related lectures (29)
Deep Learning Building Blocks
Covers tensors, loss functions, autograd, and convolutional layers in deep learning.
SGD and Mean Field Analysis
Explores Stochastic Gradient Descent and Mean Field Analysis in two-layer neural networks, emphasizing their iterative processes and mathematical foundations.
Advanced Physics I: Introduction to Mechanics
Introduces the basics of physics, including mechanics and making predictions based on observations and hypotheses.
Deep Learning for Autonomous Vehicles: Learning
Explores learning in deep learning for autonomous vehicles, covering predictive models, RNN, ImageNet, and transfer learning.
Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent
Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.
Group Theory Fundamentals
Introduces the basics of group theory, defining groups, categories, and groupoids.
Optimization: Gradient Descent and Subgradients
Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.
Landscape and Generalisation in Deep Learning
Explores the challenges and insights of deep learning, focusing on loss landscape, generalization, and feature learning.
Topological Data Science
Delves into Topological Data Analysis, emphasizing the mathematical foundations of neural networks and exploring the manifold hypothesis and persistent homology.
Variance Reduction Techniques
Covers variance reduction techniques in optimization, focusing on gradient descent and stochastic gradient descent methods.