Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.
This lecture covers Accelerated Gradient Descent, Gradient-free optimization, and their applications in machine learning. It discusses the speed of gradient descent on smooth convex functions, the concept of Nesterov's accelerated gradient descent, error bounds, potential functions, and the convergence rate for derivative-free random search. Additionally, it explores adaptive and other SGD methods such as Adagrad, Adam, and SignSGD, highlighting their advantages and practical implications.