Lecture
This lecture covers gradient descent with memory in R2, introducing momentum methods to improve convergence speed. It then delves into conjugate gradients with momentum, exploring the use of different beta rules and their impact on performance. The instructor also discusses nonlinear CG on manifolds, specifically Riemannian CG, highlighting the rich family of algorithms it yields and the delicate theory behind it.