Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
This lecture introduces optimization in machine learning, focusing on gradient descent. It covers the analytical solution for linear regression, the lack of analytical solutions for logistic regression, and the use of specialized optimizers. The concept of convexity, stochastic gradient descent, and early stopping are explained. Practical considerations such as choosing a good learning rate and preprocessing data are discussed. The lecture also delves into solving the XOR problem without feature engineering using logistic regression and neural networks.