Statistical Learning Models: Risk and Empirical Risk Minimization

This lecture introduces the concept of statistical learning models, which consist of observations, a class of functions, and a loss function. It explains the population risk and the goal of finding the function that minimizes it. The lecture then covers empirical risk minimization as an approach to approximate the optimal function by minimizing the empirical average of the loss. It delves into the strong law of large numbers and provides examples of empirical risk minimization in both parametric and non-parametric settings. The lecture concludes with discussions on estimators, loss functions, and the performance evaluation of maximum-likelihood estimators in various models.