Statistical Estimation: Models and Learning Machines

This lecture introduces the role of models and data in statistical estimation, focusing on maximum-likelihood formulations and sample complexity bounds for estimation and prediction. It covers topics such as parametric estimation models, least-squares estimators, loss functions, ML estimators, learning machines, M-estimators, and practical issues in estimation. The instructor discusses the performance of estimators, the ML estimator for quantum tomography, and the comparison between ML and James-Stein estimators. The lecture also addresses the challenges of overfitting, the role of computation in estimation, and the practical performance of learning machines. It concludes with a discussion on the estimation of parameters versus the estimation of risk.