Linear Regression: Absence or Presence of Covariates

This lecture covers the fundamental concepts of linear regression, focusing on the absence or presence of covariates. It explains the distinction between marginal inference and regression, where observations are generated under different experimental conditions. The lecture delves into the bewildering variety of models that can be captured by the general specification of independent distributions. It also discusses the tools of the trade, starting from Gaussian linear regression and gradually generalizing to subspaces, projection matrices, and optimal dimension reduction. The concept of subspaces and spectra associated with real matrices is explored, along with orthogonal projections and the singular value decomposition theorem.