Feature Expansion: Kernels and KNN

This lecture covers the concepts of feature expansion, kernels, and K-nearest neighbors (KNN). It starts with a recap of linear models and multi-output prediction, then delves into non-linearity, probabilistic interpretation, and dealing with multiple classes. The lecture explains decision boundaries, support vector machines (SVM), and handling overlapping classes. It introduces polynomial curve fitting, feature expansion, and polynomial kernels. The instructor discusses kernel functions, Gaussian kernels, and the kernel trick. The lecture also covers kernel regression, the Representer theorem, and kernelization of algorithms like SVM. It concludes with demonstrations of kernel regression and KNN for classification and regression.