Feature Expansion and Kernel Methods

This lecture covers the concepts of feature expansion and kernel methods in machine learning. It starts with a recap of linear models, margin, maximum margin classifier, and support vector machines. It then delves into topics like overlapping classes, slack variables, different linear models for classification, and the impact of loss functions. The lecture progresses to discuss polynomial curve fitting, nearest neighbor methods, k-nearest neighbors, curse of dimensionality, and nonlinear classification. It concludes with an in-depth exploration of polynomial feature expansion, kernel functions, kernel regression, kernel SVM, and the kernel trick.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (382)

Related concepts (164)

This course aims to introduce the basic principles of machine learning in the context of the digital humanities. We will cover both supervised and unsupervised learning techniques, and study and imple

Nearest neighbor search

Nearest neighbor search (NNS), as a form of proximity search, is the optimization problem of finding the point in a given set that is closest (or most similar) to a given point. Closeness is typically expressed in terms of a dissimilarity function: the less similar the objects, the larger the function values. Formally, the nearest-neighbor (NN) search problem is defined as follows: given a set S of points in a space M and a query point q ∈ M, find the closest point in S to q. Donald Knuth in vol.

K-nearest neighbors algorithm

In statistics, the k-nearest neighbors algorithm (k-NN) is a non-parametric supervised learning method first developed by Evelyn Fix and Joseph Hodges in 1951, and later expanded by Thomas Cover. It is used for classification and regression. In both cases, the input consists of the k closest training examples in a data set. The output depends on whether k-NN is used for classification or regression: In k-NN classification, the output is a class membership.

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Kernel (statistics)

The term kernel is used in statistical analysis to refer to a window function. The term "kernel" has several distinct meanings in different branches of statistics. In statistics, especially in Bayesian statistics, the kernel of a probability density function (pdf) or probability mass function (pmf) is the form of the pdf or pmf in which any factors that are not functions of any of the variables in the domain are omitted. Note that such factors may well be functions of the parameters of the pdf or pmf.

Polynomial regression

In statistics, polynomial regression is a form of regression analysis in which the relationship between the independent variable x and the dependent variable y is modelled as an nth degree polynomial in x. Polynomial regression fits a nonlinear relationship between the value of x and the corresponding conditional mean of y, denoted E(y |x). Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function E(y | x) is linear in the unknown parameters that are estimated from the data.

Feature Expansion and Kernels DH-406: Machine learning for DH

Covers feature expansion, kernels, SVM, and nonlinear classification in machine learning.

Kernel Methods: SVM and Regression DH-406: Machine learning for DH

Introduces kernel methods like SVM and regression, covering concepts such as margin, support vector machine, curse of dimensionality, and Gaussian process regression.

Feature Expansion: Kernels and KNN DH-406: Machine learning for DH

Covers feature expansion, kernels, and K-nearest neighbors, including non-linearity, SVM, and Gaussian kernels.

Document Analysis: Topic Modeling DH-406: Machine learning for DH

Explores document analysis, topic modeling, and generative models for data generation in machine learning.

Kernel Regression: Weighted Average and Feature Maps PHYS-467: Machine learning for physicists

Covers kernel regression and feature maps for data separability.