Feature Expansion and Kernels

This lecture covers the concepts of feature expansion and kernels in machine learning. It starts with a recap of linear models, margin, and maximum margin classifier. Then, it delves into support vector machines (SVM) formulation, overlapping classes, and slack variables. The lecture explains the differences between least-square classification, logistic regression, and SVM. It introduces the concept of polynomial feature expansion and its application in nonlinear classification. The lecture also discusses the curse of dimensionality, nearest neighbor methods, and k-nearest neighbors for classification and regression. It concludes with kernel functions, kernel trick, kernel regression, and kernel SVM for classification.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (104)

Related concepts (44)

DH-406: Machine learning for DH

This course aims to introduce the basic principles of machine learning in the context of the digital humanities. We will cover both supervised and unsupervised learning techniques, and study and imple

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Kernel regression

In statistics, kernel regression is a non-parametric technique to estimate the conditional expectation of a random variable. The objective is to find a non-linear relation between a pair of random variables X and Y. In any nonparametric regression, the conditional expectation of a variable relative to a variable may be written: where is an unknown function. Nadaraya and Watson, both in 1964, proposed to estimate as a locally weighted average, using a kernel as a weighting function.

Radial basis function kernel

In machine learning, the radial basis function kernel, or RBF kernel, is a popular kernel function used in various kernelized learning algorithms. In particular, it is commonly used in support vector machine classification. The RBF kernel on two samples and x', represented as feature vectors in some input space, is defined as may be recognized as the squared Euclidean distance between the two feature vectors. is a free parameter.

Document Analysis: Topic Modeling DH-406: Machine learning for DH

Explores document analysis, topic modeling, and generative models for data generation in machine learning.

Introduction to Machine Learning: Supervised Learning CS-233(a): Introduction to machine learning (BA3)

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Feature Expansion: Kernels and KNN DH-406: Machine learning for DH

Covers feature expansion, kernels, and K-nearest neighbors, including non-linearity, SVM, and Gaussian kernels.

Kernel Methods: SVM and Regression DH-406: Machine learning for DH

Introduces kernel methods like SVM and regression, covering concepts such as margin, support vector machine, curse of dimensionality, and Gaussian process regression.

Neural Networks: Training and Activation CIVIL-226: Introduction to machine learning for engineers

Explores neural networks, activation functions, backpropagation, and PyTorch implementation.