**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Concept# Radial basis function kernel

Summary

In machine learning, the radial basis function kernel, or RBF kernel, is a popular kernel function used in various kernelized learning algorithms. In particular, it is commonly used in support vector machine classification.
The RBF kernel on two samples and x', represented as feature vectors in some input space, is defined as
may be recognized as the squared Euclidean distance between the two feature vectors. is a free parameter. An equivalent definition involves a parameter :
Since the value of the RBF kernel decreases with distance and ranges between zero (in the limit) and one (when x = x'), it has a ready interpretation as a similarity measure.
The feature space of the kernel has an infinite number of dimensions; for , its expansion using the multinomial theorem is:
where ,
Because support vector machines and other models employing the kernel trick do not scale well to large numbers of training samples or large numbers of features in the input space, several approximations to the RBF kernel (and similar kernels) have been introduced.
Typically, these take the form of a function z that maps a single vector to a vector of higher dimensionality, approximating the kernel:
where is the implicit mapping embedded in the RBF kernel.
One way to construct such a z is to randomly sample from the Fourier transformation of the kernelwhere are independent samples from the normal distribution .
Theorem: .
Proof: It suffices to prove the case of . Use the trigonometric identity , the spherical symmetry of gaussian distribution, then evaluate the integral .
Theorem: . (Appendix A.2).
Another approach uses the Nyström method to approximate the eigendecomposition of the Gram matrix K, using only a random sample of the training set.

Official source

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications

Loading

Related people

No results

Related units

No results

Related concepts

Loading

Related courses (11)

DH-406: Machine learning for DH

This course aims to introduce the basic principles of machine learning in the context of the digital humanities. We will cover both supervised and unsupervised learning techniques, and study and imple

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practi

CS-233(a): Introduction to machine learning (BA3)

Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analy

Related lectures

Loading

Related MOOCs

No results

Related publications (1)

Related concepts (5)

Related lectures (77)

Kernel method

In machine learning, kernel machines are a class of algorithms for pattern analysis, whose best known member is the support-vector machine (SVM). These methods involve using linear classifiers to solve nonlinear problems. The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets.

Polynomial kernel

In machine learning, the polynomial kernel is a kernel function commonly used with support vector machines (SVMs) and other kernelized models, that represents the similarity of vectors (training samples) in a feature space over polynomials of the original variables, allowing learning of non-linear models. Intuitively, the polynomial kernel looks not only at the given features of input samples to determine their similarity, but also combinations of these. In the context of regression analysis, such combinations are known as interaction features.

Radial basis function kernel

In machine learning, the radial basis function kernel, or RBF kernel, is a popular kernel function used in various kernelized learning algorithms. In particular, it is commonly used in support vector machine classification. The RBF kernel on two samples and x', represented as feature vectors in some input space, is defined as may be recognized as the squared Euclidean distance between the two feature vectors. is a free parameter.

Incremental Regression: LWPR

Explores incremental learning with LWPR, discussing challenges, synthetic data, real-world applications, and the LWPR algorithm.

Kernel Methods: Neural Networks

Covers the fundamentals of neural networks, focusing on RBF kernels and SVM.

Kernel Regression: Weighted Average and Feature Maps

Covers kernel regression and feature maps for data separability.

Loading

Multiple generalized additive models are a class of statistical regression models wherein parameters of probability distributions incorporate information through additive smooth functions of predictor