Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Concept
Nonparametric regression
Summary
Nonparametric regression is a category of regression analysis in which the predictor does not take a predetermined form but is constructed according to information derived from the data. That is, no parametric form is assumed for the relationship between predictors and dependent variable. Nonparametric regression requires larger sample sizes than regression based on parametric models because the data must supply the model structure as well as the model estimates.
In nonparametric regression, we have random variables and and assume the following relationship:
where is some deterministic function. Linear regression is a restricted case of nonparametric regression where is assumed to be affine.
Some authors use a slightly stronger assumption of additive noise:
where the random variable is the `noise term', with mean 0.
Without the assumption that belongs to a specific parametric family of functions it is impossible to get an unbiased estimate for , however most estimators are consistent under suitable conditions.
This is a non-exhaustive list of non-parametric models for regression.
nearest neighbors, see nearest-neighbor interpolation and k-nearest neighbors algorithm
regression trees
kernel regression
local regression
multivariate adaptive regression splines
smoothing splines
neural networks
Gaussian process regression
In Gaussian process regression, also known as Kriging, a Gaussian prior is assumed for the regression curve. The errors are assumed to have a multivariate normal distribution and the regression curve is estimated by its posterior mode. The Gaussian prior may depend on unknown hyperparameters, which are usually estimated via empirical Bayes.
The hyperparameters typically specify a prior covariance kernel. In case the kernel should also be inferred nonparametrically from the data, the critical filter can be used.
Smoothing splines have an interpretation as the posterior mode of a Gaussian process regression.
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.