Nonparametric regression | EPFL Graph Search

Are you an EPFL student looking for a semester project?

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Nonparametric regression is a category of regression analysis in which the predictor does not take a predetermined form but is constructed according to information derived from the data. That is, no parametric form is assumed for the relationship between predictors and dependent variable. Nonparametric regression requires larger sample sizes than regression based on parametric models because the data must supply the model structure as well as the model estimates. In nonparametric regression, we have random variables and and assume the following relationship: where is some deterministic function. Linear regression is a restricted case of nonparametric regression where is assumed to be affine. Some authors use a slightly stronger assumption of additive noise: where the random variable is the `noise term', with mean 0. Without the assumption that belongs to a specific parametric family of functions it is impossible to get an unbiased estimate for , however most estimators are consistent under suitable conditions. This is a non-exhaustive list of non-parametric models for regression. nearest neighbors, see nearest-neighbor interpolation and k-nearest neighbors algorithm regression trees kernel regression local regression multivariate adaptive regression splines smoothing splines neural networks Gaussian process regression In Gaussian process regression, also known as Kriging, a Gaussian prior is assumed for the regression curve. The errors are assumed to have a multivariate normal distribution and the regression curve is estimated by its posterior mode. The Gaussian prior may depend on unknown hyperparameters, which are usually estimated via empirical Bayes. The hyperparameters typically specify a prior covariance kernel. In case the kernel should also be inferred nonparametrically from the data, the critical filter can be used. Smoothing splines have an interpretation as the posterior mode of a Gaussian process regression.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (8)

Related people (5)

Related concepts (5)

Related courses (22)

Related lectures (237)

Local regression

Local regression or local polynomial regression, also known as moving regression, is a generalization of the moving average and polynomial regression. Its most common methods, initially developed for scatterplot smoothing, are LOESS (locally estimated scatterplot smoothing) and LOWESS (locally weighted scatterplot smoothing), both pronounced ˈloʊɛs. They are two strongly related non-parametric regression methods that combine multiple regression models in a k-nearest-neighbor-based meta-model.

Nonparametric regression

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Data Representations: Learning Methods CS-233(a): Introduction to machine learning (BA3)

Covers polynomial feature expansion, kernel functions, regression, and SVM, emphasizing the importance of choosing functions for feature expansion.

Kernel Regression PHYS-467: Machine learning for physicists

Covers the concept of kernel regression and making data linearly separable by adding features and using local methods.

Probabilistic Models for Linear Regression PHYS-467: Machine learning for physicists

Covers the probabilistic model for linear regression and its applications in nuclear magnetic resonance and X-ray imaging.

MATH-341: Linear models

Regression modelling is a fundamental tool of statistics, because it describes how the law of a random variable of interest may depend on other variables. This course aims to familiarize students with

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practi

DH-406: Machine learning for DH

This course aims to introduce the basic principles of machine learning in the context of the digital humanities. We will cover both supervised and unsupervised learning techniques, and study and imple

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing game

EPFL2022

Locality defeats the curse of dimensionality in convolutional teacher-student scenarios

Matthieu Wyart, Alessandro Favero, Francesco Cagnetta

Convolutional neural networks perform a local and translationally-invariant treatment of the data: quantifying which of these two aspects is central to their success remains a challenge. We study this

2021

Fréchet means and Procrustes analysis in Wasserstein space

Victor Panaretos, Yoav Zemel

We consider two statistical problems at the intersection of functional and non-Euclidean data analysis: the determination of a Fréchet mean in the Wasserstein space of multivariate distributions; and

2019