Non-parametric Regression: Smoothing Techniques

This lecture by the instructor covers non-parametric regression techniques, focusing on splines, natural spline basis functions, penalized likelihood, smoothing matrices, cubic spline fits, bias-variance tradeoff, cross-validation, orthogonal functions, Fourier series expansion, kernel smoothing, wavelets, and modulation estimators. The lecture also discusses the estimation of smooth functions, orthogonal bases, Sobolev ellipsoids, modulators, risk minimization, and thresholding techniques. Additionally, it explores the use of orthogonal functions in regression, the choice of basis functions, and the implications of dimensionality in multivariate regression.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (140)

Related concepts (167)

MATH-413: Statistics for data science

Statistics lies at the foundation of data science, providing a unifying theoretical and methodological backbone for the diverse tasks enountered in this emerging field. This course rigorously develops

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Nonparametric statistics

Nonparametric statistics is the type of statistics that is not restricted by assumptions concerning the nature of the population from which a sample is drawn. This is opposed to parametric statistics, for which a problem is restricted a priori by assumptions concerning the specific distribution of the population (such as the normal distribution) and parameters (such the mean or variance).

Errors-in-variables models

In statistics, errors-in-variables models or measurement error models are regression models that account for measurement errors in the independent variables. In contrast, standard regression models assume that those regressors have been measured exactly, or observed without error; as such, those models account only for errors in the dependent variables, or responses. In the case when some regressors have been measured with errors, estimation based on the standard assumption leads to inconsistent estimates, meaning that the parameter estimates do not tend to the true values even in very large samples.

Local regression

Local regression or local polynomial regression, also known as moving regression, is a generalization of the moving average and polynomial regression. Its most common methods, initially developed for scatterplot smoothing, are LOESS (locally estimated scatterplot smoothing) and LOWESS (locally weighted scatterplot smoothing), both pronounced ˈloʊɛs. They are two strongly related non-parametric regression methods that combine multiple regression models in a k-nearest-neighbor-based meta-model.

Segmented regression

Segmented regression, also known as piecewise regression or broken-stick regression, is a method in regression analysis in which the independent variable is partitioned into intervals and a separate line segment is fit to each interval. Segmented regression analysis can also be performed on multivariate data by partitioning the various independent variables. Segmented regression is useful when the independent variables, clustered into different groups, exhibit different relationships between the variables in these regions.

Probabilistic Models for Linear Regression PHYS-467: Machine learning for physicists

Covers the probabilistic model for linear regression and its applications in nuclear magnetic resonance and X-ray imaging.

Flexibility of Models & Bias-Variance Trade-Off BIO-322: Introduction to machine learning for bioengineers

Delves into the trade-off between model flexibility and bias-variance in error decomposition, polynomial regression, KNN, and the curse of dimensionality.

Nonparametric Regression: Kernel-Based Estimation

Covers nonparametric regression using kernel-based estimation techniques to model complex relationships between variables.

Kernel Regression: Weighted Average and Feature Maps PHYS-467: Machine learning for physicists

Covers kernel regression and feature maps for data separability.

Logistic Regression: Vegetation Prediction

Explores logistic regression for predicting vegetation proportions in the Amazon region through remote sensing data analysis.