**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Concept# Ordinary least squares

Summary

In statistics, ordinary least squares (OLS) is a type of linear least squares method for choosing the unknown parameters in a linear regression model (with fixed level-one effects of a linear function of a set of explanatory variables) by the principle of least squares: minimizing the sum of the squares of the differences between the observed dependent variable (values of the variable being observed) in the input dataset and the output of the (linear) function of the independent variable.
Geometrically, this is seen as the sum of the squared distances, parallel to the axis of the dependent variable, between each data point in the set and the corresponding point on the regression surface—the smaller the differences, the better the model fits the data. The resulting estimator can be expressed by a simple formula, especially in the case of a simple linear regression, in which there is a single regressor on the right side of the regression equation.
The OLS estimator is consistent for t

Official source

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications

Loading

Related people

Loading

Related units

Loading

Related concepts

Loading

Related courses

Loading

Related lectures

Loading

Related people

No results

Related publications (16)

Related units

No results

Loading

Loading

Loading

Related concepts (82)

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variable

Regression analysis

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a '

Statistics

Statistics (from German: Statistik, "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and present

Related courses (38)

MGT-581: Introduction to econometrics

The course provides an introduction to econometrics. The objective is to learn how to make valid (i.e., causal) inference from economic data. It explains the main estimators and present methods to deal with endogeneity issues.

DH-406: Machine learning for DH

This course aims to introduce the basic principles of machine learning in the context of the digital humanities. We will cover both supervised and unsupervised learning techniques, and study and implement methods to analyze diverse data types, such as images, music and social network data.

MATH-412: Statistical machine learning

A course on statistical machine learning for supervised and unsupervised learning

Nicolas Henri Bernard Flammarion, Loucas Pillaud-Vivien, Aditya Vardhan Varre

Motivated by the recent successes of neural networks that have the ability to fit the data perfectly \emph{and} generalize well, we study the noiseless model in the fundamental least-squares setup. We assume that an optimum predictor fits perfectly inputs and outputs $\langle \theta_* , \phi(X) \rangle = Y$, where $\phi(X)$ stands for a possibly infinite dimensional non-linear feature map. To solve this problem, we consider the estimator given by the last iterate of stochastic gradient descent (SGD) with constant step-size. In this context, our contribution is two fold: (i) \emph{from a (stochastic) optimization perspective}, we exhibit an archetypal problem where we can show explicitly the convergence of SGD final iterate for a non-strongly convex problem with constant step-size whereas usual results use some form of average and (ii) \emph{from a statistical perspective}, we give explicit non-asymptotic convergence rates in the over-parameterized setting and leverage a \emph{fine-grained} parameterization of the problem to exhibit polynomial rates that can be faster than $O(1/T)$. The link with reproducing kernel Hilbert spaces is established.

2021Related lectures (122)

The article develops the approach of Ferro and Segers (2003) to the estimation of the extremal index, and proposes the use of a new variable decreasing the bias of the likelihood based on the point process character of the exceedances. Two estimators are discussed: a maximum likelihood estimator and an iterative least squares estimator based on the normalized gaps between clusters. The first provides a flexible tool for use with smoothing methods. A diagnostic is given for the condition under which maximum likelihood is valid. The performance of the new estimators were tested by extensive simulations. An application to the Central England temperature series demonstrates the use of the maximum likelihood estimator together with smoothing methods.

2007The article develops the approach of Ferro and Segers (2003) to the estimation of the extremal index, and proposes the use of a new variable decreasing the bias of the likelihood based on the point process character of the exceedances. Two estimators are discussed: a maximum likelihood estimator and an iterative least squares estimator based on the normalized gaps between clusters. The first provides a flexible tool for use with smoothing methods. A diagnostic is given for the condition under which maximum likelihood is valid. The performance of the new estimators were tested by extensive simulations. An application to the Central England temperature series demonstrates the use of the maximum likelihood estimator together with smoothing methods.