Generalized least squares | EPFL Graph Search

Are you an EPFL student looking for a semester project?

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

In statistics, generalized least squares (GLS) is a method used to estimate the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in the regression model. Least squares and weighted least squares may need to be more statistically efficient and prevent misleading inferences. GLS was first described by Alexander Aitken in 1935. In standard linear regression models one observes data on n statistical units. The response values are placed in a vector , and the predictor values are placed in the design matrix , where is a vector of the k predictor variables (including a constant) for the ith unit. The model forces the conditional mean of given to be a linear function of and assumes the conditional variance of the error term given is a known nonsingular covariance matrix . This is usually written as Here is a vector of unknown constants (known as “regression coefficients”) that must be estimated from the data. Suppose is a candidate estimate for . Then the residual vector for will be . The generalized least squares method estimates by minimizing the squared Mahalanobis length of this residual vector: where the last two terms evaluate to scalars, resulting in This objective is a quadratic form in . Taking the gradient of this quadratic form with respect to and equating it to zero (when ) gives Therefore, the minimum of the objective function can be computed yielding the explicit formula: The quantity is known as the precision matrix (or dispersion matrix), a generalization of the diagonal weight matrix. The GLS estimator is unbiased, consistent, efficient, and asymptotically normal with and . GLS is equivalent to applying ordinary least squares to a linearly transformed version of the data. To see this, factor , for instance using the Cholesky decomposition. Then if one pre-multiplies both sides of the equation by , we get an equivalent linear model where , , and . In this model , where is the identity matrix.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (1)

Related concepts (15)

Related courses (49)

Related lectures (404)

FIN-403: Econometrics

The course covers basic econometric models and methods that are routinely applied to obtain inference results in economic and financial applications.

PHYS-467: Machine learning for physicists

Machine learning and data analysis are becoming increasingly central in sciences including physics. In this course, fundamental principles and methods of machine learning will be introduced and practi

MATH-342: Time series

A first course in statistical time series analysis and applications.

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Generalized least squares

Homoscedasticity and heteroscedasticity

In statistics, a sequence (or a vector) of random variables is homoscedastic (ˌhoʊmoʊskəˈdæstɪk) if all its random variables have the same finite variance; this is also known as homogeneity of variance. The complementary notion is called heteroscedasticity, also known as heterogeneity of variance. The spellings homoskedasticity and heteroskedasticity are also frequently used.

Log Barriers for Safe Non-convex Black-box Optimization

Maryam Kamgarpour, Andreas Krause, Ilnura Usmanova

We address the problem of minimizing a smooth function f0(x) over a compact set D defined by smooth functional constraints fi(x)≤0, i=1,…,m given noisy value measurements of fi(x). This problem arises in safety-critical applications, where certain parameters need to be adapted online in a data-driven fashion, such as in personalized medicine, robotics, manufacturing, etc. In such cases, it is important to ensure constraints are not violated while taking measurements and seeking the minimum of the cost function. We propose a new algorithm s0-LBM, which provides provably feasible iterates with high probability and applies to the challenging case of uncertain zero-th order oracle. We also analyze the convergence rate of the algorithm, and empirically demonstrate its effectiveness.

2019

Density matrix and von Neumann entropy: first intro COM-309: Introduction to quantum information processing

Introduces density matrix and von Neumann entropy in quantum systems.

Heteroskedasticity: Ch. 4a FIN-403: Econometrics

Explores heteroskedasticity in econometrics, discussing its impact on standard errors, alternative estimators, testing methods, and implications for hypothesis testing.

Matrix Equivalence Theorems

Explores matrix equivalence theorems for systems of equations and least squares solutions.