Generalized least squares

In statistics, generalized least squares (GLS) is a method used to estimate the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in the regression model. Least squares and weighted least squares may need to be more statistically efficient and prevent misleading inferences. GLS was first described by Alexander Aitken in 1935. In standard linear regression models one observes data on n statistical units. The response values are placed in a vector , and the predictor values are placed in the design matrix , where is a vector of the k predictor variables (including a constant) for the ith unit. The model forces the conditional mean of given to be a linear function of and assumes the conditional variance of the error term given is a known nonsingular covariance matrix . This is usually written as Here is a vector of unknown constants (known as “regression coefficients”) that must be estimated from the data. Suppose is a candidate estimate for . Then the residual vector for will be . The generalized least squares method estimates by minimizing the squared Mahalanobis length of this residual vector: where the last two terms evaluate to scalars, resulting in This objective is a quadratic form in . Taking the gradient of this quadratic form with respect to and equating it to zero (when ) gives Therefore, the minimum of the objective function can be computed yielding the explicit formula: The quantity is known as the precision matrix (or dispersion matrix), a generalization of the diagonal weight matrix. The GLS estimator is unbiased, consistent, efficient, and asymptotically normal with and . GLS is equivalent to applying ordinary least squares to a linearly transformed version of the data. To see this, factor , for instance using the Cholesky decomposition. Then if one pre-multiplies both sides of the equation by , we get an equivalent linear model where , , and . In this model , where is the identity matrix.