Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Concept
Orthogonality principle
Summary
In statistics and signal processing, the orthogonality principle is a necessary and sufficient condition for the optimality of a Bayesian estimator. Loosely stated, the orthogonality principle says that the error vector of the optimal estimator (in a mean square error sense) is orthogonal to any possible estimator. The orthogonality principle is most commonly stated for linear estimators, but more general formulations are possible. Since the principle is a necessary and sufficient condition for optimality, it can be used to find the minimum mean square error estimator.
The orthogonality principle is most commonly used in the setting of linear estimation. In this context, let x be an unknown random vector which is to be estimated based on the observation vector y. One wishes to construct a linear estimator for some matrix H and vector c. Then, the orthogonality principle states that an estimator achieves minimum mean square error if and only if
and
If x and y have zero mean, then it suffices to require the first condition.
Suppose x is a Gaussian random variable with mean m and variance Also suppose we observe a value where w is Gaussian noise which is independent of x and has mean 0 and variance We wish to find a linear estimator minimizing the MSE. Substituting the expression into the two requirements of the orthogonality principle, we obtain
and
Solving these two linear equations for h and c results in
so that the linear minimum mean square error estimator is given by
This estimator can be interpreted as a weighted average between the noisy measurements y and the prior expected value m. If the noise variance is low compared with the variance of the prior (corresponding to a high SNR), then most of the weight is given to the measurements y, which are deemed more reliable than the prior information. Conversely, if the noise variance is relatively higher, then the estimate will be close to m, as the measurements are not reliable enough to outweigh the prior information.
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.