Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of Graph Search.
Concept
Bayes error rate
Summary
In statistical classification, Bayes error rate is the lowest possible error rate for any classifier of a random outcome (into, for example, one of two categories) and is analogous to the irreducible error.
A number of approaches to the estimation of the Bayes error rate exist. One method seeks to obtain analytical bounds which are inherently dependent on distribution parameters, and hence difficult to estimate. Another approach focuses on class densities, while yet another method combines and compares various classifiers.
The Bayes error rate finds important use in the study of patterns and machine learning techniques.
In terms of machine learning and pattern classification, the labels of a set of random observations can be divided into 2 or more classes. Each observation is called an instance and the class it belongs to is the label.
The Bayes error rate of the data distribution is the probability an instance is misclassified by a classifier that knows the true class probabilities given the predictors.
For a multiclass classifier, the expected prediction error may be calculated as follows:
where x is the instance, the expectation value, Ck is a class into which an instance is classified, P(Ck|x) is the conditional probability of label k for instance x, and L() is the 0–1 loss function:
where is the Kronecker delta.
When the learner knows the conditional probability, then one solution is:
This solution is known as the Bayes classifier.
The corresponding expected Prediction Error is called the Bayes error rate:
where the sum can be omitted in the last step due to considering the counter event.
By the definition of the Bayes classifier, it maximizes and, therefore, minimizes the Bayes error BE.
The Bayes error is non-zero if the classification labels are not deterministic, i.e., there is a non-zero probability of a given instance belonging to more than one class.. In a regression context with squared error, the Bayes error is equal to the noise variance.
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.