Maximum Likelihood Estimation: Properties and Applications

This lecture, given by the instructor on October 18, 2021, delves into the concept of Maximum Likelihood Estimation (MLE). The lecture covers topics such as taking partial derivatives to find MLE, verifying the maximum, and examples of MLE for Gaussian and Poisson trials. It also explores the equi-variance of MLEs, consistency of MLE in exponential families, and the distribution of MLE. The lecture concludes with discussions on the properties and assumptions related to MLE, including the consistency of MLE in different scenarios and the implications of regularity conditions. Through various examples and derivations, the lecture provides a comprehensive understanding of MLE and its applications.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (443)

Related concepts (78)

MATH-413: Statistics for data science

Statistics lies at the foundation of data science, providing a unifying theoretical and methodological backbone for the diverse tasks enountered in this emerging field. This course rigorously develops

Maximum likelihood estimation

In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate. The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference.

German tank problem

In the statistical theory of estimation, the German tank problem consists of estimating the maximum of a discrete uniform distribution from sampling without replacement. In simple terms, suppose there exists an unknown number of items which are sequentially numbered from 1 to N. A random sample of these items is taken and their sequence numbers observed; the problem is to estimate N from these observed numbers. The problem can be approached using either frequentist inference or Bayesian inference, leading to different results.

Bias of an estimator

In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called unbiased. In statistics, "bias" is an property of an estimator. Bias is a distinct concept from consistency: consistent estimators converge in probability to the true value of the parameter, but may be biased or unbiased; see bias versus consistency for more.

Chi-squared distribution

In probability theory and statistics, the chi-squared distribution (also chi-square or -distribution) with degrees of freedom is the distribution of a sum of the squares of independent standard normal random variables. The chi-squared distribution is a special case of the gamma distribution and is one of the most widely used probability distributions in inferential statistics, notably in hypothesis testing and in construction of confidence intervals.

Normal distribution

In statistics, a normal distribution or Gaussian distribution is a type of continuous probability distribution for a real-valued random variable. The general form of its probability density function is The parameter is the mean or expectation of the distribution (and also its median and mode), while the parameter is its standard deviation. The variance of the distribution is . A random variable with a Gaussian distribution is said to be normally distributed, and is called a normal deviate.

Maximum Likelihood Estimation MATH-413: Statistics for data science

Covers Maximum Likelihood Estimation, focusing on ML Estimation-Distribution, Shrinkage Estimation, and Loss functions.

Probability and Statistics MATH-232: Probability and statistics

Covers p-quantile, normal approximation, joint distributions, and exponential families in probability and statistics.

Interval Estimation: Method of Moments MATH-232: Probability and statistics

Covers the method of moments for estimating parameters and constructing confidence intervals based on empirical moments matching distribution moments.

Maximum Likelihood: Estimation and Inference EE-566: Adaptation and learning

Introduces maximum likelihood estimation, discussing its properties and applications in statistical analysis.

Boltzmann Machine PHYS-467: Machine learning for physicists

Introduces the Boltzmann Machine, covering expectation consistency, data clustering, and probability distribution functions.