Model Diagnostics: Outliers, Leverage, and Influential Observations

This lecture covers the identification and assessment of outliers, leverage points, and influential observations in statistical models. It discusses methods to detect uncorrelatedness, clustering, and the impact of influential observations on model validity. The instructor explains how to visually check for outliers in regression analysis and the concept of leverage in model fitting. Additionally, the lecture explores Cook's distance for assessing the influence of individual observations and diagnostic plots for model evaluation, including nested model selection. The presentation concludes with residual sums of squares and the comparison of nested models in the context of Gaussian linear models.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (37)

Related concepts (42)

MATH-413: Statistics for data science

Statistics lies at the foundation of data science, providing a unifying theoretical and methodological backbone for the diverse tasks enountered in this emerging field. This course rigorously develops

Regression analysis

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome' or 'response' variable, or a 'label' in machine learning parlance) and one or more independent variables (often called 'predictors', 'covariates', 'explanatory variables' or 'features'). The most common form of regression analysis is linear regression, in which one finds the line (or a more complex linear combination) that most closely fits the data according to a specific mathematical criterion.

Partition of sums of squares

The partition of sums of squares is a concept that permeates much of inferential statistics and descriptive statistics. More properly, it is the partitioning of sums of squared deviations or errors. Mathematically, the sum of squared deviations is an unscaled, or unadjusted measure of dispersion (also called variability). When scaled for the number of degrees of freedom, it estimates the variance, or spread of the observations about their mean value.

Lack-of-fit sum of squares

In statistics, a sum of squares due to lack of fit, or more tersely a lack-of-fit sum of squares, is one of the components of a partition of the sum of squares of residuals in an analysis of variance, used in the numerator in an F-test of the null hypothesis that says that a proposed model fits well. The other component is the pure-error sum of squares. The pure-error sum of squares is the sum of squared deviations of each value of the dependent variable from the average value over all observations sharing its independent variable value(s).

Statistical model validation

In statistics, model validation is the task of evaluating whether a chosen statistical model is appropriate or not. Oftentimes in statistical inference, inferences from models that appear to fit their data may be flukes, resulting in a misunderstanding by researchers of the actual relevance of their model. To combat this, model validation is used to test whether a statistical model can hold up to permutations in the data.

Least squares

The method of least squares is a standard approach in regression analysis to approximate the solution of overdetermined systems (sets of equations in which there are more equations than unknowns) by minimizing the sum of the squares of the residuals (a residual being the difference between an observed value and the fitted value provided by a model) made in the results of each individual equation. The most important application is in data fitting.

Weighted Least Squares Estimation: IRLS Algorithm MATH-413: Statistics for data science

Explores the IRLS algorithm for weighted least squares estimation in GLM.

Regression: Simple and Multiple Linear

Covers simple and multiple linear regression, including least squares estimation and model diagnostics.

Linear Regression: Statistical Inference and Regularization PHYS-467: Machine learning for physicists

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Model Selection Criteria: AIC, BIC, Cp MATH-413: Statistics for data science

Explores model selection criteria like AIC, BIC, and Cp in statistics for data science.

Supervised Learning: Linear Regression BIO-322: Introduction to machine learning for bioengineers

Covers supervised learning with a focus on linear regression, including topics like digit classification, spam detection, and wind speed prediction.