Least-angle regression

In statistics, least-angle regression (LARS) is an algorithm for fitting linear regression models to high-dimensional data, developed by Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani. Suppose we expect a response variable to be determined by a linear combination of a subset of potential covariates. Then the LARS algorithm provides a means of producing an estimate of which variables to include, as well as their coefficients. Instead of giving a vector result, the LARS solution consists of a curve denoting the solution for each value of the L1 norm of the parameter vector. The algorithm is similar to forward stepwise regression, but instead of including variables at each step, the estimated parameters are increased in a direction equiangular to each one's correlations with the residual. The advantages of the LARS method are: It is computationally just as fast as forward selection. It produces a full piecewise linear solution path, which is useful in cross-validation or similar attempts to tune the model. If two variables are almost equally correlated with the response, then their coefficients should increase at approximately the same rate. The algorithm thus behaves as intuition would expect, and also is more stable. It is easily modified to produce efficient algorithms for other methods producing similar results, like the lasso and forward stagewise regression. It is effective in contexts where p ≫ n (i.e., when the number of predictors p is significantly greater than the number of points n) The disadvantages of the LARS method include: With any amount of noise in the dependent variable and with high dimensional multicollinear independent variables, there is no reason to believe that the selected variables will have a high probability of being the actual underlying causal variables. This problem is not unique to LARS, as it is a general problem with variable selection approaches that seek to find underlying deterministic components.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (57)

Related people (11)

Related units (15)

Related concepts (3)

Related courses (11)

Related lectures (37)

FIN-616: Financial Econometrics II (2020 -2024)

This course has 3 parts

We understand how to use moment based estimations to obtain the parameters for explicit or implicit models.
We learn how to estimate latent parameters in a time series cont

CS-433: Machine learning

Machine learning methods are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analyzed and pr

MGT-502: Data science and machine learning

Hands-on introduction to data science and machine learning. We explore recommender systems, generative AI, chatbots, graphs, as well as regression, classification, clustering, dimensionality reduction

Lasso (statistics)

In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the resulting statistical model. It was originally introduced in geophysics, and later by Robert Tibshirani, who coined the term. Lasso was originally formulated for linear regression models. This simple case reveals a substantial amount about the estimator.

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.

Cross-validation (statistics)

Cross-validation, sometimes called rotation estimation or out-of-sample testing, is any of various similar model validation techniques for assessing how the results of a statistical analysis will generalize to an independent data set. Cross-validation is a resampling method that uses different portions of the data to test and train a model on different iterations. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice.

Model reduction of coupled systems based on non-intrusive approximations of the boundary response maps

Jan Sickmann Hesthaven, Niccolo' Discacciati

We propose a local, non -intrusive model order reduction technique to accurately approximate the solution of coupled multi -component parametrized systems governed by partial differential equations. Our approach is based on the approximation of the boundar ...

Lausanne2024

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Scott William Pesme

In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until reaching the minim ...

2023

Acute TNF alpha levels predict cognitive impairment 6-9 months after COVID-19 infection

Dimitri Nestor Alice Van De Ville, Alessandra Griffa, Idris Guessous, Alexandre Cionca

Background: A neurocognitive phenotype of post-COVID-19 infection has recently been described that is characterized by a lack of awareness of memory impairment (i.e., anosognosia), altered functional connectivity in the brain's default mode and limbic netw ...

PERGAMON-ELSEVIER SCIENCE LTD2023

Linear Regression: Statistical Inference and Regularization PHYS-467: Machine learning for physicists

Covers the probabilistic model for linear regression and the importance of regularization techniques.

Generalization Error in Learning with Random Features PHYS-754: Lecture series on scientific machine learning

Explores generalization error in learning theory, emphasizing structured data and the replica method.

ANOVA: Partitioning Total SS MSE-213: Probability and statistics for materials science

Covers ANOVA method, focusing on partitioning total sum of squares into treatment and error components, mean square calculations, Fisher statistic, and F-distribution.