Elastic net regularization

In statistics and, in particular, in the fitting of linear or logistic regression models, the elastic net is a regularized regression method that linearly combines the L1 and L2 penalties of the lasso and ridge methods. Specification The elastic net method overcomes the limitations of the LASSO (least absolute shrinkage and selection operator) method which uses a penalty function based on :|\beta|1 = \textstyle \sum{j=1}^p |\beta_j|. Use of this penalty function has several limitations. For example, in the "large p, small n" case (high-dimensional data with few examples), the LASSO selects at most n variables before it saturates. Also if there is a group of highly correlated variables, then the LASSO tends to select one variable from a group and ignore the others. To overcome these limitations, the elastic net adds a quadratic part (|\beta|^2) to the penalty, which when used alone is ridge regression (known also as Tikhonov regularization).

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications

Related people

Related units

Related concepts

Related courses

Related lectures

Summary

Official source

About this result

Related publications (8)

Related publications

Related people (1)

Related people

Related units (1)

Related units

Related concepts (6)

Related concepts

Related courses (7)

Related courses

Related lectures (8)

Related lectures

MATERIAL POINT METHOD A numerical evaluation for elastic problems

Mathilde Metral

Frozen, the movie, released in 2013, made a big impression on many people: some children were no doubt amazed by the talking snowman, others by the film’s music, the designers were probably more interesed in the reproduction quality of the snow modelled, by the Material Point Method (MPM). Many paperspraise the effectiveness of this method for simulating complex phenomena. However, very few studies have been carried out to verify the method on very simple academic problems; that is precisely what we will try to study throughout this paper. We will see that simple problem modeling is not so easy with MPM. We will try to present as well as possible the different factors influencing the results, before concluding on its advantages and disadvantages

2020

CoCoA: A General Framework for Communication-Efficient Distributed Optimization

Martin Jaggi, Chenxin Ma

The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets.

MICROTOME PUBL2018

A Combinatorial Algorithm to Compute Regularization Paths

Martin Jaggi

2009

Regularization (mathematics)

In mathematics, statistics, finance, computer science, particularly in machine learning and inverse problems, regularization is a process that changes the result answer to be "simpler". It is often u

Lasso (statistics)

In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso or LASSO) is a regression analysis method that performs both variable selection and regularizat

Linear regression

In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variable

MATH-115(a): Advanced linear algebra II

L'objectif du cours est d'introduire les notions de base de l'algèbre linéaire et de démontrer rigoureusement les résultats principaux de ce sujet.

COM-490: Large-scale data science for real-world data

This hands-on course teaches the tools & methods used by data scientists, from researching solutions to scaling up prototypes to Spark clusters. It exposes the students to the entire data science pipeline, from data acquisition to extracting valuable insights applied to real-world problems.

ME-421: System identification

Identification of discrete-time linear models using experimental data is studied. The correlation method and spectral analysis are used to identify nonparametric models and the subspace and prediction error methods to estimate the plant and noise model parameters. Hands-on labs are included.