Linear Models for Classification

This lecture covers linear models for classification, starting with a recap on simple parametric models and hyperplanes. It delves into linear regression, binary classification, adding non-linearity, logistic regression, and evaluating classifiers. The instructor explains the concept of margin between classes, the formulation for a maximum margin classifier (SVM), and basic notions of constrained optimization. The lecture also touches on decision boundaries, support vectors, and the SVM formulation. An interlude on constrained optimization and Lagrange duality is included, followed by the derivation of the Lagrange dual problem. The use of slack variables in SVM is discussed, along with the Lagrangian and the optimal values. The lecture concludes with a comparison of linear models for classification.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (342)

Related concepts (109)

CS-233(a): Introduction to machine learning (BA3)

Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analy

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Duality (optimization)

In mathematical optimization theory, duality or the duality principle is the principle that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. If the primal is a minimization problem then the dual is a maximization problem (and vice versa). Any feasible solution to the primal (minimization) problem is at least as large as any feasible solution to the dual (maximization) problem.

Multiclass classification

In machine learning and statistical classification, multiclass classification or multinomial classification is the problem of classifying instances into one of three or more classes (classifying instances into one of two classes is called binary classification). While many classification algorithms (notably multinomial logistic regression) naturally permit the use of more than two classes, some are by nature binary algorithms; these can, however, be turned into multinomial classifiers by a variety of strategies.

Nonlinear regression

In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations. In nonlinear regression, a statistical model of the form, relates a vector of independent variables, , and its associated observed dependent variables, . The function is nonlinear in the components of the vector of parameters , but otherwise arbitrary.

Bayesian linear regression

Bayesian linear regression is a type of conditional modeling in which the mean of one variable is described by a linear combination of other variables, with the goal of obtaining the posterior probability of the regression coefficients (as well as other parameters describing the distribution of the regressand) and ultimately allowing the out-of-sample prediction of the regressand (often labelled ) conditional on observed values of the regressors (usually ).

Explores linear models for classification, including parametric models, regression, and logistic regression, along with model evaluation metrics and maximum margin classifiers.

Explores linear models, logistic regression, classification metrics, SVM, and their practical use in data science methods.

Explores linear models for classification, logistic regression, SVM, k-NN, and curse of dimensionality.

Introduces linear models in machine learning, covering basics, parametric models, multi-output regression, and evaluation metrics.

Covers the basics of linear regression, binary and multi-class classification, and evaluation metrics.