Linear Binary Classification: Perceptron, SGD, Fisher's LDA

In course

Description

This lecture covers the Perceptron model, including the learning algorithm for separable and non-separable cases, Stochastic Gradient Descent, and Fisher's Linear Discriminant. It also discusses the convergence of algorithms, risk minimization, and generative classification.

Official source

https://mediaspace.epfl.ch/media/0_fgk2hw5a

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (34)

Related concepts (33)

Separable space

In mathematics, a topological space is called separable if it contains a countable, dense subset; that is, there exists a sequence of elements of the space such that every nonempty open subset of the space contains at least one element of the sequence. Like the other axioms of countability, separability is a "limitation on size", not necessarily in terms of cardinality (though, in the presence of the Hausdorff axiom, this does turn out to be the case; see below) but in a more subtle topological sense.

Linear discriminant analysis

Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in statistics and other fields, to find a linear combination of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification.

Separable extension

In field theory, a branch of algebra, an algebraic field extension is called a separable extension if for every , the minimal polynomial of over F is a separable polynomial (i.e., its formal derivative is not the zero polynomial, or equivalently it has no repeated roots in any extension field). There is also a more general definition that applies when E is not necessarily algebraic over F. An extension that is not separable is said to be inseparable.

Separable polynomial

In mathematics, a polynomial P(X) over a given field K is separable if its roots are distinct in an algebraic closure of K, that is, the number of distinct roots is equal to the degree of the polynomial. This concept is closely related to square-free polynomial. If K is a perfect field then the two concepts coincide. In general, P(X) is separable if and only if it is square-free over any field that contains K, which holds if and only if P(X) is coprime to its formal derivative D P(X).

Linear classifier

In the field of machine learning, the goal of statistical classification is to use an object's characteristics to identify which class (or group) it belongs to. A linear classifier achieves this by making a classification decision based on the value of a linear combination of the characteristics. An object's characteristics are also known as feature values and are typically presented to the machine in a vector called a feature vector.