**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Lecture# Support Vector Machines: Optimal Margin Classifiers

Description

This lecture covers the concept of Support Vector Machines, focusing on the invention by Vapnik and Cortes. It explains the training algorithm for optimal margin classifiers, extending the results to non-separable training data. The lecture discusses the high generalization ability of support-vector networks and compares their performance. It delves into the hard-SVM rule for finding the max-margin separating hyperplane and provides proofs for equivalent formulations. The lecture also introduces the soft-SVM rule as a relaxation for non-linearly separable data, detailing the introduction of slack variables. It explores losses for classification, including quadratic, logistic, and hinge losses, and their behavioral differences. The lecture concludes with optimization techniques for finding the optimal hyperplane using convex duality and risk minimization.

Official source

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts (33)

Support vector machine

In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories by Vladimir Vapnik with colleagues (Boser et al., 1992, Guyon et al., 1993, Cortes and Vapnik, 1995, Vapnik et al., 1997) SVMs are one of the most robust prediction methods, being based on statistical learning frameworks or VC theory proposed by Vapnik (1982, 1995) and Chervonenkis (1974).

Loss functions for classification

In machine learning and mathematical optimization, loss functions for classification are computationally feasible loss functions representing the price paid for inaccuracy of predictions in classification problems (problems of identifying which category a particular observation belongs to). Given as the space of all possible inputs (usually ), and as the set of labels (possible outputs), a typical goal of classification algorithms is to find a function which best predicts a label for a given input .

Convex analysis

Convex analysis is the branch of mathematics devoted to the study of properties of convex functions and convex sets, often with applications in convex minimization, a subdomain of optimization theory. Convex set A subset of some vector space is if it satisfies any of the following equivalent conditions: If is real and then If is real and with then Convex function Throughout, will be a map valued in the extended real numbers with a domain that is a convex subset of some vector space.

Convex optimization

Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets). Many classes of convex optimization problems admit polynomial-time algorithms, whereas mathematical optimization is in general NP-hard.

K-nearest neighbors algorithm

In statistics, the k-nearest neighbors algorithm (k-NN) is a non-parametric supervised learning method first developed by Evelyn Fix and Joseph Hodges in 1951, and later expanded by Thomas Cover. It is used for classification and regression. In both cases, the input consists of the k closest training examples in a data set. The output depends on whether k-NN is used for classification or regression: In k-NN classification, the output is a class membership.

Related lectures (34)

Support Vector Machines: Maximizing MarginCS-433: Machine learning

Explores Support Vector Machines, maximizing margin for robust classification and the transition to soft SVM for non-linearly separable data.

Document Analysis: Topic ModelingDH-406: Machine learning for DH

Explores document analysis, topic modeling, and generative models for data generation in machine learning.

Introduction to Machine Learning: Supervised LearningCS-233(a): Introduction to machine learning (BA3)

Introduces supervised learning, covering classification, regression, model optimization, overfitting, and kernel methods.

Kernel Methods: Nonlinear Separation SurfacesEE-566: Adaptation and learning

Explores kernel methods for nonlinear separation surfaces using polynomial and Gaussian kernels in Perceptron and SVM algorithms.

The Hidden Convex Optimization Landscape of Deep Neural Networks

Explores the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models.