**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Lecture# Kernel PCA: Nonlinear Dimensionality Reduction

Description

This lecture covers Kernel Principal Component Analysis (KPCA), a nonlinear version of PCA that uses kernels to make the problem linear. It starts with a recap of Principal Component Analysis (PCA) and its optimization. Then, it delves into the derivation of KPCA, explaining how data is sent to a feature space and the eigenvalue problem is solved using the kernel trick. The solutions to the dual eigenvalue problem are discussed, emphasizing the computational intensity of KPCA due to the eigenvalue decomposition of the Gram matrix. The lecture concludes by highlighting that KPCA offers an alternative to standard PCA by not assuming a linear transformation and utilizing the kernel trick for projection determination.

Official source

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

In course

Related concepts (34)

Instructor

MICRO-455: Applied machine learning

Real-world engineering applications must cope with a large dataset of dynamic variables, which cannot be well approximated by classical or deterministic models. This course gives an overview of method

Principal component analysis (PCA) is a popular technique for analyzing large datasets containing a high number of dimensions/features per observation, increasing the interpretability of data while preserving the maximum amount of information, and enabling the visualization of multidimensional data. Formally, PCA is a statistical technique for reducing the dimensionality of a dataset. This is accomplished by linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.

In the field of multivariate statistics, kernel principal component analysis (kernel PCA) is an extension of principal component analysis (PCA) using techniques of kernel methods. Using a kernel, the originally linear operations of PCA are performed in a reproducing kernel Hilbert space. Recall that conventional PCA operates on zero-centered data; that is, where is one of the multivariate observations.

Deep learning is part of a broader family of machine learning methods, which is based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic programming. Dimensionally cursed phenomena occur in domains such as numerical analysis, sampling, combinatorics, machine learning, data mining and databases.

Dimensionality reduction, or dimension reduction, is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally close to its intrinsic dimension. Working in high-dimensional spaces can be undesirable for many reasons; raw data are often sparse as a consequence of the curse of dimensionality, and analyzing the data is usually computationally intractable (hard to control or deal with).

Related lectures (38)

Unsupervised Learning: Clustering & Dimensionality ReductionME-390: Foundations of artificial intelligence

Introduces unsupervised learning through clustering with K-means and dimensionality reduction using PCA, along with practical examples.

Neural Networks Recap: Activation FunctionsDH-406: Machine learning for DH

Covers the basics of neural networks, activation functions, training, image processing, CNNs, regularization, and dimensionality reduction methods.

Dimensionality Reduction: PCA & t-SNEDH-406: Machine learning for DH

Explores PCA and t-SNE for reducing dimensions and visualizing high-dimensional data effectively.

Document Analysis: Topic ModelingDH-406: Machine learning for DH

Explores document analysis, topic modeling, and generative models for data generation in machine learning.

Understanding AutoencodersCS-233(a): Introduction to machine learning (BA3)

Explores autoencoders, from linear mappings in PCA to nonlinear mappings, deep autoencoders, and their applications.