Nonlinear dimensionality reduction

Are you an EPFL student looking for a semester project?

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Nonlinear dimensionality reduction, also known as manifold learning, refers to various related techniques that aim to project high-dimensional data onto lower-dimensional latent manifolds, with the goal of either visualizing the data in the low-dimensional space, or learning the mapping (either from the high-dimensional space to the low-dimensional embedding or vice versa) itself. The techniques described below can be understood as generalizations of linear decomposition methods used for dimensionality reduction, such as singular value decomposition and principal component analysis. Consider a dataset represented as a matrix (or a database table), such that each row represents a set of attributes (or features or dimensions) that describe a particular instance of something. If the number of attributes is large, then the space of unique possible rows is exponentially large. Thus, the larger the dimensionality, the more difficult it becomes to sample the space. This causes many problems. Algorithms that operate on high-dimensional data tend to have a very high time complexity. Many machine learning algorithms, for example, struggle with high-dimensional data. Reducing data into fewer dimensions often makes analysis algorithms more efficient, and can help machine learning algorithms make more accurate predictions. Humans often have difficulty comprehending data in high dimensions. Thus, reducing data to a small number of dimensions is useful for visualization purposes. The reduced-dimensional representations of data are often referred to as "intrinsic variables". This description implies that these are the values from which the data was produced. For example, consider a dataset that contains images of a letter 'A', which has been scaled and rotated by varying amounts. Each image has 32×32 pixels. Each image can be represented as a vector of 1024 pixel values. Each row is a sample on a two-dimensional manifold in 1024-dimensional space (a Hamming space). The intrinsic dimensionality is two, because two variables (rotation and scale) were varied in order to produce the data.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (17)

Related people (17)

Related units (2)

Related concepts (10)

Related courses (54)

Related lectures (391)

CS-233(a): Introduction to machine learning (BA3)

Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning will be introduced, analy

MGT-502: Data science and machine learning

This class provides a hands-on introduction to data science and machine learning topics, exploring areas such as regression, classification, clustering, dimensionality reduction, text analytics, and n

MATH-513: Metric embeddings

The course aims to introduce the basic concepts and results on metric embeddings, or more precisely on approximate embeddings. This area has been under rapid development since the 90's and it has stro

CauseOccam: Learning Interpretable Abstract Representations in Reinforcement Learning Environments via Model Sparsity

Sergei Volodin

"I choose this restaurant because they have vegan sandwiches" could be a typical explanation we would expect from a human. However, current Reinforcement Learning (RL) techniques are not able to provi

2021

Inverse Modelling and Predictive Inference in Continuum Mechanics: a Data-Driven Approach

Claire Marianne Charlotte Capelo

The explosive growth of machine learning in the age of data has led to a new probabilistic and data-driven approach to solving very different types of problems. In this paper we study the feasibility

2020

Fourier dimensionality reduction for fast radio transients

Jean-Philippe Thiran, Yves Wiaux, Vijay Kartik, Ming Jiang

In the context of next-generation radio interferometers, we are facing a big challenge of how to economically process data. The classical dimensionality reduction technique, averaging visibilities on

2019

Dimensionality reduction, or dimension reduction, is the transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data, ideally close to its intrinsic dimension. Working in high-dimensional spaces can be undesirable for many reasons; raw data are often sparse as a consequence of the curse of dimensionality, and analyzing the data is usually computationally intractable (hard to control or deal with).

A self-organizing map (SOM) or self-organizing feature map (SOFM) is an unsupervised machine learning technique used to produce a low-dimensional (typically two-dimensional) representation of a higher dimensional data set while preserving the topological structure of the data. For example, a data set with variables measured in observations could be represented as clusters of observations with similar values for the variables.

Understanding Autoencoders

Explores autoencoders, from linear mappings in PCA to nonlinear mappings, deep autoencoders, and their applications.

Dimensionality Reduction: PCA & t-SNE

Explores PCA and t-SNE for reducing dimensions and visualizing high-dimensional data effectively.

Johnson-Lindenstrauss Theorem

Covers the Johnson-Lindenstrauss Theorem, which embeds high-dimensional points into lower-dimensional space while preserving distances.