Latent Semantic Indexing: Concepts and Applications

This lecture covers Latent Semantic Indexing (LSI), a technique to map documents and queries into a lower-dimensional space of higher-level concepts. It explains how LSI uses concepts for retrieval, the concept of dimensionality reduction, and the process of similarity computation in the concept space. The lecture also delves into Singular Value Decomposition (SVD) and its application in constructing the concept space. Additionally, it discusses the implementation of LSI in Python and alternative techniques like Probabilistic Latent Semantic Analysis and Latent Dirichlet Allocation. The lecture concludes with the practical use of LSI in unsupervised learning, document organization, retrieval, and classification.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related lectures (155)

Related concepts (65)

This course introduces the foundations of information retrieval, data mining and knowledge bases, which constitute the foundations of today's Web-based distributed information systems.

Week

A week is a unit of time equal to seven days. It is the standard time period used for short cycles of days in most parts of the world. The days are often used to indicate common work days and rest days, as well as days of worship. Weeks are often mapped against yearly calendars, but are typically not the basis for them, as weeks are not based on astronomy. The modern seven-day week can be traced back to the Babylonians, who used it within their calendar.

Latent semantic analysis

Latent semantic analysis (LSA) is a technique in natural language processing, in particular distributional semantics, of analyzing relationships between a set of documents and the terms they contain by producing a set of concepts related to the documents and terms. LSA assumes that words that are close in meaning will occur in similar pieces of text (the distributional hypothesis).

Determination of the day of the week

The determination of the day of the week for any date may be performed with a variety of algorithms. In addition, perpetual calendars require no calculation by the user, and are essentially lookup tables. A typical application is to calculate the day of the week on which someone was born or a specific event occurred. In numerical calculation, the days of the week are represented as weekday numbers. If Monday is the first day of the week, the days may be coded 1 to 7, for Monday through Sunday, as is practiced in ISO 8601.

Names of the days of the week

In many languages, the names given to the seven days of the week are derived from the names of the classical planets in Hellenistic astronomy, which were in turn named after contemporary deities, a system introduced by the Sumerians and later adopted by the Babylonians from whom the Roman Empire adopted the system during Late Antiquity. In some other languages, the days are named after corresponding deities of the regional culture, beginning either with Sunday or with Monday.

Probabilistic latent semantic analysis

Probabilistic latent semantic analysis (PLSA), also known as probabilistic latent semantic indexing (PLSI, especially in information retrieval circles) is a statistical technique for the analysis of two-mode and co-occurrence data. In effect, one can derive a low-dimensional representation of the observed variables in terms of their affinity to certain hidden variables, just as in latent semantic analysis, from which PLSA evolved.

Latent semantic indexing: inverted files CS-423: Distributed information systems

Explores term-offset indices in inverted files and relevance feedback solutions.

Information Retrieval Indexing: Latent Semantic Indexing CS-423: Distributed information systems

Explores Latent Semantic Indexing in Information Retrieval, discussing algorithms, challenges in Vector Space Retrieval, and concept-focused retrieval methods.

Embedding Models: Concepts and Retrieval CS-423: Distributed information systems

Covers embedding models for document retrieval, latent semantic indexing, SVD, and topic models.

Vector Space Semantics (and Information Retrieval)CS-431: Introduction to natural language processing

Explores the Vector Space model, Bag of Words, tf-idf, cosine similarity, Okapi BM25, and Precision and Recall in Information Retrieval.

Information Retrieval Basics CS-423: Distributed information systems

Introduces the basics of information retrieval, covering document representation, query expansion, and TF-IDF for document ranking.