Contextual image classification

Contextual image classification, a topic of pattern recognition in computer vision, is an approach of classification based on contextual information in images. "Contextual" means this approach is focusing on the relationship of the nearby pixels, which is also called neighbourhood. The goal of this approach is to classify the images by using the contextual information. Introduction Similar as processing language, a single word may have multiple meanings unless the context is provided, and the patterns within the sentences are the only informative segments we care about. For images, the principle is same. Find out the patterns and associate proper meanings to them. As the image illustrated below, if only a small portion of the image is shown, it is very difficult to tell what the image is about. Even try another portion of the image, it is still difficult to classify the image. However, if we increase the contextual of the image, then it makes more sense to recognize.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (50)

Related publications

Related people (12)

Related people

Related units (4)

Related units

Related concepts

Related courses (5)

Related courses

Related lectures (13)

Related lectures

QAIR: Practical Query-efficient Black-Box Attacks for Image Retrieval

Yuan He, Jinfeng Li, Hang Su, Shaokai Ye

We study the query-based attack against image retrieval to evaluate its robustness against adversarial examples under the black-box setting, where the adversary only has query access to the top-1 ranked unlabeled images from the database. Compared with query attacks in image classification, which produce adversaries according to the returned labels or confidence score, the challenge becomes even more prominent due to the difficulty in quantifying the attack effectiveness on the partial retrieved list. In this paper, we make the first attempt in Query-based Attack against Image Retrieval (QAIR), to completely subvert the top-1 retrieval results. Specifically, a new relevance-based loss is designed to quantify the attack effects by measuring the set similarity on the top-1 retrieval results before and after attacks and guide the gradient optimization. To further boost the attack efficiency, a recursive model stealing method is proposed to acquire transferable priors on the target model and generate the prior-guided gradients. Comprehensive experiments show that the proposed attack achieves a high attack success rate with few queries against the image retrieval systems under the black-box setting. The attack evaluations on the real-world visual search engine show that it successfully deceives a commercial system such as Bing Visual Search with 98% attack success rate by only 33 queries on average.

IEEE COMPUTER SOC2021

A Geometry-Inspired Decision-Based Attack

Pascal Frossard, Yujia Liu, Seyed Mohsen Moosavi Dezfooli

Deep neural networks have recently achieved tremendous success in image classification. Recent studies have however shown that they are easily misled into incorrect classification decisions by adversarial examples. Adversaries can even craft attacks by querying the model in black-box settings, where no information about the model is released except its final decision. Such decision-based attacks usually require lots of queries, while real-world image recognition systems might actually restrict the number of queries. In this paper, we propose qFool, a novel decision-based attack algorithm that can generate adversarial examples using a small number of queries. The qFool method can drastically reduce the number of queries compared to previous decision-based attacks while reaching the same quality of adversarial examples. We also enhance our method by constraining adversarial perturbations in low-frequency subspace, which can make qFool even more computationally efficient. Altogether, we manage to fool commercial image recognition systems with a small number of queries, which demonstrates the actual effectiveness of our new algorithm in practice.

IEEE COMPUTER SOC2019

A geometry-inspired decision-based attack

Pascal Frossard, Yujia Liu, Seyed Mohsen Moosavi Dezfooli

Deep neural networks have recently achieved tremen-dous success in image classification. Recent studies havehowever shown that they are easily misled into incorrectclassification decisions by adversarial examples. Adver-saries can even craft attacks by querying the model in black-box settings, where no information about the model is re-leased except its final decision. Such decision-based at-tacks usually require lots of queries, while real-world imagerecognition systems might actually restrict the number ofqueries. In this paper, we propose qFool, a novel decision-based attack algorithm that can generate adversarial exam-ples using a small number of queries. The qFool method candrastically reduce the number of queries compared to pre-vious decision-based attacks while reaching the same qual-ity of adversarial examples. We also enhance our methodby constraining adversarial perturbations in low-frequencysubspace, which can make qFool even more computation-ally efficient. Altogether, we manage to fool commercialimage recognition systems with a small number of queries,which demonstrates the actual effectiveness of our new al-gorithm in practice.

2019

ENV-540: Image processing for Earth observation

This course covers optical remote sensing from satellites and airborne platforms. The different systems are presented. The students will acquire skills in image processing and machine/deep learning to extract end-products, such as land cover or risk maps, from the images.

CS-503: Visual intelligence : machines and minds

The course will discuss classic material as well as recent advances in computer vision and machine learning relevant to processing visual data with a primary focus on embodied intelligence and vision for active agents.

EE-556: Mathematics of data: from theory to computation

This course provides an overview of key advances in continuous optimization and statistical analysis for machine learning. We review recent learning formulations and models as well as their guarantees, describe scalable solution techniques and algorithms, and illustrate the trade-offs involved.