Memory colour segmentation and classification using class-specific eigenregions

Francisco Estrada, Sabine Süsstrunk
2009
Journal paper

Abstract

Memory colours refer to the colour of specific image regions that have the essential attribute of being perceived in a consistent manner by human observers. In colour correction-or rendering-tasks, this consistency implies that they have to be faithfully reproduced; their importance, in that respect, is greater than for other regions in an image. There are various schemes and attributes to detect memory colours, but the preferred method remains to segment the images into meaningful regions, a task for which many algorithms exist. Memory colour regions are not, however, similar in their attributes. Significant variations in shape, size, and texture exist. As such, it is unclear whether a single segmentation algorithm is the most adapted for all of these classes. Using a large database of real-world images, we calculate class-specific geometrical features, eigenregions, that can be used to evaluate how well an algorithm is adapted to segment a given class and give a measure of localisation of memory colours. We also compare the performance of our class-specific eigenregions to general ones in the task of memory colour region classification and observe that they provide a noticeable improvement in classification rates.

Official source

https://infoscience.epfl.ch/record/140519?ln=en

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts

Related publications

Segmenting memory colours

Francisco Estrada, Sabine Süsstrunk

Memory colours refer to the colour of specific image classes that have the essential attribute of being perceived in a consistent manner by human observers. In colour correction or rendering tasks, this consistency implies that they have to be faithfully reproduced; their importance, in that respect, is greater than other regions in an image. Before these regions can be properly addressed, one must in general detect them. There are various schemes and attributes to do so, but the preferred method remains to segment the images into meaningful regions, a task for which many algorithms exist. Memory colours’ regions are not, however, similar in their attributes. Significant variations in shape, size, and texture do exist. As such, it is unclear whether a single algorithm is the most adapted for all of these classes. In this work, we concern ourselves with three memory colours: blue sky, green vegetation, and skin tones. Using a large database of real-world images, we (randomly) select and manually segment 900 images that contain one of the three memory colours. The same images are then automatically segmented with four classical algorithms. Using class-specific eigenregions, we are able to provide insights into the underlying structures of the considered classes and class-specific features that can be used to improve classification’s accuracy. Finally, we propose a distance measure that effectively results in determining how well is an algorithm is adapted to segment a given class.

2008

Finding relations between image semantics and image characteristics is a problem of long standing in computer vision and related fields. Despite persistent efforts and significant advances in the field, today’s computers are still strikingly unable to achieve the same complex understanding of semantic image content as human users do with ease. This is a problem when large sets of images have to be interpreted or somehow processed by algorithms. This problem becomes increasingly urgent with the rapid proliferation of digital image content due to the massive spreading of digital imaging devices such as smartphone cameras. This thesis develops a statistical framework to relate image keywords to image characteristics and is based on a large database of annotated images. The design of the framework respects two equally important properties. First, the output of the framework, i.e. a relatedness measure, is compact and easy-to-use for subsequent applications. We achieve this by using a simple, yet effective significance test. It measures a given keyword’s impact on a given image characteristic, which results in a significance value that serves as input for successive applications. Second, the framework is of very low complexity in order to scale well to large datasets. The test can be implemented very efficiently so that the statistical framework easily scales to millions of images and thousands of keywords The first application we present is semantic image enhancement. The enhancement framework takes two independent inputs, which are an image and a keyword, i.e. a semantic concept. The algorithm then re-renders the image to match the semantic concept. We implement this framework for four different tasks: tone-mapping, color enhancement, color transfer and depth-of-field adaptation. Unlike conventional image enhancement algorithms, our proposed approach is able to re-render a single input image for different semantic concepts, producing different image versions at the output to reflect the image context. We evaluate the proposed semantic image enhancement with two psychophysical experiments. The first experiment comprises almost 30’000 image comparisons of the original and the enhanced images. Due to the large scale, we crowdsourced the experiment on Amazon Mechanical Turk. The majority of the enhanced images was proven to be significantly better than the original images. The second experiment contains images that were enhanced for two different keywords. We compare our proposed algorithm against histogram equalization, Photoshop auto-contrast and the original. Our proposed method outperforms the others by a factor of at least 2.5. The second application is color naming, which aims at relating color values to color names and vice versa. Whereas conventional color naming depends on psychophysical experiments, we are able to solve this task fully automatically using the significance values. We first demonstrate the usefulness of our approach with an example of 50 color names and then extend it to the estimation of memory colors and color values for arbitrary semantic expressions. In a second study, we use a list of over 900 English color names and translate it to 9 other European and Asian languages. We estimate color values for these over 9000 color names and analyze the results from a language and color science point of view. Overall, we present a statistical framework that relates image keywords to image characteristics and apply it to two common imaging applications that benefit from a semantic understanding of the images. Further we outline the applicability of the framework to other applications and areas.

EPFL2013

Material-Based Object Segmentation Using Near-Infrared Information

Neda Salamati, Sabine Süsstrunk

We present a framework to incorporate near-infrared (NIR) information into algorithms to better segment objects by isolating material boundaries from color and shadow edges. Most segmentation algorithms assign individual regions to parts of the object that are colorized differently. Similarly, the presence of shadows and thus large changes in image intensities across objects can also result in mis-segmentation. We first form an intrinsic image from the R, G, B, and NIR channels based on a 4-sensor camera calibration model that is invariant to shadows. The regions obtained by the segmentation algorithms are thus only due to color and material changes and are independent of the illumination. Additionally, we also segment the NIR channel only. Near-infrared (NIR) image intensities are largely dependent on the chemistry of the material and have no general correlation with visible color information. Consequently, the NIR segmentation only highlights material and lighting changes. The union of both segmentations obtained from the intrinsic and NIR images results in image partitions that are only based on material changes and not on color or shadows. Experiments show that the proposed method provides good object-based segmentation results on diverse images.

2010

EPFL2013