Keyword-based Image Color Re-rendering with Semantic Segmentation

Keyword-based image color re-rendering is a convenient way to enhance the color of images. Most methods only focus on the color characteristics of the image while ignoring the semantic meaning of different regions. We propose to incorporate semantic information into the color re-rendering pipeline through semantic segmentation. Using semantic segmentation masks, we first generate more accurate correlations between keywords and color characteristics than the state-ofthe- art approach. Such correlations are then adopted for rerendering the color of the input image, where the segmentation masks are used to indicate the regions for color rerendering. Qualitative comparisons show that our method generates visually better results than the state-of-the-art approach. We further validate with a psychophysical experiment, where the participants prefer the results of our method.

Compact representations for static and dynamic texture synthesis

Roberto Costantini

In nature, textures can be found everywhere. Texture, together with color and shape, represents the fundamental characteristics of objects. Texture conveys an idea of repetition of a certain structure, which is not limited only to the visual domain. We can refer to the texture of a sound, a fabric, or a certain wine. In image processing, a static texture is defined as an image showing spatial stationarity, while a dynamic texture is a sequence of images showing temporal stationarity. Texture synthesis is the process of producing artificial textures starting from a given texture sample. In this thesis, we consider texture synthesis starting from compact representations. For static, this reduces to define the smallest portion of a texture that, when used for synthesis, produces a synthetic texture indistinguishable from the original image. We designed a psychophysical experiment to quantify this size for a collection of textures and two synthesis algorithms: the parametric algorithm by Portilla and Simoncelli and the procedural algorithm by Nealen and Alexa. The results of the experiment show that for pre-attentive vision condition the size of the compact texture does not depend on the algorithm used for synthesis but is a characteristic of the texture. We evaluated if an objective function, based on spatial correlation between pixels, is able to predict the texture size necessary for synthesis. We found that a statistical measure based on random walks correlated well with the results of the subjective experiment. In the case of dynamic textures, the term "compact" applies to the model size that is used for synthesis. We propose a dynamic texture analysis that is able to obtain a more compact model starting from the linear model of Soatto and Doretto. Current methods perform a dimension reduction of the data by applying the SVD to the video frames unfolded into column vectors. This permits only to exploit the temporal correlation. We avoid the unfolding operations and decompose the signal directly using a multidimensional decomposition known as Higher-Order SVD (HOSVD). Chromatic components are exploited more efficiently by combining the HOSVD decomposition with the YCbCr color encoding for the input data. Tests show that the combined model has far fewer parameters than models derived with other algorithms, for the same visual quality and approximately the same computational synthesis cost. The HOSVD-based model was used in two applications for portable devices where memory and computational power are limited. We created a video effect for webcams and smartphones where a dynamic texture is synthesized and blended to the video in real time.

EPFL2007

Saliency Detection using Maximum Symmetric Surround

Radhakrishna Achanta, Sabine Süsstrunk

Detection of visually salient image regions is useful for applications like object segmentation, adaptive compression, and object recognition. Recently, full-resolution salient maps that retain well-defined boundaries have attracted attention. In these maps, boundaries are preserved by retaining substantially more frequency content from the original image than older techniques. However, if the salient regions comprise more than half the pixels of the image, or if the background is complex, the background gets highlighted instead of the salient object. In this paper, we introduce a method for salient region detection that retains the advantages of such saliency maps while overcoming their shortcomings. Our method exploits features of color and luminance, is simple to implement and is computationally efficient. We compare our algorithm to six state-of-the-art salient region detection methods using publicly available ground truth. Our method outperforms the six algorithms by achieving both higher precision and better recall. We also show application of our saliency maps in an automatic salient object segmentation scheme using graph-cuts.

Ieee Service Center, 445 Hoes Lane, Po Box 1331, Piscataway, Nj 08855-1331 Usa2010

Material-Based Object Segmentation Using Near-Infrared Information

Neda Salamati, Sabine Süsstrunk

We present a framework to incorporate near-infrared (NIR) information into algorithms to better segment objects by isolating material boundaries from color and shadow edges. Most segmentation algorithms assign individual regions to parts of the object that are colorized differently. Similarly, the presence of shadows and thus large changes in image intensities across objects can also result in mis-segmentation. We first form an intrinsic image from the R, G, B, and NIR channels based on a 4-sensor camera calibration model that is invariant to shadows. The regions obtained by the segmentation algorithms are thus only due to color and material changes and are independent of the illumination. Additionally, we also segment the NIR channel only. Near-infrared (NIR) image intensities are largely dependent on the chemistry of the material and have no general correlation with visible color information. Consequently, the NIR segmentation only highlights material and lighting changes. The union of both segmentations obtained from the intrinsic and NIR images results in image partitions that are only based on material changes and not on color or shadows. Experiments show that the proposed method provides good object-based segmentation results on diverse images.

2010

Compact representations for static and dynamic texture synthesis

Roberto Costantini

EPFL2007