Automatic Skin Enhancement with Visible and Near-Infrared Image Fusion

Sabine Süsstrunk, Daniel Tamburrino
2010
Conference paper

Abstract

Skin tones, portraits in particular, are of critical importance in photography and video, but a number of factors, such as pigmentation irregularities (e.g., moles, freckles), irritation, roughness, or wrinkles can reduce their appeal. Moreover, such “defects” are oftentimes enhanced by scene lighting conditions. Starting with the observations that melanin and hemoglobin, the key components of skin color, have little absorption in the near-infrared (NIR) part of the spectrum, and that the depth of light penetration in the epidermis is proportional to the incident light’s wavelength, we show that near-infrared images provide information that can be used to automatically smooth skin tones in a physically realistic manner. Specifically, we developed a prototype camera system that consists of capturing a pair of visible/near-infrared images and separating both of them into base and detail layers (akin to a low/high frequency decomposition) with the fast bilateral filter. Smooth and realistic output images are obtained by fusing the base layer of the visible image with the near-infrared detail layer. The proposed method delivers consistently good results across various skin types. The prototype system is currently in use at the Swiss Camera Museum in Vevey, Switzerland, where the visitors can take their pictures and e-mail themselves the results. In the process, we are collecting the users’ preference for either the “original” (visible) image or the “enhanced” (visible and NIR fused) image. The system has been deployed for three months. Preliminary statistics indicate that a large majority (79%) prefers the enhanced image.

Official source

https://infoscience.epfl.ch/record/153994?ln=en

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts

Related publications

Sabine Süsstrunk, Daniel Tamburrino
2010
Conference paper

Abstract

Official source

https://infoscience.epfl.ch/record/153994?ln=en

About this result

Related concepts (8)

Related concepts

Related publications (3)

Related publications

Skin tone images, portraits in particular, are of tremendous importance in digital photography, but a number of factors, such as pigmentation irregularities (e.g., moles, freckles), irritation, roughness, or wrinkles can reduce their appeal. Moreover, such “defects” are oftentimes enhanced by lighting conditions, e.g., when a flash is used. Starting with the observations that melanin and hemoglobin, the key components of skin colour, have little absorption in the near-infrared part of the spectrum, and that the depth of light penetration in the epidermis is proportional to the incident light’s wavelength, we propose that near-infrared images provide information that can be used to automatically smooth skin tones in a physically realistic manner. Specifically, we develop a framework that consists of capturing a pair of visible/near-infrared images and separating both of them into base and detail layers (akin to a low/high frequency decomposition) with the fast bilateral filter. We show that a smooth, realistic, output image can be obtained by fusing the base layer of the visible image with the near-infrared detail layer. This method not only outperforms equivalent decomposition in the wavelet domain, but the results also look more realistic than with a simple luminance transfer. Moreover, the proposed method delivers consistently good results across various skin types.

2009

Illuminant estimation and detection using near infrared

Sabine Süsstrunk

Digital camera sensors are sensitive to wavelengths ranging from the ultraviolet (200-400nm) to the near-infrared (700-100nm) bands. This range is, however, reduced because the aim of photographic cameras is to capture and reproduce the visible spectrum (400-700nm) only. Ultraviolet radiation is filtered out by the optical elements of the camera, while a specifically designed "hot-mirror" is placed in front of the sensor to prevent near-infrared contamination of the visible image. We propose that near-infrared data can actually prove remarkably useful in colour constancy, to estimate the incident illumination as well as providing to detect the location of different illuminants in a multiply lit scene. Looking at common illuminants spectral power distribution show that very strong differences exist between the near-infrared and visible bands, e.g., incandescent illumination peaks in the near-infrared while fluorescent sources are mostly confined to the visible band. We show that illuminants can be estimated by simply looking at the ratios of two images: a standard RGB image and a near-infrared only image. As the differences between illuminants are amplified in the near-infrared, this estimation proves to be more reliable than using only the visible band. Furthermore, in most multiple illumination situations, one of the light will be predominantly near-infrared emitting (e.g., flash, incandescent) while the other will be mostly visible emitting (e.g., fluorescent, skylight). Using near-infrared and RGB image ratios allow us to accurately pinpoint the location of diverse illuminant and recover a lighting map.

2009

Shadows, due to their prevalence in natural images, are a long studied phenomenon in digital photography and computer vision. Indeed, their presence can be a hindrance for a number of algorithms; accurate detection (and sometimes subsequent removal) of shadows in images is thus of paramount importance. In this paper, we present a method to detect shadows in a fast and accurate manner. To do so, we employ the inherent sensitivity of digital camera sensors to the near-infrared (NIR) part of the spectrum. We start by observing that commonly encountered light sources have very distinct spectra in the NIR, and propose that ratios of the colour channels (red, green and blue) to the NIR image gives valuable information about impinging illumination. In addition, we assume that shadows are contained in the darker parts of an image for both visible and NIR. This latter assumption is corroborated by the fact that a number of colorants are transparent to the NIR, thus making parts of the image that are dark in both the visible and NIR prime shadow candidates. These hypotheses allow for fast, accurate shadow detection in real, complex, scenes, including soft and occlusion shadows. We demonstrate that the process is reliable enough to be performed in-camera on still mosaicked images by simulating a modified colour filter array (CFA) that can simultaneously capture NIR and visible images. Finally, we show that our binary shadow maps can be the input of a matting algorithm to improve their precision in a fully automatic manner.

EPFL Tech Report 1655272010