Automatic Skin Enhancement with Visible and Near-Infrared Image Fusion

Skin tones, portraits in particular, are of critical importance in photography and video, but a number of factors, such as pigmentation irregularities (e.g., moles, freckles), irritation, roughness, or wrinkles can reduce their appeal. Moreover, such “defects” are oftentimes enhanced by scene lighting conditions. Starting with the observations that melanin and hemoglobin, the key components of skin color, have little absorption in the near-infrared (NIR) part of the spectrum, and that the depth of light penetration in the epidermis is proportional to the incident light’s wavelength, we show that near-infrared images provide information that can be used to automatically smooth skin tones in a physically realistic manner. Specifically, we developed a prototype camera system that consists of capturing a pair of visible/near-infrared images and separating both of them into base and detail layers (akin to a low/high frequency decomposition) with the fast bilateral filter. Smooth and realistic output images are obtained by fusing the base layer of the visible image with the near-infrared detail layer. The proposed method delivers consistently good results across various skin types. The prototype system is currently in use at the Swiss Camera Museum in Vevey, Switzerland, where the visitors can take their pictures and e-mail themselves the results. In the process, we are collecting the users’ preference for either the “original” (visible) image or the “enhanced” (visible and NIR fused) image. The system has been deployed for three months. Preliminary statistics indicate that a large majority (79%) prefers the enhanced image.