Enhancing the Visible with the Invisible: Exploiting Near-Infrared to Advance Computational Photography and Computer Vision

Silicon-based digital camera sensors exhibit significant sensitivity beyond the visible spectrum (400-700nm). They are able to capture wavelengths up to 1100 nm, i.e., they are sensitive to near-infrared (NIR) radiation. This additional information is conventionally treated as noise and is absorbed by a NIR-blocking filter affixed to the sensor. We show that retaining instead of removing NIR information can significantly improve certain computational photography and computer vision tasks. Indeed, intrinsic properties of the NIR wavelength band guarantee that images can be sharper, less affected by man-made colorants, and more resilient to changing light conditions. The benefits of using NIR images in conjunction with standard color images in applications such as haze removal, skin smoothing, single and multiple illuminant detection, shadow detection, and material classification is discussed.