Computational photography

Computational photography refers to digital image capture and processing techniques that use digital computation instead of optical processes. Computational photography can improve the capabilities of a camera, or introduce features that were not possible at all with film based photography, or reduce the cost or size of camera elements. Examples of computational photography include in-camera computation of digital panoramas, high-dynamic-range images, and light field cameras. Light field cameras use novel optical elements to capture three dimensional scene information which can then be used to produce 3D images, enhanced depth-of-field, and selective de-focusing (or "post focus"). Enhanced depth-of-field reduces the need for mechanical focusing systems. All of these features use computational imaging techniques. The definition of computational photography has evolved to cover a number of subject areas in computer graphics, computer vision, and applied optics. These areas are given

Incorporating Near-Infrared into Scene Understanding

Neda Salamati

Recent progress in computational photography has shown that we can acquire near-infrared (NIR) information in addition to the normal visible (RGB) information with only slight modification to the standard digital camera. In this thesis, we study if this extra channel can improve one of the more difficult computer vision tasks: Scene understanding. Due to the proximity of NIR to visible radiation, NIR images share many properties with visible images. However, as a result of the material dependency of reflection in the NIR part of the spectrum, such images reveal different characteristics of the scene. In this work we study how to effectively exploit these differences to improve scene recognition and semantic segmentation performance. An initial psycho-physical test that we carried out gave promising evidence that humans understand the content of a scene more effectively when presented with the NIR image as opposed to the visible image. Motivated by this, we first formulate a novel framework that incorporates NIR information into a low-level segmentation algorithm to better detect the material boundaries of an object. This goal is achieved by first forming a illumination-invariant representation, i.e., the intrinsic image, and then by employing the material dependent properties of NIR images. Secondly, by leveraging on state-of-the-art segmentation frameworks and a novel manually segmented image database, we study how to best incorporate the specific characteristics of the NIR response into high-level semantic segmentation tasks. We show through extensive experimentation that introducing NIR information significantly improves the performance of automatic labeling for certain object classes, like fabrics and water, whose response in the NIR domain is particularly discriminant. We then thoroughly discuss the results with respect to both physical properties of the NIR response and the characteristics of the segmentation framework.

EPFL2013

Enhancing the Visible with the Invisble: RGB plus Near-Infrared

Nathalie Barbuscia, Yue Lu, Neda Salamati, Lex Schaul, Sabine Süsstrunk, Daniel Tamburrino

Silicon-based digital camera sensors exhibit significant sensitivity beyond the visible spectrum (400-700nm). They are able to capture wavelengths up to 1100 nm, i.e., they are sensitive to near-infrared (NIR) radiation. This additional information is conventionally treated as noise and is absorbed by a NIR-blocking filter affixed to the sensor. Retaining instead of removing NIR information can significantly improve certain computational photography and computer vision tasks. Indeed, intrinsic properties of the NIR wavelength band guarantee that images can be sharper, less affected by man-made colorants, and more resilient to changing light conditions. Exploiting the inherent correlation and de-correlation of the different waveband images can lead to improved color rendering and object recognition.

2010

Incorporating Near-Infrared into Scene Understanding

Neda Salamati

EPFL2013

Full-field surface and interior deformation measurement of sandwich core material using digital speckle photography

Incorporating Near-Infrared into Scene Understanding

Enhancing the Visible with the Invisble: RGB plus Near-Infrared

Incorporating Near-Infrared into Scene Understanding

Enhancing the Visible with the Invisble: RGB plus Near-Infrared

Full-field surface and interior deformation measurement of sandwich core material using digital speckle photography