Illuminant estimation and detection using near infrared

Digital camera sensors are sensitive to wavelengths ranging from the ultraviolet (200-400nm) to the near-infrared (700-100nm) bands. This range is, however, reduced because the aim of photographic cameras is to capture and reproduce the visible spectrum (400-700nm) only. Ultraviolet radiation is filtered out by the optical elements of the camera, while a specifically designed "hot-mirror" is placed in front of the sensor to prevent near-infrared contamination of the visible image. We propose that near-infrared data can actually prove remarkably useful in colour constancy, to estimate the incident illumination as well as providing to detect the location of different illuminants in a multiply lit scene. Looking at common illuminants spectral power distribution show that very strong differences exist between the near-infrared and visible bands, e.g., incandescent illumination peaks in the near-infrared while fluorescent sources are mostly confined to the visible band. We show that illuminants can be estimated by simply looking at the ratios of two images: a standard RGB image and a near-infrared only image. As the differences between illuminants are amplified in the near-infrared, this estimation proves to be more reliable than using only the visible band. Furthermore, in most multiple illumination situations, one of the light will be predominantly near-infrared emitting (e.g., flash, incandescent) while the other will be mostly visible emitting (e.g., fluorescent, skylight). Using near-infrared and RGB image ratios allow us to accurately pinpoint the location of diverse illuminant and recover a lighting map.

Mid-infrared surface plasmon resonance sensing applied to refractive index measurements of liquids

Sylvain Herminjard

There has been a very strong development of the sensors based on surface plasmon resonance during the last thirty years, mostly for biological and biomedical applications. If the first experiments in this field were carried out at the beginning of the 20t century, it would have been necessary to wait for the end of the sixties to see the first devices performing accurate measurements of the refractive indices of gas and liquid samples. Indeed, the surface plasmon resonance phenomenon allows to measure refractive index variations with a high sensitivity that is hardly obtained with other optical measurement techniques. The basic principle consists of exciting the free electrons of a metallic layer with light, which induces the formation of a surface wave called surface plasmon. This resonant phenomenon depends on the wavelength of the excitation light and the refractive indices of the metallic layer and the material located close to the metallic layer. A variation of the refractive index of the latter induces a variation of the resonance condition, which allows to realize optical sensors designed to measure determined samples. An appropriate calibration allows to set a relationship between the variation of the concentration of the sample and the measured refractive index value. Among the various parameters that describe the performances of the sensor, one of the most important is the sensitivity. It is possible to demonstrate that the excitation wavelength has an impact on the sensitivity of the system : the latter increases if the measured sample has an absorption peak at a wavelength close to the excitation wavelength. The main goal of this thesis is to explore this sensitivity enhancement technique based on the absorption properties of the probed medium. A proof-of-principle is experimentally demonstrated with an optical setup constituted of laser sources working in the visible range of the electromagnetic spectrum. However, many materials have absorption peaks located in the mid-infrared range due to their fundamental molecular vibrations. Therefore, the realization of a plasmonic sensor working in the mid-infrared could allow to measure these materials with an increased sensitivity. To the best of our knowledge, surface plasmon resonance based sensors have never been developed above the near-infrared. This thesis presents for the first time the design and realization of a plasmonic sensor working in the mid-infrared range able to measure refractive index variations of gas and liquids samples. Even if the optical sources, components and detectors designed for the mid-infrared are clearly different from the ones designed for the visible range, it has been possible to successfully realize two systems working in the mid-infrared range during this project. The first is able to perform gas measurements. As the excitation wavelength depends on the measured sample, it was necessary to select a laser source that had a wavelength located close to the absorption peak of the gas under measurement. For this purpose, we designed a plasmonic sensor working at a 4.4 µm wavelength able to measure carbon dioxide (CO2) absorbing at 4.3 µm. It has been demonstrated that the experimental sensitivity of this system is five times higher than the theoretical sensitivity obtained with a system working in the visible range at a 633 nm wavelength. The second has been conceived to measure liquid samples in order to measure variations of concentration of CO2 dissolved in water. If measurements of refractive index variations (without using the sensitivity enhancement technique) could be performed successfully with variations of concentration of alcohol in water, further work is necessary to get decisive results for CO2 dissolved in water. This project was carried out in collaboration with an industrial partner in order to take into account some industrial constraints during the development of the sensor. Therefore, particular care has been taken in order to design a system that is compact, mechanically robust, without any mechanical moving parts, cheap and easily reproducible. The technological development presented in this thesis finds many applications mainly in the chemical, biological and biomedical fields. Indeed, many proteins show absorption peaks in the mid-infrared. It could be possible to measure their concentration or the modification of their properties – like their composition or structure – with an increased sensitivity. The interest of surface plasmons is that the measurement is not a transmission-based technique, but rather a "reflection-based technique", as the light does not go through the gas or liquid sample. This measurement technique is also adapted for high volume samples that are present in industrial production processes like water or soft drink quality control.

EPFL2010

Automatic and accurate shadow detection from (potentially) a single image using near-infrared information

Sabine Süsstrunk

Shadows, due to their prevalence in natural images, are a long studied phenomenon in digital photography and computer vision. Indeed, their presence can be a hindrance for a number of algorithms; accurate detection (and sometimes subsequent removal) of shadows in images is thus of paramount importance. In this paper, we present a method to detect shadows in a fast and accurate manner. To do so, we employ the inherent sensitivity of digital camera sensors to the near-infrared (NIR) part of the spectrum. We start by observing that commonly encountered light sources have very distinct spectra in the NIR, and propose that ratios of the colour channels (red, green and blue) to the NIR image gives valuable information about impinging illumination. In addition, we assume that shadows are contained in the darker parts of an image for both visible and NIR. This latter assumption is corroborated by the fact that a number of colorants are transparent to the NIR, thus making parts of the image that are dark in both the visible and NIR prime shadow candidates. These hypotheses allow for fast, accurate shadow detection in real, complex, scenes, including soft and occlusion shadows. We demonstrate that the process is reliable enough to be performed in-camera on still mosaicked images by simulating a modified colour filter array (CFA) that can simultaneously capture NIR and visible images. Finally, we show that our binary shadow maps can be the input of a matting algorithm to improve their precision in a fully automatic manner.

EPFL Tech Report 1655272010

A high sensitivity, low noise and high spatial resolution multi-band infrared reflectography camera for the study of paintings and works on paper

Marie Eve Pascale Didier

Background: Infrared reflectography (IRR) remains an important method to visualize underdrawing and compositional changes in paintings. Older IRR camera systems are being replaced with near-infrared cameras consisting of room temperature infrared detector arrays made out of indium gallium arsenide (InGaAs) that operate over the spectral range of similar to 900 to 1700 nm. Two camera types are becoming prevalent. The first is staring array infrared cameras having 0.25-1 Megapixels where the camera or painting is moved to acquire tens of individual images that are later mosaicked together to create the infrared reflectogram. The second camera type is scanning back cameras in which a small InGaAs array (linear or area array) is mechanically scanned over a large image formed by the camera lens to create the reflectogram, typically 16 Megapixels. Both systems have advantages and disadvantages. The staring IR array cameras offer more flexible collection formats, provide live images, and allow for the use of spectral bandpass filters that can provide reflectograms with better contrast in some cases. They do require a mechanical system for moving the camera or the artwork and post-capture image mosaicking. Scanning back cameras eliminate or reduce the amount of mosaicking and movement of the camera, however the need to minimize light exposure to the artwork requires short integration times, and thus limits the use of spectral bandpass filters. In general, InGaAs cameras are not sensitive in the 1700 to similar to 2300 nm spectral region, which has been identified in prior studies as useful for examining paintings with copper green pigments or thick lead white paints. Prior studies using cameras with sensitivity from 1000 to 2500 nm have found in general the performance at wavelengths longer than 1700 nm degraded relative to the performance at shorter wavelengths. Thus, there is interest in a camera system having improved performance out to 2500 nm that can utilize spectral bandpass filters. Methods: Design requirements for such an improved IRR camera system were determined by first re-examining the optimal spectral window for detecting underdrawing. Thus the wavelength dependence of the clarity of carbon black underdrawings on a chalk ground covered by various paint swatches was measured from 750 to 2500 nm. Second, analysis of the loss of light transmission (1000-2500 nm) and the impact of thermal radiation (3000-5000 nm) on the performance of IR arrays sensitive to the 1000-5000 nm region was analyzed. From the results of these studies, a high sensitivity, near-infrared (1000-2450 nm) indium antimonide (InSb) staring array camera with a cold filter that blocked light >2450 nm was constructed with a color-corrected macro lens capable of holding interference filters. The camera system was characterized in three spectral bands (1100-1400, 1500-1800, and 2100-2400 nm) using test targets and art objects. Results: The experimental results of the contrast difference between the model underdrawing on the chalk ground showed the optimal spectral window for a given pigment varied over the range from 700 to 2300 nm. The contrast in all cases was found to be low from 2300 to 2600 nm and even lower from 2600 to 5000 nm. This is attributable in part to broad absorption by the drying oil paint binder. Performance testing of the IRR camera found high signal-to-noise was observed in all three spectral bands due to the optimized macro lens having high transmission from 1000 to 2600 nm, a 100% efficiency cold stop, and a cold filter that blocks light >2450 nm. The camera was found to have high light sensitivity, requiring only similar to 30 to 50 lx from incandescent lamps having a color temperature of similar to 2060 K. The images produced in each spectral band were sharp, and the modulation at Nyquist (1/2 the sampling frequency) was 20%. IRR images of Old Master paintings, works on paper, and a warped panel painting were collected to test the practical utility of the camera system. Conclusions: The IR camera system presented here was found to produce a variety of high-resolution image products. The ability to extend image collection over the 1700-2450 nm spectral range was found in some examples to provide improved visibility of underdrawing. The ability to register the resulting multi-band IRR images with the color image offers opportunities to produce new image products such as false-color images and principal component images. These image products were found to give new insights into the construction of works of art. A collection scheme based on a 'Z-stack' method was found to solve the problem of producing high-resolution IRR images of highly warped panel paintings.