Automatic and Accurate Shadow Detection using Near-Infrared Information

We present a method to automatically detect shadows in a fast and accurate manner by taking advantage of the inherent sensitivity of digital camera sensors to the near-infrared (NIR) part of the spectrum. Dark objects, which confound many shadow detection algorithms, often have much higher reflectance in the NIR. We can thus build an accurate shadow candidate map based on image pixels that are dark both in the visible and NIR representations. We further refine the shadow map by incorporating ratios of the visible to the NIR image, based on the observation that commonly encountered light sources have very distinct spectra in the NIR band. The results are validated on a new database, which contains visible/NIR images for a large variety of real-world shadow creating illuminant conditions, as well as manually labelled shadow ground truth. Both quantitative and qualitative evaluations show that our method outperforms current state-of-the-art shadow detection algorithms in terms of accuracy and computational efficiency.

Dominic Rüfenacht, Sabine Süsstrunk

We present a method to automatically detect shadows in a fast and accurate manner by employing the inherent sensitivity of digital camera sensors to the near-infrared (NIR) part of the spectrum. Shadows are generally found in the dark parts of an image, be it color or NIR. We consider these dark pixels as shadow candidates. By observing that commonly encountered light sources have very distinct spectra in the NIR, we propose that the ratios of the color channels (red, green and blue) to the NIR give valuable information about impinging illumination, which we employ to assess the shadow candidate pixels.

2012

Mid-infrared surface plasmon resonance sensing applied to refractive index measurements of liquids

Sylvain Herminjard

There has been a very strong development of the sensors based on surface plasmon resonance during the last thirty years, mostly for biological and biomedical applications. If the first experiments in this field were carried out at the beginning of the 20t century, it would have been necessary to wait for the end of the sixties to see the first devices performing accurate measurements of the refractive indices of gas and liquid samples. Indeed, the surface plasmon resonance phenomenon allows to measure refractive index variations with a high sensitivity that is hardly obtained with other optical measurement techniques. The basic principle consists of exciting the free electrons of a metallic layer with light, which induces the formation of a surface wave called surface plasmon. This resonant phenomenon depends on the wavelength of the excitation light and the refractive indices of the metallic layer and the material located close to the metallic layer. A variation of the refractive index of the latter induces a variation of the resonance condition, which allows to realize optical sensors designed to measure determined samples. An appropriate calibration allows to set a relationship between the variation of the concentration of the sample and the measured refractive index value. Among the various parameters that describe the performances of the sensor, one of the most important is the sensitivity. It is possible to demonstrate that the excitation wavelength has an impact on the sensitivity of the system : the latter increases if the measured sample has an absorption peak at a wavelength close to the excitation wavelength. The main goal of this thesis is to explore this sensitivity enhancement technique based on the absorption properties of the probed medium. A proof-of-principle is experimentally demonstrated with an optical setup constituted of laser sources working in the visible range of the electromagnetic spectrum. However, many materials have absorption peaks located in the mid-infrared range due to their fundamental molecular vibrations. Therefore, the realization of a plasmonic sensor working in the mid-infrared could allow to measure these materials with an increased sensitivity. To the best of our knowledge, surface plasmon resonance based sensors have never been developed above the near-infrared. This thesis presents for the first time the design and realization of a plasmonic sensor working in the mid-infrared range able to measure refractive index variations of gas and liquids samples. Even if the optical sources, components and detectors designed for the mid-infrared are clearly different from the ones designed for the visible range, it has been possible to successfully realize two systems working in the mid-infrared range during this project. The first is able to perform gas measurements. As the excitation wavelength depends on the measured sample, it was necessary to select a laser source that had a wavelength located close to the absorption peak of the gas under measurement. For this purpose, we designed a plasmonic sensor working at a 4.4 µm wavelength able to measure carbon dioxide (CO2) absorbing at 4.3 µm. It has been demonstrated that the experimental sensitivity of this system is five times higher than the theoretical sensitivity obtained with a system working in the visible range at a 633 nm wavelength. The second has been conceived to measure liquid samples in order to measure variations of concentration of CO2 dissolved in water. If measurements of refractive index variations (without using the sensitivity enhancement technique) could be performed successfully with variations of concentration of alcohol in water, further work is necessary to get decisive results for CO2 dissolved in water. This project was carried out in collaboration with an industrial partner in order to take into account some industrial constraints during the development of the sensor. Therefore, particular care has been taken in order to design a system that is compact, mechanically robust, without any mechanical moving parts, cheap and easily reproducible. The technological development presented in this thesis finds many applications mainly in the chemical, biological and biomedical fields. Indeed, many proteins show absorption peaks in the mid-infrared. It could be possible to measure their concentration or the modification of their properties – like their composition or structure – with an increased sensitivity. The interest of surface plasmons is that the measurement is not a transmission-based technique, but rather a "reflection-based technique", as the light does not go through the gas or liquid sample. This measurement technique is also adapted for high volume samples that are present in industrial production processes like water or soft drink quality control.

EPFL2010

Automatic and accurate shadow detection from (potentially) a single image using near-infrared information

Sabine Süsstrunk

Shadows, due to their prevalence in natural images, are a long studied phenomenon in digital photography and computer vision. Indeed, their presence can be a hindrance for a number of algorithms; accurate detection (and sometimes subsequent removal) of shadows in images is thus of paramount importance. In this paper, we present a method to detect shadows in a fast and accurate manner. To do so, we employ the inherent sensitivity of digital camera sensors to the near-infrared (NIR) part of the spectrum. We start by observing that commonly encountered light sources have very distinct spectra in the NIR, and propose that ratios of the colour channels (red, green and blue) to the NIR image gives valuable information about impinging illumination. In addition, we assume that shadows are contained in the darker parts of an image for both visible and NIR. This latter assumption is corroborated by the fact that a number of colorants are transparent to the NIR, thus making parts of the image that are dark in both the visible and NIR prime shadow candidates. These hypotheses allow for fast, accurate shadow detection in real, complex, scenes, including soft and occlusion shadows. We demonstrate that the process is reliable enough to be performed in-camera on still mosaicked images by simulating a modified colour filter array (CFA) that can simultaneously capture NIR and visible images. Finally, we show that our binary shadow maps can be the input of a matting algorithm to improve their precision in a fully automatic manner.

EPFL Tech Report 1655272010

Mid-infrared surface plasmon resonance sensing applied to refractive index measurements of liquids

Sylvain Herminjard

EPFL2010