Gradient-Based Correction of Chromatic Aberration in the Joint Acquisition of Color and Near-Infrared Images

Yue Lu, Zahra Sadeghipoor Kermani, Sabine Süsstrunk
Spie-Int Soc Optical Engineering, 2015
Conference paper

Abstract

Chromatic aberration distortions, such as wavelength-dependent blur caused by imperfections in photographic lenses, have long been studied in color imaging. The problem becomes more challenging to solve in the case of color and near-infrared joint acquisition, as a wider range of wavelengths is captured. In this paper, we assume that the color image is in focus, hence, the NIR image captured with the same focus settings is blurred. We propose an algorithm that estimates the blur kernel and deblurs the NIR image, using the in-focus color image as a guide in both steps. In the deblurring step, we retrieve the lost details of the NIR image by using the sharp edges of the color image. The main diculty in this task is caused by the fact that the edges of color and NIR images are not always correlated. To handle this issue, the algorithm analyzes the dierences between the gradients of NIR and color channels. Simulation results verify the eectiveness of our algorithm, both in estimating the blur kernel and deblurring the NIR image, without producing ringing artifacts inherent to the results of other deblurring methods.

Official source

https://infoscience.epfl.ch/record/204519?ln=en

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts

Related publications

In the past few years, fusing NIR and color images has been explored in general computational photography and computer vision tasks, where traditionally only color images are used. The additional information provided by the differences of light and scene reflections in the visible and NIR bands of the electromagnetic spectrum is used in several applications such as image denoising, image dehazing, shadow detection and removal, and high dynamic-range imaging. In this thesis, we study a system that simultaneously captures color and NIR images on a single silicon sensor. Such a camera could be manufactured with minor changes in the hardware of consumer color cameras and it could be integrated inside small devices such as cell phones. We address two main challenges in color and NIR acquisition. First, we study the spatial and spectral sampling of the scene, which is inevitable in single-sensor acquisition of multiple spectral channels. We then focus on chromatic aberration distortions. Similarly to color imaging, we use a color filter array (CFA) to sample the scene in visible and NIR bands. We address two main challenges regarding the CFA: (1) designing the CFA, and (2) developing a demosaicing algorithm to reconstruct full-resolution images from the subsampled measurements. We consider a general CFA with filters that transmit different mixtures of color and NIR channels. We develop a framework that, by exploiting the spatial and spectral correlations of color and NIR images, computes the transmittance of each filter and the demosaicing matrix. Our optimized CFA and demosaicing outperform other solutions developed for single-sensor color and NIR acquisition. We also investigate a CFA that is formed by one blue, one green, one red, and one NIR-pass filter. We call it the RGBN CFA and assume that, similarly to color cameras, it uses dye filters that do not have sharp cut-offs. Hence, color and NIR radiations leak into NIR and color filters, respectively. We devise an algorithm that reconstructs full-resolution images from mixed and subsampled sensor measurements. The RGBN CFA and our reconstruction algorithm perform as well as or even better than other single-sensor acquisition techniques that use more complicated hardware components. The problem of chromatic aberration is caused by deficiencies of optical elements. A simple lens converges light rays with different wavelengths at different distances from the lens. Hence, if the color image is in focus and sharp on the sensor plane, the NIR image captured with the same focus settings is out of focus and blurred. We propose an algorithm that retrieves the lost details in NIR using the gradients of the sharp color image. As the high-frequency details of color and NIR images are not strongly correlated in all image patches, our method locally adapts the contribution of color gradients in deblurring. To achieve this, we develop a multiscale scheme that iterates between deblurring NIR and estimating the correlation between color and NIR high-frequency components. Our algorithm outperforms both blind and guided deblurring approaches. We also design a method that estimates a dense blur-kernel map when the severity of chromatic aberration changes as the depths of objects vary across the image. Our method performs better than the competing methods both in estimating the blur-kernel map and in deblurring.

EPFL2015

Material-Based Object Segmentation Using Near-Infrared Information

Neda Salamati, Sabine Süsstrunk

We present a framework to incorporate near-infrared (NIR) information into algorithms to better segment objects by isolating material boundaries from color and shadow edges. Most segmentation algorithms assign individual regions to parts of the object that are colorized differently. Similarly, the presence of shadows and thus large changes in image intensities across objects can also result in mis-segmentation. We first form an intrinsic image from the R, G, B, and NIR channels based on a 4-sensor camera calibration model that is invariant to shadows. The regions obtained by the segmentation algorithms are thus only due to color and material changes and are independent of the illumination. Additionally, we also segment the NIR channel only. Near-infrared (NIR) image intensities are largely dependent on the chemistry of the material and have no general correlation with visible color information. Consequently, the NIR segmentation only highlights material and lighting changes. The union of both segmentations obtained from the intrinsic and NIR images results in image partitions that are only based on material changes and not on color or shadows. Experiments show that the proposed method provides good object-based segmentation results on diverse images.

2010

MULTISCALE GUIDED DEBLURRING: CHROMATIC ABERRATION CORRECTION IN COLOR AND NEAR-INFRARED IMAGING

Yue Lu, Zahra Sadeghipoor Kermani, Sabine Süsstrunk

Chromatic aberration, caused by photographic lens imperfections, results in the image of only one spectral channel being sharp, while the other channels are blurred depending on their wavelengths difference with the sharp channel. We study chromatic aberration for a system that jointly records color and near-infrared (NIR) images on a single sensor. Chromatic aberration in such a system leads to a blurred NIR image when the color image is in-focus and sharp. We propose an algorithm that deblurs the NIR image using the gradients of the sharp color image, as both scene representations are generally similar. However, the details of these images often exhibit significant differences due to varying scene reflection and absorption in the corresponding bands. To account for this, we compute the correlation between color and NIR gradients, and use the gradients of the color image in reconstructing NIR only where the gradients are highly correlated. We propose a multiscale scheme that gradually deblurs NIR and accurately computes similarities between color and NIR gradients. Experimental results show that our algorithm recovers details of NIR without producing visible artifacts.

2015

EPFL2015