Exploring information retrieval using image sparse representations

New advances in the field of image sensors (especially in CMOS technology) tend to question the conventional methods used to acquire the image. Compressive Sensing (CS) plays a major role in this, especially to unclog the Analog to Digital Converters which are generally representing the bottleneck of this type of sensors. In addition, CS eliminates traditional compression processing stages that are performed by embedded digital signal processors dedicated to this purpose. The interest is twofold because it allows both to consistently reduce the amount of data to be converted but also to suppress digital processing performed out of the sensor chip. For the moment, regarding the use of CS in image sensors, the main route of exploration as well as the intended applications aims at reducing power consumption related to these components (i.e. ADC & DSP represent 99% of the total power consumption). More broadly, the paradigm of CS allows to question or at least to extend the Nyquist-Shannon sampling theory. This thesis shows developments in the field of image sensors demonstrating that is possible to consider alternative applications linked to CS. Indeed, advances are presented in the fields of hyperspectral imaging, super-resolution, high dynamic range, high speed and non-uniform sampling. In particular, three research axes have been deepened, aiming to design proper architectures and acquisition processes with their associated reconstruction techniques taking advantage of image sparse representations. How the on-chip implementation of Compressed Sensing can relax sensor constraints, improving the acquisition characteristics (speed, dynamic range, power consumption) ? How CS can be combined with simple analysis to provide useful image features for high level applications (adding semantic information) and improve the reconstructed image quality at a certain compression ratio ? Finally, how CS can improve physical limitations (i.e. spectral sensitivity and pixel pitch) of imaging systems without a major impact neither on the sensing strategy nor on the optical elements involved ? A CMOS image sensor has been developed and manufactured during this Ph.D. to validate concepts such as the High Dynamic Range - CS. A new design approach was employed resulting in innovative solutions for pixels addressing and conversion to perform specific acquisition in a compressed mode. On the other hand, the principle of adaptive CS combined with the non-uniform sampling has been developed. Possible implementations of this type of acquisition are proposed. Finally, preliminary works are exhibited on the use of Liquid Crystal Devices to allow hyperspectral imaging combined with spatial super-resolution. The conclusion of this study can be summarized as follows: CS must now be considered as a toolbox for defining more easily compromises between the different characteristics of the sensors: integration time, converters speed, dynamic range, resolution and digital processing resources. However, if CS relaxes some material constraints at the sensor level, it is possible that the collected data are difficult to interpret and process at the decoder side, involving massive computational resources compared to so-called conventional techniques. The application field is wide, implying that for a targeted application, an accurate characterization of the constraints concerning both the sensor (encoder), but also the decoder need to be defined.

Bits from Photons

Feng Yang

The trends in the design of image sensors are to build sensors with low noise, high sensitivity, high dynamic range, and small pixel size. How can we benefit from pixels with small size and high sensitivity? In this dissertation, we study a new image sensor that is reminiscent of traditional photographic film. Each pixel in the sensor has a binary response, giving only a one-bit quantized measurement of the local light intensity. The response function of the image sensor is non-linear and similar to a logarithmic function, which makes the sensor suitable for high dynamic range imaging. We first formulate the oversampled binary sensing scheme as a parameter estimation problem based on quantized Poisson statistics. We show that, with a single-photon quantization threshold and large oversampling factors, the Cramér-Rao lower bound (CRLB) of the estimation variance approaches that of an ideal unquantized sensor, that is, as if there were no quantization in the sensor measurements. Furthermore, the CRLB is shown to be asymptotically achievable by the maximum likelihood estimator (MLE). By showing that the log-likelihood function is concave, we guarantee the global optimality of iterative algorithms in finding the MLE. We study the performance of the oversampled binary sensing scheme in presence of dark current noise. The noise model is an additive Bernoulli noise with a known parameter, and the noise only flips the binary output from "0" to "1". We show that the binary sensor is quite robust with respect to noise and its dynamic range is only slightly reduced. The binary sensor first benefits from the increasing of the oversampling factor and then suffers in term of dynamic range. We again use the MLE to estimate the light intensity. When the threshold is a single photon, we show that the log-likelihood function is still concave. Thus, the global optimality can be achieved. But for thresholds larger than "1", this property does not hold true. By proving that when the light intensity is piecewise-constant, the likelihood function is a strictly pseudoconcave function, we guarantee to find the optimal solution of the MLE using iterative algorithms for arbitrary thresholds. For the general linear light field model, the log-likelihood function is not even quasiconcave when thresholds are larger than "1". In this circumstance, we find an initial solution by approximating the light intensity field with a piecewise-constant model, and then we use Newton's method to refine the estimation using the exact model. We then examine one of the most important parameters in the binary sensor, i.e., the threshold used to generate binary values. We prove the intuitive result that large thresholds achieve better estimation performance for strong light intensities, while small thresholds work better for low light intensities. To make a binary sensor that works in a larger range of light intensities, we propose to design a threshold array containing multiple thresholds instead of a single threshold for the binary sensing. The criterion is to minimize the average CRLB which is a good approximation of the mean squared error (MSE). The performance analysis on the new binary sensor verifies the effectiveness of our design. Again, the MLE is used for reconstructing the light intensity field from the binary measurements. By showing that the log-likelihood function is concave for arbitrary threshold arrays, we ensure that the iterative algorithms can find the optimal solution of the MLE. Finally, we study the reconstruction problem for the binary image sensor under a generalized piecewise-constant light intensity field model, which is quite useful when parameters like oversampling factors are unknown. We directly estimate light exposure values, i.e., the number of photons hitting on each pixel. We assume that the light exposure values are piecewise-constant and we use the MLE for the reconstruction. This optimization problem is solved by iteratively working out two subproblems. The first one is to find the optimal light exposure value for each segment, given the optimal segmentation of the binary measurements. The second one is to find the optimal segmentation of the binary measurements given the optimal light exposure values for each segment. Several algorithms are provided for solving this optimization problem. Dynamic programming can obtain the optimal solution for 1-D signals, but the computation is quite heavy. To reduce the burden of computation, we propose a greedy algorithm and a method based on pruning of binary trees or quadtrees.

EPFL2012

Highly sensitive vertical hall sensors in CMOS technology

Enrico Schurig

Vertical Hall sensors are capable of measuring surface-parallel components of the magnetic field. They allow therefore relatively easy conception of single-chip multi-axial magnetic sensors compared to solutions using horizontal Hall plates. The modern trend in the field of Hall sensors is to integrate them into electronic circuitry for signal processing. Before this thesis, highly sensitive vertical Hall sensors completely compatible to a technology adequate for co-integration of electronic circuitry were not available. This is the reason why we present in this thesis the developed knowledge that is necessary for the design and manufacture of highly sensitive vertical Hall sensors in CMOS technology. We first present the principles of the technology choice. A wide selection of CMOS processes is presently available, but in order to obtain optimal sensitivity we have to choose "the right" one. The performances can vary easily by 50% from one technology to another. The best choice are CMOS high-voltage technologies since they deliver n-diffusion layers with relatively low doping level and deep junction depth. We present a novel layout of vertical Hall sensors that has six contacts in the active sensor zone and is a development of the known layout that uses only four contacts. The additional contacts improve the sensitivity and reduce the systematic offset considerably. If we re-use layouts that were optimal for not-CMOS compatible technologies, we will see that the results are not satisfying. We have to develop our specific design parameters that are optimal for the chosen technology. The key for high sensitivities is the strong miniaturization of the devices down to technological limits given by the design rules, but sometimes even beyond that. Some of the design rules can be broken with benefit for the obtained sensor devices. In general, minimum contact sizes and contact distances are preferable as well as a minimum sensor thickness. The difficulty is however to find the absolute minimum before device degradation or even failure will occur. Our optimized sensors achieved voltage related sensitivities of about 0.04 V/(VT) and current related-sensitivities up to 400 V/(AT) that are comparable to existing CMOS compatible horizontal Hall plates. Such sensitivities were never reported for a standard CMOS process, without any additional pre- or post-processing steps. The miniaturization of the Hall devices has however also negative aspects e.g. an increase in sensor offset, noise and output non-linearity. Although Hall sensors have many advantages in comparison to other magnetic sensors (simple structure, low fabrication costs, very good linearity, robustness) they have the drawback of a big Abstract offset voltage that is often too big for many applications. This is why an enormous effort was made by researchers in the past to develop offset reduction methods for horizontal Hall plates. We state in this thesis that, in principle, the same techniques are applicable on vertical Hall sensors but for an optimal efficiency certain principles/parameters have to be respected. We reveal the relations of important parameters as the type of layout (number of sensor contacts), the use of single or coupled sensors, the type of spinning-current, bias level etc. In the optimal case we can obtain residual offset values lower than 200 μT for bias voltages up to 2 V. The spinning current method was originally developed for offset compensation, but it has other beneficial influences on the sensor output signal as well. It is capable to remove a large amount of flicker noise if the spinning is executes at adequately high clock frequencies. We show that the efficiency of flicker noise removal is dependent on the bias current through the sensor. We discovered another positive side effect of the spinning-current method: the elimination of the planar Hall effect. While the planar Hall sensitivity for our sensors is initially about 1% of the normal one at B = 2 T, we obtain with the compensation technique values of only 0.02%. After the presentation of general characteristics of the separate sensors, we present two microsystems based on the developed vertical Hall sensors. At first, we present a 2D magnetic microsystem which is well adapted for the construction of contactless angular encoders with a measurement range of 360°. We profit here especially from the very low offset (< 400 μT) obtained with the spinning current method, since offset is in general the main source of angular errors in such systems. Existing angular measurement systems need in general a special (offset) calibration of the sensor unit in order to achieve an accuracy of 2.5‰ full scale. We obtain such a precision directly with our system for a wide temperature range from -30 to 100 °C. With one specific sensor calibration for the compensation of residual offset, sensitivity mismatch and phase mismatch of the two axes, we attain a precision of about 0.5‰ FS over the same temperature range, a performance never before reported with a magnetic sensor system. The microsystem is therefore an excellent candidate for the majority of low-cost angular measurement applications. As a second example we presented the first fully CMOS integrated three-dimensional Hall probe, consisting of a combination of one Hall plate and four vertical Hall devices. The microsystem allows an application as a universal magnetic field probe in the wide field range from 0.1 mT to 20 T. The probe can be easily used for small volumes because of its small size and the fact that the signal processing circuitry already is directly integrated onto the chip. The precision as a 3D magnetometer is limited by the sensitivity mismatch of about 5% FS. The probe is an ideal candidate for low-cost teslameter applications with limited precision.

EPFL2005

Hybrid continuous-discrete-time multi-bit delta-sigma A/D converters with auto-ranging algorithm

Sergio Pesenti

In wireless portable applications, a large part of the signal processing is performed in the digital domain. Digital circuits show many advantages. The power consumption and fabrication costs are low even for high levels of complexity. A well established and highly automated design flow allows one to benefit from the constant progress in CMOS technologies. Moreover, digital circuits offer robust and programmable signal processing means and need no external components. Hence, the trend in consumer electronics is to further reduce the part of analog signal processing in the receiver chain of wireless transceivers. Consequently, analog-to-digital converters with higher resolutions and bandwidths are constantly required. The ultimate goal is the direct digitization of radio frequency signals, where the conversion would be performed immediately after the front-end amplifier. ΔΣ-modulation-based converters have proved to be the most suitable to achieve the required performance. Switched-capacitor implementations have been widely used over the last two decades. However, recent publications and books have shown that continuous-time architectures can achieve the same performance with lower power consumption. Most designs found throughout the literature use a single- or few-bit internal quantizer with a high-order modulation. As a result, in order to achieve the resolutions and bandwidths required today, the sampling frequency must exceed 100MHz. This approach leads to non-negligible power consumption in the clock generation. Moreover, the presence of such fast squared signals is not suitable for a system-on-chip comprising radio frequency receivers. In this thesis we propose a low-power strategy relying on a large number of internal levels rather than on a high sampling frequency or modulation order. Besides, a hybrid continuous-discrete-time approach is used to take advantage of the accuracy of switched-capacitor circuits and the low power consumption of continuous-time implementation. The sensitivity to clock jitter brought about by the continuous-time stage is reduced by the use of a large number of levels. An auto-ranging algorithm is developed in this thesis to overcome the limitation of a large-size quantizer under low-voltage supply. Finally, the strategy is applied to a design example addressing typical specifications for a Bluetooth receiver with direct conversion.

EPFL2007