Sampling Geometry and Colour | EPFL Graph Search

In this thesis, we take a signal-processing approach to two research areas outside of the core of the signal processing research: geometry reconstruction and light propagation through non-uniform media.In the first area, we consider new sampling schemes inspired by problems of surface recovery and continuous localisation. We analyse sampling at unknown locations and show signal recovery uniqueness under constraints. We also study non-synchronised sampling of vector-valued signals or fields, that arise in the problem of localisation of a smoothly moving device. We provide sufficient conditions for device localisation. To this end, we formulate the problem of rank-one sensing of full-rank matrices and develop sufficient conditions for matrix recovery when some measurements are missing.In the second area, we study Lippmann photography, a century-old colour photography technique. We model recording and viewing of a Lippmann photograph as analysis and synthesis operators and study the invertibility of their composition. We extend our analysis to a potential method of creating digital photographs, namely internal modification of glass with femto-second lasers.

Sampling the Multiple Facets of Light

Gilles Baechler

The theme of this thesis revolves around three important manifestations of light, namely its corpuscular, wave and electromagnetic nature. Our goal is to exploit these principles to analyze, design and build imaging modalities by developing new signal processing and algorithmic tools, based in particular on sampling and sparsity concepts. First, we introduce a new sampling scheme called variable pulse width, which is based on the finite rate of innovation (FRI) sampling paradigm. This new framework enables to sample and perfectly reconstruct weighted sums of Lorentzians; perfect reconstruction from sampled signals is guaranteed by a set of theorems. Second, we turn to the context of light and study its reflection, which is based on the corpuscular model of light. More precisely, we propose to use our FRI-based model to represent bidirectional reflectance distribution functions. We develop dedicated light domes to acquire reflectance functions and use the measurements obtained to demonstrate the usefulness and versatility of our model. In particular, we concentrate on the representation of specularities, which are sharp and bright components generated by the direct reflection of light on surfaces. Third, we explore the wave nature of light through Lippmann photography, a century-old photography technique that acquires the entire spectrum of visible light. This fascinating process captures interferences patterns created by the exposed scene inside the depth of a photosensitive plate. By illuminating the developed plate with a neutral light source, the reflected spectrum corresponds to that of the exposed scene. We propose a mathematical model which precisely explains the technique and demonstrate that the spectrum reproduction suffers from a number of distortions due to the finite depth of the plate and the choice of reflector. In addition to describing these artifacts, we describe an algorithm to invert them, essentially recovering the original spectrum of the exposed scene. Next, the wave nature of light is further generalized to the electromagnetic theory, which we invoke to leverage the concept of polarization of light. We also return to the topic of the representation of reflectance functions and focus this time on the separation of the specular component from the other reflections. We exploit the fact that the polarization of light is preserved in specular reflections and investigate camera designs with polarizing micro-filters with different orientations placed just in front of the camera sensor; the different polarizations of the filters create a mosaic image, from which we propose to extract the specular component. We apply our demosaicing method to several scenes and additionally demonstrate that our approach improves photometric stereo. Finally, we delve into the problem of retrieving the phase information of a sparse signal from the magnitude of its Fourier transform. We propose an algorithm that resolves the phase retrieval problem for sparse signals in three stages. Unlike traditional approaches that recover a discrete approximation of the underlying signal, our algorithm estimates the signal on a continuous domain, which makes it the first of its kind. The concluding chapter outlines several avenues for future research, like new optical devices such as displays and digital cameras, inspired by the topic of Lippmann photography.

EPFL2018

Uncertain Sampling with Certain Priors

Golnooshsadat Elhami

Sampling has always been at the heart of signal processing providing a bridge between the analogue world and discrete representations of it, as our ability to process data in continuous space is quite limited. Furthermore, sampling plays a key part in understanding how to efficiently capture, store and process signals. Shannon's sampling theorem states that if the original signal is known to have a limited bandwidth, we can retrieve the signal from uniformly-spaced samples, provided that the sampling rate is greater than twice the highest frequency in the signal. Here, we see two key attributes: prior knowledge on the original signal (limited bandwidth) and a constrained sampling setup (uniform samples at a particular rate). In this thesis, we make weaker assumptions on the sampling setup by assuming that some information, such as the sample positions, is lost. We show that under proper prior knowledge, we can reconstruct the signal from its samples uniquely or up to some equivalence class. We start by the problem of linear sampling of discrete signals, where the sample values are known, but their order is lost. In general, the original signal is impossible to retrieve from the samples, but we show that by taking out symmetry from the sampling vectors, we can reconstruct the signal uniquely. We provide an efficient algorithm to find the sample orders and thus reconstruct the original signal. We also study the problem of reconstructing a continuous signal from samples taken at unknown locations. The lost sample locations take away any hope of uniquely retrieving the signal without prior knowledge. We show that this problem is equivalent to reconstructing a composite of functions from uniformly spaced samples. Then we provide an efficient algorithm that can recover bandlimited signals warped by a linear function uniquely given enough sampling frequency. We then investigate a problem, dubbed shape from bandwidth, where we have uniform samples from a picture (projection) of an unknown surface that is painted with an unknown texture. The goal is to reconstruct the shape of the surface from these samples. We show that having prior knowledge of the texture bandwidth provides us with enough information to reconstruct the surface from its picture. We provide reconstruction algorithms for both orthogonal and central projections and provide equivalence classes of solutions in each case. Next, in two consecutive chapters, using techniques from geometrical signal processing, we offer new designs for 3-D barcodes, whose information can be retrieved from a single projection using penetrating waves from an unknown direction. Because of the unknown scan direction, the correct correspondence of the samples to the information bits in the barcodes is lost. In this case, we use the known shape of the barcode as prior knowledge to estimate the unknown scan direction from the samples, and then transform the reconstruction as a linear problem that can be solved efficiently. Finally, we cover the theory of coordinate difference matrices (CDMs): matrices that store mutual differences between coordinates of points (sensors, microphones, molecules, etc.) in space. We show how we can leverage specific properties of these matrices, such as their low rank, as prior knowledge in order to reconstruct the position of the points in space using CDMs. We use our reconstruction algorithm to solve many real-life signal processing problems.

EPFL2021

Lippmann Photography: A Signal Processing Perspective

Gilles Baechler, Arnaud Latty, Michalina Wanda Pacholska, Adam James Scholefield, Martin Vetterli

Lippmann (or interferential) photography is the first and only analog photography method that can capture the full color spectrum of a scene in a single take. This technique, invented more than a hundred years ago, records the colors by creating interference patterns inside the photosensitive plate. Lippmann photography provides a great opportunity to demonstrate several fundamental concepts in signal processing. Conversely, a signal processing perspective enables us to shed new light on the technique. In our previous work (G. Baechler et al., 2021), we analyzed the spectra of historical Lippmann plates using our own mathematical model. In this paper, we provide the derivation of this model and validate it experimentally. We highlight new behaviors whose explanations were ignored by physicists to date. In particular, we show that the spectra generated by Lippmann plates are in fact distorted versions of the original spectra. We also show that these distortions are influenced by the thickness of the plate and the reflection coefficient of the reflective medium used in the capture of the photographs. We verify our model with extensive experiments on our own Lippmann photographs.