Time–frequency representation

A time–frequency representation (TFR) is a view of a signal (taken to be a function of time) represented over both time and frequency. Time–frequency analysis means analysis into the time–frequency domain provided by a TFR. This is achieved by using a formulation often called "Time–Frequency Distribution", abbreviated as TFD. TFRs are often complex-valued fields over time and frequency, where the modulus of the field represents either amplitude or "energy density" (the concentration of the root mean square over time and frequency), and the argument of the field represents phase. Background and motivation A signal, as a function of time, may be considered as a representation with perfect time resolution. In contrast, the magnitude of the Fourier transform (FT) of the signal may be considered as a representation with perfect spectral resolution but with no time information because the magnitude of the FT conveys frequency content but it fails to convey when, in time, differ

Source coding in sensor networks

Robert Konsbruck

Wireless sensor networks have emerged as versatile tools to monitor the evolution of physical fields over large areas by means of self-powered and low-cost sensing devices. Although their development was originally motivated by military applications, they are nowadays used in numerous civilian application areas. In many sensor network scenarios, including the ones studied in this thesis, the goal of data acquisition by means of sensor nodes is to reproduce at a base station the observed field under a fidelity constraint, using limited communication resources. Due to the scarcity of the energy resources, the physical nature of the observed fields, the constraints on spatial sampling, the high cost of inter-node communication and the reliance on a shared communication medium, wireless sensor networks raise challenging questions in the areas of multidimensional sampling, multiterminal source coding and network communications. In this thesis we address the issue of how to efficiently represent the data collected by the sensor nodes. This data has some inherent structure that is determined by the laws of physics. Since the design of efficient sampling geometries and source coding schemes requires a thorough understanding of the physical properties of the observed phenomenon, the first part of our work is devoted to setting up a mathematical model for physical fields that is amenable to the tools of information theory while applying to most physical phenomena of interest. The question regarding the sufficiency of a discrete representation of an analog field is addressed in two steps. First, we prove a multidimensional sampling theorem for homogeneous random fields with compactly supported spectral measures. While, under appropriate conditions, a discrete-space and -time representation of the analog field is sufficient, discretizing the field's amplitude, also referred to as source coding, implies an unavoidable loss of information, which may be expressed in terms of a rate distortion function. In the second step we study various source coding schemes, differing by the amount of required inter-node communication and the complexity of the involved maps. We derive lower and upper bounds for the rate distortion function of the multiterminal source coding scheme, which is of particular interest in sensor network engineering as it makes use of the spatio-temporal structure of the collected data without requiring inter-node communication. In the second part of the thesis we study some applications of sensor networks. In particular, we consider the acquisition of acoustic waves, the monitoring of temperature distributions and the compression of data sequences generated by random walks. For the setup of sound field acquisition, we show that, under the assumption of spectral whiteness of the sound field, the afore-mentioned bounds coincide, thus establishing the multiterminal rate distortion function for this setup.

EPFL2009

Musical Audio Source Separation Based on User-Selected F0 Track

Jean-Philippe Thiran

A system for user-guided audio source separation is presented in this article. Following previous works on time-frequency music representations, the proposed User Interface allows the user to select the desired audio source, by means of the assumed fundamental frequency (F0) track of that source. The system then automatically refines the selected F0 tracks, estimates and separates the corresponding source from the mixture. The interface was tested and the separation results compare positively to the results of a fully automatic system, showing that the F0 track selection improves the separation performance.

2012

Unpaired Cross-Spectral Pedestrian Detection Via Adversarial Feature Learning

Seungryong Kim

Even though there exist significant advances in recent studies, existing methods for pedestrian detection still have shown limited performances under challenging illumination conditions especially at nighttime. To address this, cross-spectral pedestrian detection methods have been presented using color and thermal, and shown substantial performance gains on the challenging circumstances. However, their paired cross-spectral settings have limited applicability in real-world scenarios. To overcome this, we propose a novel learning framework for cross-spectral pedestrian detection in an unpaired setting. Based on an assumption that features from color and thermal images share their characteristics in a common feature space to benefit their complement information, we design the separate feature embedding networks for color and thermal images followed by sharing detection networks. To further improve the cross-spectral feature representation, we apply an adversarial learning scheme to intermediate features of the color and thermal images. Experiments demonstrate the outstanding performance of the proposed method on the KAIST multi-spectral benchmark in comparison to the state-of-the-art methods.

IEEE2019

Source coding in sensor networks

Robert Konsbruck

EPFL2009