Aural and automatic forensic speaker recognition in mismatched conditions

In this article, we compare aural and automatic speaker recognition in the context of forensic analyses, using a Bayesian framework for the interpretation of evidence. We use perceptual tests performed by non-experts and compare their performance with that of an automatic speaker recognition system. These experiments are performed with 90 phonetically untrained subjects. Several forensic cases were simulated, using the Polyphone IPSC-02 database, varying in linguistic content and technical conditions of recording. We estimate the strength of evidence for both humans and the baseline automatic system, calculating likelihood ratios using perceptual scores for humans and log-likelihood scores for the automatic system. A methodology analogous to the Bayesian interpretation in forensic automatic speaker recognition is applied to the perceptual scores given by humans in order to estimate the strength of evidence. The degradation of the accuracy of human recognition in mismatched recording conditions is contrasted with that of the automatic system under similar recording conditions. The conditions considered are fixed telephone, cellular telephone and noisy speech in forensically realistic conditions. The perceptual cues that the human subjects use to perceive differences in voices are studied, along with their importance in different recording conditions. We observe that while automatic speaker recognition shows higher accuracy in matched conditions of training and testing, its performance degrades significantly in mismatched conditions. Aural recognition accuracy is also observed to degrade from matched conditions to mismatched conditions and in mismatched conditions, the baseline automatic systems showed comparable or slightly degraded performance compared to the aural recognition systems. The baseline automatic system with adaptation to noisy conditions showed comparable or better performance than aural recognition. The higher level perceptual cues used by human listeners in order to recognise speakers are discussed. We also discuss the possibility of increasing the accuracy of automatic systems using the perceptual cues that remain robust to mismatched recording conditions.

Unified Framework of Feature Based Adaptation for Statistical Speech Synthesis and Recognition

Lakshmi Babu Saheer

The advent of statistical parametric speech synthesis has paved new ways to a unified framework for hidden Markov model (HMM) based text to speech synthesis (TTS) and automatic speech recognition (ASR). The techniques and advancements made in the field of ASR can now be adopted in the domain of synthesis. Speaker adaptation is a well-advanced topic in the area of ASR, where the adaptation data from a target speaker is used to transform the canonical model parameters to represent a speaker specific model. Feature adaptation techniques like vocal tract length normalization (VTLN) perform the same task by transforming the features ; this can be shown to be equivalent to model transformation. The main advantage of VTLN is that it can demonstrate noticeable improvements in performance with very little adaptation data and can be classified as a rapid adaptation technique. VTLN is a widely used technique in ASR, and can be used in TTS to improve the rapid adaptation performance. In TTS, the task is to synthesize speech that sounds like a particular target speaker. Using VTLN for TTS is found to make the output synthesized speech sound quite similar to the target speaker from his very first utterance. An all-pass filter based bilinear transform was implemented for the mel-generalized cepstral (MGCEP) features of the HMM-based speech synthesis system (HTS). The initial implementation was using a grid search approach that selects the best warping factor for the speech spectrum from a grid of available values using the maximum likelihood criterion. VTLN was shown to give performance improvements in the rapid adaptation framework where the number of adaptation sentences from the target speaker was limited. But, this technique involves huge time and space complexities and the rapid adaptation demands for an efficient implementation of the VTLN technique. To this end, an efficient expectation maximization (EM) based VTLN approach was implemented for HTS using Brent’s search. Unlike the ASR features, MGCEP does not use a filter bank (in order to facilitate the speech reconstruction) and this provides equivalence to the model transformation for the EM implementation. This facilitates the estimation of warping factors to be embedded in the HMM training using the same sufficient statistics as in constrained maximum likelihood linear regression (CMLLR). This work addresses a lot of challenges faced in the process of adopting VTLN for synthesis due to the higher dimensionality of the cepstral features used in the TTS models. The main idea was to unify the theory and practise in the implementation of VTLN for both ASR and TTS. Several techniques have been proposed in this thesis, in order to find the best feasible warping factor estimation procedure. Estimation of the warping factor using the lower order cepstral features representing the spectral envelope is demonstrated to be the best approach. Different evaluations on standard databases are performed in this work to illustrate the performance improvements and perceptual challenges involved in the VTLN adaptation. VTLN has only a single parameter to represent the speaker characteristics and hence, has the limitation of not scaling to the performance of other linear transform based adaptation methods with the availability of large amounts of adaptation data. Several techniques are demonstrated in this work to combine the model based adaptation like constrained structural maximum a posteriori linear regression (CSMAPLR) with VTLN, one such technique being using VTLN as the prior transform at the root node of the tree structure of the CSMAPLR system. Thus, along with rapid adaptation, the performance scales with the availability of more adaptation data. These techniques although developed for TTS, can also be effectively used in ASR. It was also shown to give improvements in ASR especially for scenarios like noisy speech conditions. Other improvements to rapid adaptation including a bias term for VTLN, multiple transform based VTLN using regression classes and VTLN prior for non-structural MAPLR adaptation are also proposed. These techniques also demonstrated both ASR and TTS performance improvements. Also, a few special scenarios, specifically cross-lingual speech, cross-gender speech, child speech and noisy speech evaluations are presented where the rapid adaptation methods presented in this work was shown to be highly beneficial. Most of these methods will be published as extensions to the open-source HTS toolkit.

EPFL2013

Analysis of Anticipation Related Potentials for Brain Computer Interaction

Gangadhar Garipelli

Anticipation is a mental process during which a person actively engages in a phase required for the sensory perception and execution of the optimal actions at the arrival of the relevant future events. Since this process occurs before the execution of an intended action, it may be used as a control signal for Brain Computer Interface (BCI) applications. Recognition of neural correlates of this process can enhance the performance of a BCI and in turn reduce mental workload of its users. To this end, it is vital to understand the neural correlates involved in this process and to design robust methods for its recognition in single trials. The analysis of these correlates may also contribute to the basic knowledge of the mechanisms underlying this behavior. The thesis provides three major contributions: (i) it reports methods for the robust recognition of anticipation related Electroencephalogram (EEG) potentials (ii) it provides insights into the selection of appropriate preprocessing steps required for enhancing the Signal-to-Noise Ratio (SNR) of anticipatory slow cortical potentials (SCPs) and (iii) it identifies scalp area specific oscillatory activity related to different aspects of anticipatory behavior. First, we focus on methods for the single trial recognition of anticipatory SCPs using the widely known classical contingent negative variation (CNV) paradigm. Using this paradigm, we demonstrate the feasibility of recognizing the anticipatory SCPs (CNV potentials) using features thatmodel its temporal pattern. We propose a Bayesian approach that exploits temporal evolution and redundancy to quickly classify (e.g., within half of the anticipatory period), without compromising classification accuracy. We then improve upon these recognition rates by using a source localization technique based on the biophysical model of the human head. We further validate the feasibility of recognizing CNV potentials in an online experiment, and report for the first time that, under controlled conditions, these potentials can be reproduced and recognized in realistic interaction scenarios (assistive technology web-browsing) with high accuracies. Second, the thesis provides insights into the selection of appropriate preprocessing stages required for improving the SNR of SCPs. The CNV potentials are characterized by low frequencies that are usually recorded with full-DC, and hence suffer from task-irrelevant high amplitude fluctuations and spatial noise. To account for this, we identified appropriate spectral and spatial filters to improve the SNR. We demonstrate the potential of these preprocessing stages by using fusing multiple electrode specific linear classifiers, which achieve recognition performances of 90±2% (area under curve of receiver operating characteristic), where the classifiers are trained using recordings from one day and tested on the recordings from several days apart. Finally, the thesis identifies different facets of anticipatory behavior. Apart from the widely known CNV potentials, it is not clear which other spectral bands could be related to anticipatory behavior. Using recordings from an experiment (i.e., the assistive technology web browser) where multiple warning stimuli predicted an imperative stimulus, we explored the phase and amplitude response of various oscillatory sub-bands for the identification of markers that could be associated with different aspects of anticipation. From this study we report that: (i) there are duration (4-10 seconds) specific changes in Electroencephalography (EEG) activity in the range 0-1 Hz in the central electrodes correlate with reaction time, (ii) there exist slow oscillations (0.1-1 Hz) in the central electrodes that exhibit phase tuning up to 4 seconds before the onset of a target cue, (iii) there are delta oscillations (1.5-3 Hz), which are entrained to predictive rhythmic warning cues, (iv) there is a selective modulation (increase or decrease) in the amplitude of occipital alpha band (8-12 Hz) based on the relevance of forthcoming visual cue, and (v) there exist a reduction in the beta band (14-30 Hz) amplitude in the sensory motor and association areas lasting up to 10 seconds. The phase tuning and entrainment resulted in a low variance of phase values at the arrival of the imperative stimulus, which may be required for its optimal processing. The amplitude modulation of alpha band activity is likely to be a resultant of sensory suppression and attention. The reduction of beta-band activity over long periods of time suggests holding of sensory-motor association areas until the execution of a planned action. We believe that these observations are the consequence of the endogenous drive on the ongoing oscillations to enhance the processing of the forthcoming stimuli and preparation for an intended action. In summary, the thesis provides methods for the recognition of anticipatory SCPs by exploiting spectral and spatio-temporal characteristics with high performances. By exploring various other oscillatory sub-bands, the spectral characteristics of different aspects of anticipatory behavior are also identified. Further, methods modeling these characteristics can bring forth more robust and faster techniques for BCI systems.

EPFL2011

Inference in switching linear dynamical systems applied to noise robust speech recognition of isolated digits

Bertrand Mesot

Real world applications such as hands-free dialling in cars may have to perform recognition of spoken digits in potentially very noisy environments. Existing state-of-the-art solutions to this problem use feature-based Hidden Markov Models (HMMs), with a preprocessing stage to clean the noisy signal. However, the effect that the noise has on the induced HMM features is difficult to model exactly and limits the performance of the HMM system. An alternative to feature-based HMMs is to model the clean speech waveform directly, which has the potential advantage that including an explicit model of additive noise is straightforward. One of the most simple model of the clean speech waveform is the autoregressive (AR) process. Being too simple to cope with the nonlinearity of the speech signal, the AR process is generally embedded into a more elaborate model, such as the Switching Autoregressive HMM (SAR-HMM). In this thesis, we extend the SAR-HMM to jointly model the clean speech waveform and additive Gaussian white noise. This is achieved by using a Switching Linear Dynamical System (SLDS) whose internal dynamics is autoregressive. On an isolated digit recognition task where utterances have been corrupted by additive Gaussian white noise, the proposed SLDS outperforms a state-of-the-art HMM system. For more natural noise sources, at low signal to noise ratios (SNRs), it is also significantly more accurate than a feature-based HMM system. Inferring the clean waveform from the observed noisy signal with a SLDS is formally intractable, resulting in many approximation strategies in the literature. In this thesis, we present the Expectation Correction (EC) approximation. The algorithm has excellent numerical performance compared to a wide range of competing techniques, and provides a stable and accurate linear-time approximation which scales well to long time series such as those found in acoustic modelling. A fundamental issue faced by models based on AR processes is that they are sensitive to variations in the amplitude of the signal. One way to overcome this limitation is to use Gain Adaptation (GA) to adjust the amplitude by maximising the likelihood of the observed signal. However, adjusting model parameters without constraint may lead to overfitting when the models are sufficiently flexible. In this thesis, we propose a statistically principled alternative based on an exact Bayesian procedure in which priors are explicitly defined on the parameters of the underlying AR process. Compared to GA, the Bayesian approach enhances recognition accuracy at high SNRs, but is slightly less accurate at low SNRs.

Ecole Polytechnique Fédérale de Lausanne2008