On The Relationship Between Speech-Based Breathing Signal Prediction Evaluation Measures And Breathing Parameters Estimation

Mathew Magimai Doss, Zohreh Mostaani, Venkata Srikanth Nallanthighal
IEEE, 2021
Article de conférence

Résumé

The respiratory system is one of the major components of the speech production system. Any alteration in breathing can result in changes in speech. Specific breathing characteristics, such as breathing rate and tidal volume, can indicate a person's pathological condition. More recently, neural network-based methods have started emerging for predicting the breathing signal from the speech signal. The neural networks are trained and evaluated with different objective measures, such as mean squared error (MSE) and Pearson's correlation. This paper investigates whether there is a systematic relationship between the different objective measures used for training and evaluating the neural network models and the end-goal, i.e. estimation of breathing parameters such as, breathing rate and tidal volume. Our investigations on two different data sets with two different neural network-based approaches show that there is no clear systematic relationship. In other words, obtaining a high Pearson's correlation on the evaluation set does not necessarily mean better breathing parameter estimation. Thus, indicating the need for developing other objective evaluation measures.

Official source

https://infoscience.epfl.ch/record/290529?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

ON THE RELATIONSHIP BETWEEN SPEECH-BASED BREATHING SIGNAL PREDICTION EVALUATION MEASURES AND BREATHING PARAMETERS ESTIMATION

Zohreh Mostaani, Venkata Srikanth Nallanthighal

2021

Chargement

Novel Methods for Incorporating Prior Knowledge for Automatic Speech Assessment

Subrahmanya Pavankumar Dubagunta

Speech signal conveys several kinds of information such as a message, speaker identity, emotional state of the speaker and social state of the speaker. Automatic speech assessment is a broad area that refers to using automatic methods to predict human judgements regarding different kinds of information conveyed in speech, such as intelligibility of the spoken message, dialect and fluency of the speaker. Unlike other speech technology areas, such as automatic speech recognition, text-to-speech synthesis and automatic speaker recognition, automatic speech assessment is an emerging direction of research. One of the challenges in this field is that there is no single method or framework that scales across diverse speech assessment tasks. Thus, this thesis takes a broader outlook and focuses on prior knowledge incorporation for diverse data-driven speech assessment problems.First, we focus on the development of end-to-end acoustic modelling methods for non-verbal cue-based speech assessment. More precisely, we develop neural network-based methods that can integrate prior knowledge about speech production to learn to assess speech from raw waveform. We validate the developed methods through investigations on several speech assessment tasks, viz. dialect identification, depression detection and speech fluency rating prediction.Second, we focus on advancing a recently proposed phone posterior feature-based intelligibility estimation technique. Specifically, to enhance phone posterior probability estimation, we propose two novel approaches to incorporate linguistic segment level knowledge during the training of neural networks through estimation of confidence measures. We validate the two proposed approaches through automatic speech recognition and dysarthric speech intelligibility assessment studies.Finally, in the context of privacy preservation, we develop a signal processing-based speech pseudonymization approach that alters voice source information and vocal tract system information based on prior knowledge to obfuscate the speaker identity, while retaining intelligibility, i.e. the phones and words remain recognizable. We validate the proposed pseudonymization approach through listening experiments and automatic evaluations.

EPFL2021

Chargement

Thanks to Deep Learning Text-To-Speech (TTS) has achieved high audio quality with large databases. But at the same time the complex models lost any ability to control or interpret the generation process. For the big challenge of affective TTS it is infeasible to record databases for all varieties. We belive that affective TTS can only be enabled with models which generalise better to the variability in speech thanks to components which are interpretable by humans.In this thesis we aim to do so by incorporating prior knowledge about speech and the physiological production of it in the TTS framework. We introduce well-established signal processing techniques to Neural Networks. Starting from emphasised speech we investigate the intonation production with a physiological plausible intonation model previously developed at Idiap. In order to generalise the model to longer prosodic sequences, we emulate a Spiking Neural Network (SNN) with a Recurrent Neural Network with trainable second-order recurrent elements trained with a learning function inspired from SNNs. The model synthesises neutral intonation with high naturalness and retains the physiological plausibility and controllability of the intonation model. After intonation, we look into spectral features in the aspect of formant frequencies, which have shown to be indicators of certain emotions. Based on the speaker adaptation technique Vocal Tract Length Normalisation we propose a back-propagatable time-varying All Pass Warp (APW ). Experiments of the APW in few- and zero-shot speaker adaptation shows its effectiveness in low-data regimes. In emotional TTS it is not able to increase expressiveness or audio quality, but our analysis shows that the warping correlates with the level of valence in the emotion. We assume that localisation of emotion within an utterance is necessary for the warping (and an affective TTS model in general). We suggest to extract frame-level emotion intensity with an emotion recogniser in an unsupervised manner. The emotion intensity input is a scalable and interpretable control and is able to increase the amount of correctly perceived emotion by humans. We also propose to increase the quantity of emotional data with a language-agnostic emotional voice conversion model. The model achieves high-quality emotion conversion in German by exploiting large amounts of emotional data in English. To train it we develop a novel contribution to gradient reversal.We are able to demonstrate that Deep Learning TTS models can benefit from well-established signal processing techniques and interpretable low-dimensional controls to improve their generalisability in low-data regimes and/or allow simple controllability without loosing on quality. The developed models also allow interpretability for future physiological and linguistic analysis. While this thesis provides a toolbox of independent controls their combination presents itself as a next step towards a comprehensive framework for affective TTS.

EPFL2022

Chargement

Novel Methods for Incorporating Prior Knowledge for Automatic Speech Assessment

Subrahmanya Pavankumar Dubagunta

EPFL2021

EPFL2022