Bastian Schnell | EPFL Graph Search

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Controllability and Interpretability in Affective Speech Synthesis

Bastian Schnell

Thanks to Deep Learning Text-To-Speech (TTS) has achieved high audio quality with large databases. But at the same time the complex models lost any ability to control or interpret the generation process. For the big challenge of affective TTS it is infeasible to record databases for all varieties. We belive that affective TTS can only be enabled with models which generalise better to the variability in speech thanks to components which are interpretable by humans.In this thesis we aim to do so by incorporating prior knowledge about speech and the physiological production of it in the TTS framework. We introduce well-established signal processing techniques to Neural Networks. Starting from emphasised speech we investigate the intonation production with a physiological plausible intonation model previously developed at Idiap. In order to generalise the model to longer prosodic sequences, we emulate a Spiking Neural Network (SNN) with a Recurrent Neural Network with trainable second-order recurrent elements trained with a learning function inspired from SNNs. The model synthesises neutral intonation with high naturalness and retains the physiological plausibility and controllability of the intonation model. After intonation, we look into spectral features in the aspect of formant frequencies, which have shown to be indicators of certain emotions. Based on the speaker adaptation technique Vocal Tract Length Normalisation we propose a back-propagatable time-varying All Pass Warp (APW ). Experiments of the APW in few- and zero-shot speaker adaptation shows its effectiveness in low-data regimes. In emotional TTS it is not able to increase expressiveness or audio quality, but our analysis shows that the warping correlates with the level of valence in the emotion. We assume that localisation of emotion within an utterance is necessary for the warping (and an affective TTS model in general). We suggest to extract frame-level emotion intensity with an emotion recogniser in an unsupervised manner. The emotion intensity input is a scalable and interpretable control and is able to increase the amount of correctly perceived emotion by humans. We also propose to increase the quantity of emotional data with a language-agnostic emotional voice conversion model. The model achieves high-quality emotion conversion in German by exploiting large amounts of emotional data in English. To train it we develop a novel contribution to gradient reversal.We are able to demonstrate that Deep Learning TTS models can benefit from well-established signal processing techniques and interpretable low-dimensional controls to improve their generalisability in low-data regimes and/or allow simple controllability without loosing on quality. The developed models also allow interpretability for future physiological and linguistic analysis. While this thesis provides a toolbox of independent controls their combination presents itself as a next step towards a comprehensive framework for affective TTS.

EPFL2022

An End-to-End Networks to Synthetize Intonation using a Generalized Command Response Model

Hervé Bourlard, Philip Neil Garner, Francois Thierry M Marelli, Bastian Schnell

The generalized command response (GCR) model represents intonation as a superposition of muscle responses to spike command signals. We have previously shown that the spikes can be predicted by a two-stage system, consisting of a recurrent neural network and a post-processing procedure, but the responses themselves were fixed dictionary atoms. We propose an end-to-end neural architecture that replaces the dictionary atoms with trainable second-order recurrent elements analogous to recursive filters. We demonstrate gradient stability under modest conditions, and show that the system can be trained by imposing temporal sparsity constraints. Subjective listening tests demonstrate that the system can synthesize intonation with high naturalness, comparable to state-of-the-art acoustic models, and retains the physiological plausibility of the GCR model.

Idiap2019

Neural VTLN for Speaker Adaptation in TTS

Philip Neil Garner, Bastian Schnell

Vocal tract length normalisation (VTLN) is well established as a speaker adaptation technique that can work with very little adaptation data. It is also well known that VTLN can be cast as a linear transform in the cepstral domain. Building on this latter property, we show that it can be cast as a (linear) layer in a deep neural network (DNN) for speech synthesis. We show that VTLN parameters can then be trained in the same framework as the rest of the DNN using automatic gradients. Experimental results show that the DNN is capable of predicting phone-dependent warpings on artificial data, and that such warpings improve the quality of an acoustic model on real data in subjective listening tests.

2019