Lexique

Le lexique d'une langue est l'ensemble de ses mots (somme des vocabulaires utilisés), ou de façon plus précise en linguistique de ses lemmes. Les mots d'un lexique forment un tout, une sorte de système sémantique, qui évolue donc au fil du temps. Les rapports entre les mots, de forme et surtout de sens, ainsi qu'entre les sens d'un même mot, sont très divers. La taille d'un lexique est très variable et dépend notamment de la diversité des domaines de connaissance ou techniques qu'elle permet d'exprimer. L'ampleur du lexique connu ou utilisé par chacun varie aussi très fortement, en fonction des milieux mais aussi des histoires et situations de vie. Les différences souvent fondamentales entre lexiques de différentes langues posent un problème essentiel de traduction, sans solution générale. De nouveaux mots peuvent être créés dans toute langue, suivant des modes différents, et plus ou facilement, suivant les langues. Inversement, des mots disparaissent, remplacés ou non dans leur se

Efficient integration of automated speech recognition in the framework of dialogue-based vocal systems

In this work, we propose different strategies for efficiently integrating an automated speech recognition module in the framework of a dialogue-based vocal system. The aim is the study of different ways leading to the improvement of the quality and robustness of the recognition. We first concentrate on the choice of the type of acoustic models that should be used for the speech recognition. Our goal is to evaluate the hypothesis that hybrid acoustic models, in which estimation of frame-based phoneme probabilities is made through artificial neural networks, provide performance results similar to the "classical" Hidden-Markov models using Multi-Gaussian estimations, while being more robust in generalization across tasks. We experimentally show that, due to the size of the parameter space to be explored, it is not always practically possible to achieve a performance comparable to the one of Multi-Gaussian models, and that in fact hybrid models often lead to worse recognition performance. In a second part, we focus on one of the main limitations of state-of-the-art speech recognition: the inadequacy of the one-best approach to yield a hypothesis corresponding to the right transcription. For that, we explore the solution consisting in producing, during acoustic decoding, a word lattice containing a very large number of hypotheses, that is then filtered by a syntactic analyzer using more sophisticated syntactic models, such as stochastic context-free grammars. The goal of this approach is to yield syntactically correct hypotheses for further processing. More precisely, we study the approach proach consisting in dynamically tuning the relative importance of the acoustic and language models, resulting in the increase of the lexical and syntacticonsisting variability in the word lattice. We identify and experimentally quantify two important drawbacks for this approach: its high computational cost and the impossibility to guarantee that, in practice, the correct solution is indeed present in the lattice. Finally, we study the problem of the inadequacy of the use of generic linguistic resources (language models and phonetic lexica) to yield robust and efficient recognition results. In this context, we explore the solution consisting in the integration of dynamic phonetic and language models controlled by an associated dialogue model. In this approach, restricted lexicon and language models dependent on the context of the dialogue are used in place of the complete ones. We first experimentally verify that this approach indeed yields a significant increase in speech recognition performance, and we then focus on the problem of producing, for a given application, the adequate dialogue model that can efficiently integrate the speech recognition module. In this perspective, we propose an enhancement of the used dialogue model prototyping methodology by integrating speech recognition error simulation within the Wizard-of-Oz dialogue simulation. We show that such an approach enables a more complete prototyping of the dialogue model that guarantees a better adequacy of the resulting dialogue model to the targeted vocal application.

EPFL2005

On Modeling the Synergy Between Acoustic and Lexical Information for Pronunciation Lexicon Development

Marzieh Razavi

State-of-the-art automatic speech recognition (ASR) and text-to-speech systems require a pronunciation lexicon that maps each word to a sequence of phones. Manual development of lexicons is costly as it needs linguistic knowledge and human expertise. To facilitate this process, grapheme-to-phone (G2P) conversion approaches are used, in which given a seed lexicon provided by linguistic experts, the G2P relationship is learned by applying statistical techniques. Despite advances in these approaches, there are two challenges remaining: (1) the seed lexicon development through linguistic expertise incorporates limited acoustic information, which may not necessarily cover all natural phonological variations, and (2) the linguistic expertise required for the development of the seed lexicon may not be available for all languages, particularly under-resourced languages. The goal of this thesis is to address these challenges by developing a framework that effectively integrates linguistic information and acoustic data for pronunciation lexicon development. To achieve that goal, we first study the problem of matching a word hypothesis to the acoustic signal, and show that the hidden Markov model-based ASR approach achieves that match via a latent symbol set. Building on that understanding, we develop a data-driven G2P conversion approach in which a probabilistic G2P relationship is learned by matching the acoustic signal with the word hypothesis represented by graphemes, using phones as the latent symbols. Through a theoretical development, we show that this acoustic G2P conversion approach is a particular case of an abstract posterior-based G2P conversion formalism, which requires estimation of phone class conditional probabilities. Through studies on two languages, we show that the acoustic G2P conversion approach yields lexicons that can perform comparable to state-of-the-art G2P conversion methods at the ASR level, despite performing relatively poorly at pronunciation level. We build on the posterior-based formalism to show that different G2P conversion approaches in the literature can be regarded as different estimators of phone class conditional probabilities, and can be combined in a multi-stream fashion to yield better lexicons. We also demonstrate that the multi-stream formulation can be further extended to unify acoustic-to-phone conversion and G2P conversion. We validate the proposed multi-stream formulation on two challenging tasks on English. Finally, we address the issue of developing lexical resources for under-resourced languages by proposing an acoustic subword unit (ASWU)-based lexicon development approach. In this approach, ASWU derivation is cast as the problem of determining a latent symbol space given the word hypothesis and acoustics, and the pronunciations are generated using the proposed acoustic G2P conversion approach. Through experimental studies and analysis on well-resourced and under-resourced languages, we show that the derived ASWUs are "phone-like", and the ASWU-based lexicons yield better ASR systems compared to the alternative grapheme-based lexicons.

EPFL2017

Efficient integration of automated speech recognition in the framework of dialogue-based vocal systems

EPFL2005

On Modeling the Synergy Between Acoustic and Lexical Information for Pronunciation Lexicon Development

Marzieh Razavi

EPFL2017

Subunits Inference and Lexicon Development Based on Pairwise Comparison of Utterances and Signs

Efficient integration of automated speech recognition in the framework of dialogue-based vocal systems

On Modeling the Synergy Between Acoustic and Lexical Information for Pronunciation Lexicon Development

Efficient integration of automated speech recognition in the framework of dialogue-based vocal systems

Subunits Inference and Lexicon Development Based on Pairwise Comparison of Utterances and Signs

On Modeling the Synergy Between Acoustic and Lexical Information for Pronunciation Lexicon Development