Semi-supervised Adaptation of Assistant Based Speech Recognition Models for different Approach Areas

Air Navigation Service Provider (ANSPs) replace paper flight strips through different digital solutions. The instructed commands from an air traffic controller (ATCOs) are then available in computer readable form. However, those systems require manual controller inputs, i.e. ATCOs’ workload increases. The Active Listening Assistant (AcListant®) project has shown that Assistant Based Speech Recognition (ABSR) is a potential solution to reduce this additional workload. However, the development of an ABSR application for a specific target-domain usually requires a large amount of manually transcribed audio data in order to achieve task- sufficient recognition accuracies. MALORCA project developed an initial basic ABSR system and semi-automatically tailored its recognition models for both Prague and Vienna approach by machine learning from automatically transcribed audio data. Command recognition error rates were reduced from 7.9% to under 0.6% for Prague and from 18.9% to 3.2% for Vienna.

Building Blocks of Assistant Based Speech Recognition for Air Traffic Management Applications

Petr Motlicek, Mittul Singh

In air traffic control rooms around the world paper flight strips are replaced through different digital solutions. This enables other systems to access the instructed air traffic controller (ATCo) commands and use them for other purposes. Digital flight strip solutions, however, require manual input from the ATCo and, therefore, increase the workload. Recently the AcListant® project has validated that Assistant Based Speech Recognition (ABSR, which integrates a speech recognizer with an assistant system) could be a solution to avoid this increase of workload. However, adaptation of ABSR to new environments usually requires a lot of data, time and expertise, which makes the process expensive. The MALORCA project used machine learning (ML) algorithms to provide a generic, cheap and effective approach for adaptation. Therefore, ABSR was divided into conceptual modules that contain generic parts (building blocks) and domain specific models. As first show case ABSR was auto-matically adapted with radar data and voice recordings from Prague and Vienna approach. The fully trained system reaches command recognition rates (RR) of 92% (Prague) resp. 83% (Vienna) and command recognition error rates (ER) of 0.6% (Prague) resp. 3.2% (Vienna). The building blocks and models and their effect on RR and ER are presented in this paper.

SESARJU2018

Adaptation of Assistant Based Speech Recognition to New Domains and Its Acceptance by Air Traffic Controllers

Petr Motlicek, Mittul Singh

In air traffic control rooms, paper flight strips are more and more replaced by digital solutions. The digital systems, however, increase the workload for air traffic controllers: For instance, each voice-command must be manually inserted into the system by the controller. Recently the AcListant® project has validated that Assistant Based Speech Recognition (ABSR) can replace the manual inputs by automatically recognized voice commands. Adaptation of ABSR to different environments, however, has shown to be expensive. The Horizon 2020 funded project MALORCA (MAchine Learning Of Speech Recognition Models for Controller Assistance), proposed a more effective adaptation solution integrating a machine learning framework. As a first showcase, ABSR was automatically adapted with radar data and voice recordings for Prague and Vienna. The system reaches command recognition error rates of 0.6% (Prague) resp. 3.2% (Vienna). This paper describes the feedback trials with controllers from Vienna and Prague.

2019

Spatio-temporal human pose detection

Miodrag Dimitrijevic

This thesis proposes a Chamfer-based method for human body pose detection that combines silhouette matching, motion information, and statistical relevance estimates in an original way. We demonstrate that our method can not only detect people but also recover their full 3D pose when they are seen from different viewpoints and at different scales, when the background is cluttered and background subtraction is impractical because camera moves. We introduce spatio-temporal templates that consist of short sequences of 2D silhouettes obtained from motion capture data. This way, the motion information is inherent in the templates, which is important because human motion is very different from other kinds of motions and can be effectively used to distinguish humans from both static and moving background objects. The templates can handle different camera views, as well as different scales. They are matched against short image sequences. During a training phase, we use statistical learning techniques to estimate and store the relevance of the different silhouette parts to the recognition task. At run-time, we use it to convert Chamfer distances into meaningful probability estimates. For example, for walking motions, this accounts for the fact that feet and shoulders provide much more discriminant information than the trunk. Using the probability estimates makes the recognition algorithm much more discriminating. To demonstrate our approach we chose two types of motion: walking and golf swings. All the walking templates represent the specific part on the walking cycle where the feet are on the ground and the angle between the legs is greatest. The characteristic pose that we have chosen for a golf swing is the beginning of the downswing, when the arms are at the highest position. To further improve the performance of our algorithm we use dynamic programming to link the various detections of walking people to create plausible trajectories. We filter out the detections which are not lying on the recovered trajectory. This way, all detections whose orientation is wrong are eliminated as well as false positives on the background, if any. Finally, we show that the reliable specific 3D poses provided by our approach, allow us to treat the 3D tracking of human motion as an interpolation problem, which unlike traditional tracking approaches is both robust and fully automated.

EPFL2007

Spatio-temporal human pose detection

Miodrag Dimitrijevic

EPFL2007