Audio-based Relative Positioning System for Multiple Micro Air Vehicle Systems

Employing a group of independently controlled flying micro air vehicles (MAVs) for aerial coverage missions, instead of a single flying robot, increases the robustness and efficiency of the missions. Designing a group of MAVs requires addressing new challenges, such as inter-robot collision avoidance and formation control, where individual's knowledge about the relative location of their local group members is essential. A relative positioning system for a MAV needs to satisfy severe constraints in terms of size, weight, processing power, power consumption, three-dimensional coverage and price. In this paper we present an on-board audio based system that is capable of providing individuals with relative positioning information of their neighbouring sound emitting MAVs. We propose a method based on coherence testing among signals of a small onboard microphone array to obtain relative bearing measurements, and a particle filter estimator to fuse these measurements with information about the motion of robots throughout time to obtain the desired relative location estimates. A method based on fractional Fourier transform (FrFT) is used to identify and extract sounds of simultaneous chirping robots in the neighbourhood. Furthermore, we evaluate our proposed method in a real world experiment with three simultaneously flying micro air vehicles.

Audio-based Positioning and Target Localization for Swarms of Micro Aerial Vehicles

Meysam Basiri

Swarms of autonomous MAVs show great potential in many diverse applications. In a search and rescue mission, a group of MAVs could quickly reach disaster areas by flying over obstacles, cluttered and inaccessible terrains, and work in parallel to detect and locate people that are in need of help. However, multiple challenges still remain in the design of truly autonomous MAV teams. Due to the strict constraints imposed on the MAVs in terms of size, weight, 3D coverage, processing power, power consumption and price, there are not many technological possibilities that could provide individuals with information abut the position of themselves and their team mates without the aid of external systems. Inspired by the sense of hearing among many animal groups which use sound for localization purposes, we propose an on-board audio-based system for allowing individuals in an MAV swarm to use sound waves for obtaining the relative positioning, and furthermore, the self-localization information. We show that not only such a system fully satisfies the constraints of MAVs and is capable of obtaining these information, but also provides additional important opportunities, such as the detection and localization of crucial acoustic targets in the environment. Operating during night time, through foliage and in adverse weather conditions such as fog, dust and smoke, and detection and collision avoidance with non-cooperative noise-emitting aerial platforms are some of the potential advantages of audio-based swarming MAVs. In this thesis, we firstly describe an on-board sound-source localization system and novel methods for the innovative idea of localizing emergency sound sources on the ground from airborne MAVs. The ability of MAVs to locate source of distress sound signals, such as the sound of an emergency whistle blown by a person in need of help or the sound of a personal alarm, is significantly important and would allow fast localization of victims and rescuers during night time and in fog, dust, smoke and dense forests. We propose three different methods for overcoming the ambiguities related with localizing emergency sound sources or, in general, narrowband sound sources. Furthermore, we present the on-board audio-based relative positioning system for obtaining information about the position of other MAVs in their vicinity. We initially describe a passive method that exploits only the engine sound of other robots, in the absence of the self-engine noise, to measure their relative directions. We then extend this method to overcome some of its limitations, where individuals generate a chirping sound similar to the sound of birds to assist others in obtaining their positions. We then describe an estimator based on particle filters that fuses the relative bearing measurements with information about the motion of the robots, provided by their onboard sensors, to also obtain an estimate about the relative range of the robots. Finally, we present a cooperative method to address the self-localization problem for a team of MAVs, while accommodating the motion constraints of flying robots, where individuals obtain their three dimensional positions through perceiving a sound-emitting beacon MAV. All methods are based on coherence testing among signals of a small on-board microphone array, to measure the probable direction of incoming sound waves, and estimators for robust estimation of the desired information throughout time.

EPFL2015

Error handling in multimodal voice-enabled interfaces of tour-guide robots using graphical models

Plamen Prodanov

Mobile service robots are going to play an increasing role in the society of humans. Voice-enabled interaction with service robots becomes very important, if such robots are to be deployed in real-world environments and accepted by the vast majority of potential human users. The research presented in this thesis addresses the problem of speech recognition integration in an interactive voice-enabled interface of a service robot, in particular a tour-guide robot. The task of a tour-guide robot is to engage visitors to mass exhibitions (users) in dialogue providing the services it is designed for (e.g. exhibit presentations) within a limited time. In managing tour-guide dialogues, extracting the user goal (intention) for requesting a particular service at each dialogue state is the key issue. In mass exhibition conditions speech recognition errors are inevitable because of noisy speech and uncooperative users of robots with no prior experience in robotics. They can jeopardize the user goal identification. Wrongly identified user goals can lead to communication failures. Therefore, to reduce the risk of such failures, methods for detecting and compensating for communication failures in human-robot dialogue are needed. During the short-term interaction with visitors, the interpretation of the user goal at each dialogue state can be improved by combining speech recognition in the speech modality with information from other available robot modalities. The methods presented in this thesis exploit probabilistic models for fusing information from speech and auxiliary modalities of the robot for user goal identification and communication failure detection. To compensate for the detected communication failures we investigate multimodal methods for recovery from communication failures. To model the process of modality fusion, taking into account the uncertainties in the information extracted from each input modality during human-robot interaction, we use the probabilistic framework of Bayesian networks. Bayesian networks are graphical models that represent a joint probability function over a set of random variables. They are used to model the dependencies among variables associated with the user goals, modality related events (e.g. the event of user presence that is inferred from the laser scanner modality of the robot), and observed modality features providing evidence in favor of these modality events. Bayesian networks are used to calculate posterior probabilities over the possible user goals at each dialogue state. These probabilities serve as a base in deciding if the user goal is valid, i.e. if it can be mapped into a tour-guide service (e.g. exhibit presentation) or is undefined – signaling a possible communication failure. The Bayesian network can be also used to elicit probabilities over the modality events revealing information about the possible cause for a communication failure. Introducing new user goal aspects (e.g. new modality events and related features) that provide auxiliary information for detecting communication failures makes the design process cumbersome, calling for a systematic approach in the Bayesian network modelling. Generally, introducing new variables for user goal identification in the Bayesian networks can lead to complex and computationally expensive models. In order to make the design process more systematic and modular, we adapt principles from the theory of grounding in human communication. When people communicate, they resolve understanding problems in a collaborative joint effort of providing evidence of common shared knowledge (grounding). We use Bayesian network topologies, tailored to limited computational resources, to model a state-based grounding model fusing information from three different input modalities (laser, video and speech) to infer possible grounding states. These grounding states are associated with modality events showing if the user is present in range for communication, if the user is attending to the interaction, whether the speech modality is reliable, and if the user goal is valid. The state-based grounding model is used to compute probabilities that intermediary grounding states have been reached. This serves as a base for detecting if the the user has reached the final grounding state, or wether a repair dialogue sequence is needed. In the case of a repair dialogue sequence, the tour-guide robot can exploit the multiple available modalities along with speech. For example, if the user has failed to reach the grounding state related to her/his presence in range for communication, the robot can use its move modality to search and attract the attention of the visitors. In the case when speech recognition is detected to be unreliable, the robot can offer the alternative use of the buttons modality in the repair sequence. Given the probability of each grounding state, and the dialogue sequence that can be executed in the next dialogue state, a tour-guide robot has different preferences on the possible dialogue continuation. If the possible dialogue sequences at each dialogue state are defined as actions, the introduced principle of maximum expected utility (MEU) provides an explicit way of action selection, based on the action utility, given the evidence about the user goal at each dialogue state. Decision networks, constructed as graphical models based on Bayesian networks are proposed to perform MEU-based decisions, incorporating the utility of the actions to be chosen at each dialogue state by the tour-guide robot. These action utilities are defined taking into account the tour-guide task requirements. The proposed graphical models for user goal identification and dialogue error handling in human-robot dialogue are evaluated in experiments with multimodal data. These data were collected during the operation of the tour-guide robot RoboX at the Autonomous System Lab of EPFL and at the Swiss National Exhibition in 2002 (Expo.02). The evaluation experiments use component and system level metrics for technical (objective) and user-based (subjective) evaluation. On the component level, the technical evaluation is done by calculating accuracies, as objective measures of the performance of the grounding model, and the resulting performance of the user goal identification in dialogue. The benefit of the proposed error handling framework is demonstrated comparing the accuracy of a baseline interactive system, employing only speech recognition for user goal identification, and a system equipped with multimodal grounding models for error handling.

EPFL2006

Enactive robot vision

Mototaka Suzuki

The complexity of today's autonomous robots poses a major challenge for Artificial Intelligence. These robots are equipped with sophisticated sensors and mechanical abilities that allow them to enter our homes and interact with humans. For example, today's robots are almost all equipped with vision and several of them can move over rough terrain with wheels or legs. The methods developed so far in Artificial Intelligence, however, are not yet ready to cope with the complexity of the information gathered through the robot sensors and the need for rapid action in partially unknown and dynamic environments. In this thesis, I will argue that the apparent complexity of the environment and of the robot brain can be significantly simplified if perception, behavior, and learning are allowed to co-develop on the same time scale. In doing so, robots become sensitive to, and actively exploit, characteristics of the environment that they can tackle within their own computational and physical constraints. This line of work is grounded on philosophical and psychological research showing that perception is an active process mediated by behavior. However, computational models of active vision are very rare and often rely on architectures that are preprogrammed to detect certain characteristics of the environment. Previous work have shown that complex visual tasks, such as position and size invariant shape recognition as well as navigation, can be tackled with remarkably simple neural architectures generated by a coevolutionary process of active vision and feature selection. Behavioral machines equipped with primitive vision systems and direct pathways between visual and motor neurons were evolved while freely interacting with their environments. I proceed further on this line of investigation and describe the application of this methodology in three situations, namely car driving with an omnidirectional camera, goal-oriented navigation of a humanoid robot, and cooperative tasks by two agents. I will show that these systems develop sensitivity to a number of oriented, retinotopic, visual features – oriented edges, corners, height – and a behavioral repertoire to locate, bring, and keep these features in sensitive regions of the vision system that allow them to accomplish their goals. In a second set of experiments, I will show that active vision can be exploited by the robot to perform anticipatory exploration of the environment in a task that requires landmark-based navigation. Evolved robots exploit an internal expectation system that makes use of active exploration to check for expected events in the environment. I will describe a third set of experiments where, in addition to an evolutionary process, the visual system of the robot can develop receptive fields by means of unsupervised Hebbian learning and show that these receptive fields are significantly affected by the behavior of the system and differ from those predicted by most computational models of visual cortex. Finally, I will show that these robots replicate the performance deficiencies observed in experiments of motor deprivation with kitten when they are exposed to the same type of motor deprivations. Furthermore, the analyses of our robot brains suggest an explanation for the deficiencies observed in kitten that have not yet been fully understood.

EPFL2007