A Complex Mechatronic System: from Design to Application

Progress in mobile robotics requires the researchers to access and improve all modules that compose the robot, from low-level mechanical components to high-level reasoning systems. This paper presents the development process of the robots built at the Autonomous Systems Lab, EPFL Lausanne, Switzerland. Starting from the mechanical and electrical design up to the application, we show the challenges that needed to be faced as well as the solutions that have been devised. The description covers aspects like the operating system and framework, because of its role in the overall safety and dependability of the whole software system, the research as a precondition for innovative products, and the man-machine interface, which is indispensable for conveying information to the user as well as allowing the user to interact with the robot. The issues that have been faced stem from the hierarchical, layered construction of a complex mechatronic product, where the operation of the machine depends on the smooth cooperation of each layer; In the same way, the overall safety is undermined by the least reliable piece building the system.

Design, Modeling and Optimization of Stochastic Reactive Distributed Robotic Systems

Grégory Mermoud

This dissertation describes a complete methodological framework for designing, modeling and optimizing a specific class of distributed systems whose dynamics result from the multiple, stochastic interactions of their constitutive components. These components can be robots endowed with very minimal capabilities, or even simpler entities such as insects, bacteria, particles, or molecules. We refer to such components as Smart Minimal Particles (SMPs). One of the main difficulties facing the modeling of SMPs is the potential complexity and richness of their dynamics. On the one hand, one needs detailed models that account for the physico-chemical properties of the lower-level components (e.g., shape, material, surface chemistry, charge, etc.), which, in turn, determine the nature and the magnitude of their interactions. On the other hand, one is also interested in models that can yield accurate numerical predictions of macroscopic quantities, and investigate formally their dependence on the system’s design and control parameters. These competing requirements motivate a combination of models at multiple levels of abstraction, as advocated by the Multi-Level Modeling Methodology (MLMM), which was introduced in prior works. The MLMM enables the fulfillment of both requirements in a very efficient way by incrementally building up models at increasing levels of abstraction in order to capture the relevant features of the system. This thesis extends and consolidates the MLMM along several axes. In a first step, we provide a theoretical consolidation of the MLMM. We propose a thorough classification of the different models of SMPs, and we discuss their underlying assumptions and simplifications. We shed light on the fundamental impact of embodiment and spatiality on models’ accuracy, and we define the conditions under which the macro-deterministic approximation is valid. These theoretical considerations are experimentally supported by five case studies of aggregation and Self-Assembly (SA) at different scales. The five case studies utilize three types of components: (i) miniature wheeled robots (Alice, 2 cm in size) endowed with limited computation, sensing, actuation, and communication capabilities, (ii) water-floating passive modules (Lily, 3 cm in size) endowed with four permanent magnets for mutual latching, and (iii) micro-fabricated cylinders (about 100 μm in diameter, studied in realistic simulation only) that can achieve SA in liquids. In a second step, we introduce the core contribution of this thesis, that is, a systematic and generic methodology for bridging the gap between real, physical systems and computationally efficient models at multiple abstraction levels. In particular, we describe the M3 computational framework, which enables the automated construction of models of SMPs. Our approach consists in observing (or simulating realistically) a system of interest, and building a hierarchical suite of models based on the observations (i.e., trajectories) collected during these experiments (or simulations). Internally, the framework first builds up a microscopic representation of the system based on these observations and on a list of interactions of interest specified by the user. This representation, called the Canonical Microscopic Model (CMM), is a formal and generic description of SMPs, and it serves as a blueprint for the construction of a macroscopic model, specified using the Chemical Reaction Network (CRN) formalism. The rates of the CRN are finally calibrated using a Maximum Likelihood Estimation (MLE) scheme. We validate the M3 framework on each of the three platforms discussed earlier, thereby illustrating its relevance both as a modeling and as an analysis tool. Finally, we discuss the role of multi-level modeling when designing and optimizing SMPs. In particular, we show that top-down model-based design of multi-robot systems is generally not amenable to efficient implementations when dealing with very resource-constrained robots. Instead, faithful and computationally efficient models built incrementally from the bottom up prove to be an essential tool for designing such systems. We further corroborate this claim by applying our automated modeling framework to the real-time control of the stochastic SA of Lily modules. Our scientific contribution is therefore three-fold. First, we provide a solid experimental and theoretical consolidation of the MLMM, which has been the subject of intense research efforts for the last decade. Second, we propose, for the first time, an approach to generate models at high abstraction level in a completely automated fashion, based solely on observations of the system of interest. Third, we provide deep insights into the modeling and the design of SMPs, with a specific emphasis on self-assembling systems ranging from the centimeter scale down to the micrometer scale.

EPFL2012

Neuroprosthetics to support impaired hand functions: A multidisciplinary approach combining brain-machine interfaces and wearable exoskeletons

Luca Randazzo

Neuroprosthetics, the discipline that aims at interfacing neural systems to artificially engineered devices, has witnessed in recent years important advancements towards the ultimate goal of augmenting and restoring human functions through technology and artificial systems. Research in this field covers a wide range of efforts, from advancing our basic understanding of the neural processes involved into human sensory, motor and cognitive functions, to developing artificial intelligence algorithms for their decoding, to the physical interfacing between artificial devices and the human nervous system and biomechanics. Ultimately, this discipline aims at fusing these systems in order to restore, replace and rehabilitate lost and impaired functions due to motor disabling conditions or traumatic accidents. In this thesis, I present a multidisciplinary approach that aimed at developing a neuroprosthesis for the assistance and restoration of hand functions impaired by neurological disorders or traumatic accidents, such as cerebrovascular accidents and spinal cord injuries. These efforts encompassed the conceptualization and development of a robotic hand exoskeleton and brain-machine interface (BMI) algorithms for its control. Specifically, this thesis focused on the design, development and testing of (i) a novel mechatronic hand exoskeleton to assist hand motions within domestic and clinical settings, (ii) non-invasive BMI approaches based on electroencephalography (EEG) to decode neural correlates of intended hand movements, and on (iii) the closed-loop integration of the proposed exoskeleton and BMI for the sake of providing continuous feedback about ongoing neural modulations through hand motions and to trigger sensorimotor rehabilitation within clinical scenarios. Results showed that the proposed mechatronic system can successfully control hand opening and closing within a fully wearable, portable and lightweight package. The system was tested with users who suffered from motor disabling impairments, showing that it could help them in performing several activities of daily living for the first time since their accidents. From a brain-machine interfacing perspective, this work shows how imagined hand movements can be decoded, through EEG, in parallel with exoskeleton-induced motions, with important implications for the development of more embodied human-machine interaction protocols. Finally, this work shows how the closed-loop control of exoskeleton motions by means of the decoded ongoing neural activity enhances the discriminability of sensorimotor neural patterns and improves the performance of the brain-machine control channel. Overall, the results presented here represent important advancements within the field of neuroprosthetics, with interesting implications for the development of assistive exoskeletal technologies and non-invasive brain-machine interfaces aimed at controlling such systems in clinical or domestic settings, for both assistive and neurorehabilitative purposes.

EPFL2018

Error handling in multimodal voice-enabled interfaces of tour-guide robots using graphical models

Plamen Prodanov

Mobile service robots are going to play an increasing role in the society of humans. Voice-enabled interaction with service robots becomes very important, if such robots are to be deployed in real-world environments and accepted by the vast majority of potential human users. The research presented in this thesis addresses the problem of speech recognition integration in an interactive voice-enabled interface of a service robot, in particular a tour-guide robot. The task of a tour-guide robot is to engage visitors to mass exhibitions (users) in dialogue providing the services it is designed for (e.g. exhibit presentations) within a limited time. In managing tour-guide dialogues, extracting the user goal (intention) for requesting a particular service at each dialogue state is the key issue. In mass exhibition conditions speech recognition errors are inevitable because of noisy speech and uncooperative users of robots with no prior experience in robotics. They can jeopardize the user goal identification. Wrongly identified user goals can lead to communication failures. Therefore, to reduce the risk of such failures, methods for detecting and compensating for communication failures in human-robot dialogue are needed. During the short-term interaction with visitors, the interpretation of the user goal at each dialogue state can be improved by combining speech recognition in the speech modality with information from other available robot modalities. The methods presented in this thesis exploit probabilistic models for fusing information from speech and auxiliary modalities of the robot for user goal identification and communication failure detection. To compensate for the detected communication failures we investigate multimodal methods for recovery from communication failures. To model the process of modality fusion, taking into account the uncertainties in the information extracted from each input modality during human-robot interaction, we use the probabilistic framework of Bayesian networks. Bayesian networks are graphical models that represent a joint probability function over a set of random variables. They are used to model the dependencies among variables associated with the user goals, modality related events (e.g. the event of user presence that is inferred from the laser scanner modality of the robot), and observed modality features providing evidence in favor of these modality events. Bayesian networks are used to calculate posterior probabilities over the possible user goals at each dialogue state. These probabilities serve as a base in deciding if the user goal is valid, i.e. if it can be mapped into a tour-guide service (e.g. exhibit presentation) or is undefined – signaling a possible communication failure. The Bayesian network can be also used to elicit probabilities over the modality events revealing information about the possible cause for a communication failure. Introducing new user goal aspects (e.g. new modality events and related features) that provide auxiliary information for detecting communication failures makes the design process cumbersome, calling for a systematic approach in the Bayesian network modelling. Generally, introducing new variables for user goal identification in the Bayesian networks can lead to complex and computationally expensive models. In order to make the design process more systematic and modular, we adapt principles from the theory of grounding in human communication. When people communicate, they resolve understanding problems in a collaborative joint effort of providing evidence of common shared knowledge (grounding). We use Bayesian network topologies, tailored to limited computational resources, to model a state-based grounding model fusing information from three different input modalities (laser, video and speech) to infer possible grounding states. These grounding states are associated with modality events showing if the user is present in range for communication, if the user is attending to the interaction, whether the speech modality is reliable, and if the user goal is valid. The state-based grounding model is used to compute probabilities that intermediary grounding states have been reached. This serves as a base for detecting if the the user has reached the final grounding state, or wether a repair dialogue sequence is needed. In the case of a repair dialogue sequence, the tour-guide robot can exploit the multiple available modalities along with speech. For example, if the user has failed to reach the grounding state related to her/his presence in range for communication, the robot can use its move modality to search and attract the attention of the visitors. In the case when speech recognition is detected to be unreliable, the robot can offer the alternative use of the buttons modality in the repair sequence. Given the probability of each grounding state, and the dialogue sequence that can be executed in the next dialogue state, a tour-guide robot has different preferences on the possible dialogue continuation. If the possible dialogue sequences at each dialogue state are defined as actions, the introduced principle of maximum expected utility (MEU) provides an explicit way of action selection, based on the action utility, given the evidence about the user goal at each dialogue state. Decision networks, constructed as graphical models based on Bayesian networks are proposed to perform MEU-based decisions, incorporating the utility of the actions to be chosen at each dialogue state by the tour-guide robot. These action utilities are defined taking into account the tour-guide task requirements. The proposed graphical models for user goal identification and dialogue error handling in human-robot dialogue are evaluated in experiments with multimodal data. These data were collected during the operation of the tour-guide robot RoboX at the Autonomous System Lab of EPFL and at the Swiss National Exhibition in 2002 (Expo.02). The evaluation experiments use component and system level metrics for technical (objective) and user-based (subjective) evaluation. On the component level, the technical evaluation is done by calculating accuracies, as objective measures of the performance of the grounding model, and the resulting performance of the user goal identification in dialogue. The benefit of the proposed error handling framework is demonstrated comparing the accuracy of a baseline interactive system, employing only speech recognition for user goal identification, and a system equipped with multimodal grounding models for error handling.

EPFL2006