Topics in Educational Cyber-Physical Labs

Wissam Halimi
EPFL, 2018
Thèse EPFL

Résumé

Recent advances in remote sensing and actuation technologies, coupled with the large reach of the internet, allowed for the emergence of applications such as cyber-physical labs. Cyber-physical labs are the digital and remotely-accessible equivalent of the lab equipment students use in school to experiment, through web-based interfaces such as web applications. Students, teachers and lab owners derive value from these systems, they are our stakeholders. Students are the intended users, teachers are the educational content curators and lab owners are the service providers. In this thesis, we take a close look at issues pertaining to cyber-physical labs and propose new approaches to address them. We also analyze the use of such systems in a MOOC, to detect the impact of the exherted experimental behavior of students on their academic performance. First, we tackle the case of the generation of web apps interfacing cyber-physical labs. It is the equivalent of preparing experiments for teachers by arranging the equipment for multiple experiments with the same equipment. We propose an extension to the Smart Device Specification for cyber-phyiscal labs, and a tool which generates these apps based on it. The automatically generated apps implement the necessary functions to use a cyber-physical lab, and are ready to be integrated in online learning platfroms. Next, we investigate issues related to the collection and retrieval of students' generated data through their interaction with cyber-physical labs. We consider the needs of students and lab owners. Through questionnaires sent to both parties, we elicit the requirements for an activity-tracking infrastructure composed of a vocabulary and an architectural model. The proposed vocabulary ensures deriving value from the recorded activity, and the proposed architecture addresses privacy and data access issues pertaining to students and lab owners respectively. We evaluate our proposal with two example cyber-physical labs. Last, we collect the interaction data with a cyber-physical lab used in a MOOC. We devise computational analyses on the students activity statistics, in search for indicators of academic performance. We find that high and low performing students show statistically different activity statistics. Then, we sequence the steps students did in an experiment, and don't find any statistically significant patterns for low and high-performing students. This analysis provides insights on the usage of installed facilities to service a potential massive access to limited resources (lab installations), and shed light on possible indicators for academic performance.

Official source

https://infoscience.epfl.ch/record/256537?ln=fr

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Concepts associés

Chargement

Publications associées

Chargement

Wissam Halimi
EPFL, 2018
Thèse EPFL

Résumé

Official source

https://infoscience.epfl.ch/record/256537?ln=fr

À propos de ce résultat

Concepts associés (14)

Concepts associés

Chargement

Publications associées (11)

Publications associées

Chargement

Mobile service robots are going to play an increasing role in the society of humans. Voice-enabled interaction with service robots becomes very important, if such robots are to be deployed in real-world environments and accepted by the vast majority of potential human users. The research presented in this thesis addresses the problem of speech recognition integration in an interactive voice-enabled interface of a service robot, in particular a tour-guide robot. The task of a tour-guide robot is to engage visitors to mass exhibitions (users) in dialogue providing the services it is designed for (e.g. exhibit presentations) within a limited time. In managing tour-guide dialogues, extracting the user goal (intention) for requesting a particular service at each dialogue state is the key issue. In mass exhibition conditions speech recognition errors are inevitable because of noisy speech and uncooperative users of robots with no prior experience in robotics. They can jeopardize the user goal identification. Wrongly identified user goals can lead to communication failures. Therefore, to reduce the risk of such failures, methods for detecting and compensating for communication failures in human-robot dialogue are needed. During the short-term interaction with visitors, the interpretation of the user goal at each dialogue state can be improved by combining speech recognition in the speech modality with information from other available robot modalities. The methods presented in this thesis exploit probabilistic models for fusing information from speech and auxiliary modalities of the robot for user goal identification and communication failure detection. To compensate for the detected communication failures we investigate multimodal methods for recovery from communication failures. To model the process of modality fusion, taking into account the uncertainties in the information extracted from each input modality during human-robot interaction, we use the probabilistic framework of Bayesian networks. Bayesian networks are graphical models that represent a joint probability function over a set of random variables. They are used to model the dependencies among variables associated with the user goals, modality related events (e.g. the event of user presence that is inferred from the laser scanner modality of the robot), and observed modality features providing evidence in favor of these modality events. Bayesian networks are used to calculate posterior probabilities over the possible user goals at each dialogue state. These probabilities serve as a base in deciding if the user goal is valid, i.e. if it can be mapped into a tour-guide service (e.g. exhibit presentation) or is undefined – signaling a possible communication failure. The Bayesian network can be also used to elicit probabilities over the modality events revealing information about the possible cause for a communication failure. Introducing new user goal aspects (e.g. new modality events and related features) that provide auxiliary information for detecting communication failures makes the design process cumbersome, calling for a systematic approach in the Bayesian network modelling. Generally, introducing new variables for user goal identification in the Bayesian networks can lead to complex and computationally expensive models. In order to make the design process more systematic and modular, we adapt principles from the theory of grounding in human communication. When people communicate, they resolve understanding problems in a collaborative joint effort of providing evidence of common shared knowledge (grounding). We use Bayesian network topologies, tailored to limited computational resources, to model a state-based grounding model fusing information from three different input modalities (laser, video and speech) to infer possible grounding states. These grounding states are associated with modality events showing if the user is present in range for communication, if the user is attending to the interaction, whether the speech modality is reliable, and if the user goal is valid. The state-based grounding model is used to compute probabilities that intermediary grounding states have been reached. This serves as a base for detecting if the the user has reached the final grounding state, or wether a repair dialogue sequence is needed. In the case of a repair dialogue sequence, the tour-guide robot can exploit the multiple available modalities along with speech. For example, if the user has failed to reach the grounding state related to her/his presence in range for communication, the robot can use its move modality to search and attract the attention of the visitors. In the case when speech recognition is detected to be unreliable, the robot can offer the alternative use of the buttons modality in the repair sequence. Given the probability of each grounding state, and the dialogue sequence that can be executed in the next dialogue state, a tour-guide robot has different preferences on the possible dialogue continuation. If the possible dialogue sequences at each dialogue state are defined as actions, the introduced principle of maximum expected utility (MEU) provides an explicit way of action selection, based on the action utility, given the evidence about the user goal at each dialogue state. Decision networks, constructed as graphical models based on Bayesian networks are proposed to perform MEU-based decisions, incorporating the utility of the actions to be chosen at each dialogue state by the tour-guide robot. These action utilities are defined taking into account the tour-guide task requirements. The proposed graphical models for user goal identification and dialogue error handling in human-robot dialogue are evaluated in experiments with multimodal data. These data were collected during the operation of the tour-guide robot RoboX at the Autonomous System Lab of EPFL and at the Swiss National Exhibition in 2002 (Expo.02). The evaluation experiments use component and system level metrics for technical (objective) and user-based (subjective) evaluation. On the component level, the technical evaluation is done by calculating accuracies, as objective measures of the performance of the grounding model, and the resulting performance of the user goal identification in dialogue. The benefit of the proposed error handling framework is demonstrated comparing the accuracy of a baseline interactive system, employing only speech recognition for user goal identification, and a system equipped with multimodal grounding models for error handling.

EPFL2006

Chargement

Exploitation de connaissances sémantiques externes dans les représentations vectorielles en recherche documentaire

Florian Seydoux

The work presented in this thesis deals with several problems met in information retrieval (IR), task which one can summarise as identifying, in a collection of "documents", a subset of documents carrying a sought information, i.e.. relevant for a request expressed by a user. In the case of textual documents, to which we limited ourselves within the framework of this thesis, a significant part of the difficulty lies in ambiguity inherent to human languages. The interaction with the user is also approached in our work, by studying a tool enabling a natural language access to a database. Finally, some techniques which permit the visualisation of large collections of documents are also presented. In this document we first of all describe the principal models of IR by highlighting the relations which exist with some manual technics of IR and document retrieval, developed during the past centuries. We present the principle of document indexing, allowing us to represent documents in a multidimensional space, and the use of this representation by a vectorial model. After having reviewed the principal improvements made these last years with vectorial research systems, including the preprocessings of collections, the indexing mechanism and measurements of similarities between documents, we detail some recent usecases of additional semantic resources (semantic dictionaries, thesaurus, networks, ontologies) reported in scientific literature for the indexing task. We then present more in detail the semantic indexing principle of textual documents by using a thesaurus, consisting in integrating in the document's representation space at least part of the informational contents of hierarchical semantic resources. We propose a general framework allowing us to describe and position various possible techniques to carry out the semantic indexing by adapting, if possible, the specificity of the descriptions resulting from the semantic resources to the data to be represented. We use this framework to describe three families of criteria usable for semantic indexing, each one having its own characteristics. For each of these families, we give the specific algorithms allowing the computation of the criteria. The first two families allow us to consider several criteria already known in feature selection. Moreover we show that, unfortunately, many of these criteria are in fact not very effective for the considered task. The third family allows us to introduce a completely new criterion, the Minimum Redundancy Cut criterion (MRC), built on the basis of the information theory and allowing us to obtain index terms having a probability of occurrence in the collection of documents as well balanced as possible. Finally, we treat the case of semantic index independent of the data (statically choosen), allowing a parameterisation of the level of generality of the index terms. Some of the criteria suggested for semantic indexing has been empirically evaluated. To judge their relevance, we used a well known vectorial system (the Smart IR system) and measured the performances of IR obtained with various reference collections. Those collections was indexed on the basis of the studied criterion, by taking into account the strongly structuring semantic relation of hyper/hyponymy ("is-a" relation), given by two different semantic resources. By comparing results obtained with the performances of a traditional indexing (using the lemmas of the words as representation space), we can show on one hand the relevance of the semantic indexings (in RD) and on the other hand the quality of the proposed criterion (MRC). Concerning man-machine interaction, we present a general outline allowing to build in a relatively fast and systematic way systems with mixed initiative, giving the human user a large (and natural) latitude in the control of the dialogue. This outline is usable in typical database research-task applications (where the database is hidden to the user, but the latter knows exactly which information they wish to find) as well as advice-task applications, for which the users does not necessarily have a precise idea of their needs, and uses the system not only for specifing their wishes, but also a set of propositions as a final result. We particularly stress the techniques allowing us to obtain a robust system, able to deal with speech recognizer failures. Concerning the visualisation of large textual data collections, we present an application of the correspondences analysis (allowing to highlight similarities and oppositions for various groups of entity, built on the basis of additional features present in the DB) to the case of patents data. In addition, we propose a method (based on the bootstrap replication principle) allowing us to determine a confidence interval for relative positionings of various groups, thus permit to immediately judge the reliability of the visually apparent similarities or oppositions.

EPFL2006

Chargement

M/ORIS - Medical / Operating Room Interaction System

Sébastien Grange

The routine use of Computer-Assisted Surgery (CAS) in a wide range of surgical procedures and specialties is being hindered by the lack of suitable user interface for surgical environments. While many computer techniques and software exist to assist surgeons in their tasks, the inability of these tools to fit into existing surgical setups and procedures (both physically and practically) significantly reduces their efficiency and limits their deployment. There are two major drawbacks to existing user interfaces for CAS: (1) they consist mostly of poor adaptations of the existing desktop computer keyboard and mouse interaction paradigm, and (2) they are designed to function just like desktop applications, where the software is the focus of attention of the user, therefore adding to the per-operative workload of the surgeon. I propose a new approach to user interfaces for CAS by introducing M/ORIS, a Medical/ Operating Room Interaction System. M/ORIS is an intelligent collection of sensors and logic modules that are designed to specifically address the need for suitable user interaction in the operating room (OR). M/ORIS provides two layers of interaction that address the fundamental limitations of existing surgical user interfaces: (1) a set of new interaction modalities that are compatible with surgical constraints (i.e. sterile and non-obstructing) and (2) a new interaction paradigm, based on these new modalities, that automates some interaction tasks in order to reduce the interruptions to the surgical process currently required to interact with the computer in the OR. This work describes a detailed analysis of Human-Computer Interaction methods for CAS, and proposes a new approach to Surgeon-Computer Interaction. The design of the new interaction paradigm, as well as the implementation of its building modules, are presented, including a new method to perform real-time, visual detection and tracking of obstructed body parts in indoors environments and a mechanism to perform surgical activity monitoring. The performance of each component of M/ORIS and the behavior of the whole system are then analyzed, and considerations for future improvements are given.

EPFL2007

Chargement

Exploitation de connaissances sémantiques externes dans les représentations vectorielles en recherche documentaire

Florian Seydoux

EPFL2006

M/ORIS - Medical / Operating Room Interaction System

Sébastien Grange

EPFL2007