Active: a unified platform for building intelligent applications

Computers have become affordable, small, omnipresent and are often connected to the Internet. However, despite the availability of such rich environments, user interfaces have not been adapted to fully leverage their potential. In our view, user interfaces will evolve to become more than simple tools that act using a click-and-act paradigm – people want software that can act as assistants to whom tasks can be delegated. This new type of software will provide us with more user-centric systems, able to interact naturally with human users and with the information environment. Although progress has been made in this direction over the last decade, current research in the field has shown that building intelligent assistants is a complex task that requires expertise in many fields, ranging from artificial intelligence to core software and hardware engineering. The difficulty of deeply integrating all the technologies and AI methodologies required to produce robust intelligent assistants has greatly limited the impact and widespread adoption of this form of software. In this thesis, we propose to design, implement and evaluate a new methodology and associated tool suite aimed to ease and accelerate the development of intelligent assistant software. Active, our solution, introduces the original concept of Active Ontologies, and combines it with a service oriented architecture to serve as the foundation for user-centric applications. The Active software suite features a programming editor, a runtime server and associated management tools. Using this unified platform, we developed techniques for rapidly creating intelligent assistant applications that weave together language processing, process modeling and dynamic service orchestration. To validate our approach, three prototypes have been implemented and evaluated. First, we created an assistant that helps mobile users retrieve data and access online services. The system allows users to fetch information through natural language dialog about restaurants, hotels, points of interest, flights status and weather forecasts. As a second prototype, we have implemented an assistant to help surgeons in the context of the operating room. The system uses a combination of voice and gesture recognition to help surgeons navigate through pre-operative information, visualize and control a live video stream coming from an endoscope mounted on a robotic arm. Lastly, we have created a system that helps organize meetings through emails and instant messages with the organizer and attendees. These three prototypes are deployed in very different application domains, and yet are built with the same tools and methods; this demonstrates the flexibility and versatility of our approach. We conducted a user study to validate our claim that, in specific domains, intelligent assistant software can perform better than conventional software approaches. First, we compared our mobile assistant against Google Mobile™, the leading commercially available search application for mobile users. We asked a population of users to accomplish ten travel-related tasks with both systems. Results show that, for the travel domain, our assistant-based system performs significantly better both in terms of effectiveness and time to completion than the more conventional, keyword-based search engine. In the medical domain, we asked a small population of surgeons to accomplish a set of tasks with our second prototype. After providing us with a positive feedback, surgeons used our prototype to discuss and experiment with innovative forms of user interaction in the context of the operating room. We also performed evaluations to support our claim that our unified approach accelerates the software development of intelligent assistants. First, we gave basic training on the Active platform to a population of software developers, asking them to go through a simple tutorial. Once trained, they were able to successfully program the core component of the mobile assistant prototype in under two hours. As a second validation, the meeting organizing assistant was designed, implemented and tested within three working days – comparable systems published in the literature were the product of significantly more time and resource investment. This demonstrates that a system encompassing numerous AI and software development aspects, including language processing, multiple modalities, contextual dialog, processing logic and backend services integration, can be rapidly implemented and tested using the Active framework. Our work shows that for many task-oriented domains, intelligent assistant applications can be significantly more effective than conventional systems. Through our Active Ontology approach and associated methodologies and tools, we have shown that complex intelligent assistant applications, perhaps for the first time, can be developed with low cost and high reliability. In addition to receiving a positive echo from the research world, our work is also causing ripples in the commercial world.

Modeling Human-Agent Interaction with Active Ontologies

Charles Baur, Didier Guzzoni

As computer systems continue to grow in power and access more networked content and services, we believe there will be an increasing need to provide more user-centric systems that act as intelligent assistants, able to interact naturally with human users and with the information environment. Building such systems is a difficult task that requires expertise in many AI fields, ranging from reasoning, planning, scheduling, natural language and multimodal user interfaces. In contrast to many approaches to building agent assistants where diverse AI components are stitched together at a surface level, we propose an approach, called ”Active Ontologies”, and a toolset, called ”Active”, where a developer can model all aspects of an intelligent assistant: ontology-based knowledge structures, servicebased primitive actions, composite processes and procedures, and natural language and dialog structures. We demonstrate this approach through an example prototype of an intelligent meeting scheduling assistant that communicates using instant messages and emails.

2007

Active, a Tool for Building Intelligent User Interfaces

Charles Baur, Didier Guzzoni

Computers have become affordable, small, omnipresent and are often connected to the Internet. However, despite the availability of such rich environment, user interfaces have not been adapted to fully leverage its potential. To help with complex tasks, a new type of software is needed to provide more user-centric systems that act as ”intelligent assistants”, able to interact naturally with human users and with the information environment. Building an intelligent assistant is a difficult task that requires expertise in many ﬁelds ranging from artificial intelligence to core software and hardware engineering. We believe that providing a unified tool and methodology to create intelligent software will bring many benefits to this area of research. Our solution, the Active framework, combines an innovative production rule engine with communities of services to model and implement intelligent assistants. In the medical ﬁeld, our approach is used to build an operating room assistant. Using natural modalities such as speech recognition and hand gestures, it enables surgeons to interact with computer based equipments of the operating room as if they were ac tive members of the team. In a broader context, Active aims to ease the development of intelligent software by making required technologies more accessible.

ACTA Press2007