User Interface Design in a Just-in-time Retrieval System for Meetings

The Automatic Content Linking Device (ACLD) is a just-in-time multimedia retrieval system that monitors and supports the conversation among a small group of people within a meeting. The ACLD retrieves from a repository, at regular intervals, information that might be relevant to the group's activity, and presents it through a graphical user interface (GUI). The repository contains documents from past meetings such as slides or reports along with processed meeting recordings; in parallel, Web searches are run as well. The acceptance by users of such a system depends considerably on the GUI, along with the performance of retrieval. The trade-off between informativeness and unobtrusiveness is studied here through the design of a series of GUIs. The requirements and feedback collected while demonstrating the successive versions show that users vary considerably in their preferences for a given style of interface. After studying two extreme options, a widget vs. a wide-screen UI, we conclude that a modular UI, which can be flexibly structured and resized by users, is the most sensible design for a just-in-time multimedia retrieval system.

Design and evaluation issues for user-centric online product search

Jiyong Zhang

Nowadays more and more people are looking for products online, and a massive amount of products are being sold through e-commerce systems. It is crucial to develop effective online product search tools to assist users to find their desired products and to make sound purchase decisions. Currently, most existing online product search tools are not very effective in helping users because they ignore the fact that users only have limited knowledge and computational capacity to process the product information. For example, a search tool may ask users to fill in a form with too many detailed questions, and the search results may either be too minimal or too vast to consider. Such system-centric designs of online product search tools may cause some serious problems to end-users. Most of the time users are unable to state all their preferences at one time, so the search results may not be very accurate. In addition, users can either be impatient to view too much product information, or feel lost when no product appears in the search results during the interaction process. User-centric online product search tools can be developed to solve these problems and to help users make buying decisions effectively. The search tool should have the ability to recommend suitable products to meet the user's various preferences. In addition, it should help the user navigate the product space and reach the final target product without too much effort. Furthermore, according to behavior decision theory, users are likely to construct their preferences during the decision process, so the tool should be designed in an interactive way to elicit users' preferences gradually. Moreover, it should be decision supportive for users to make accurate purchasing decisions even if they don't have detail domain knowledge of the specific products. To develop effective user-centric online product search tools, one important task is to evaluate their performance so that system designers can obtain prompt feedback. Another crucial task is to design new algorithms and new user interfaces of the tools so that they can help users find the desired products more efficiently. In this thesis, we first consider the evaluation issue by developing a simulation environment to analyze the performance of generic product search tools. Compared to earlier evaluation methods that are mainly based on real-user studies, this simulation environment is faster and less expensive. Then we implement the CritiqueShop system, an online product search tool based on the well-known critiquing technique with two aspects of novelties: a user-centric compound critiquing generation algorithm which generates search results efficiently, and a visual user interface for enhancing user's satisfaction degree. Both the algorithm and the user interface are validated by large-scale comparative real-user studies. Moreover, the collaborative filtering approach is widely used to help people find low-risk products in domains such as movies or books. Here we further propose a recursive collaborative filtering approach that is able to generate search results more accurately without requiring additional effort from the users.

EPFL2008

Answering questions about archived, annotated meetings

Marita Ailomaa

Retrieving information from archived meetings is a new domain of information retrieval that has received increasing attention in the past few years. Search in spontaneous spoken conversations has been recognized as more difficult than text-based document retrieval because meeting discussions contain two levels of information: the content itself, i.e. what topics are discussed, but also the argumentation process, i.e. what conflicts are resolved and what decisions are made. To capture the richness of information in meetings, current research focuses on recording meetings in Smart-Rooms, transcribing meeting discussion into text and annotating discussion with semantic higher-level structures to allow for efficient access to the data. However, it is not yet clear what type of user interface is best suited for searching and browsing such archived, annotated meetings. Content-based retrieval with keyword search is too naive and does not take into account the semantic annotations on the data. The objective of this thesis is to assess the feasibility and usefulness of a natural language interface to meeting archives that allows users to ask complex questions about meetings and retrieve episodes of meeting discussions based on semantic annotations. The particular issues that we address are: the need of argumentative annotation to answer questions about meetings; the linguistic and domain-specific natural language understanding techniques required to interpret such questions; and the use of visual overviews of meeting annotations to guide users in formulating questions. To meet the outlined objectives, we have annotated meetings with argumentative structure and built a prototype of a natural language understanding engine that interprets questions based on those annotations. Further, we have performed two sets of user experiments to study what questions users ask when faced with a natural language interface to annotated meeting archives. For this, we used a simulation method called Wizard of Oz, to enable users to express questions in their own terms without being influenced by limitations in speech recognition technology. Our experimental results show that technically it is feasible to annotate meetings and implement a deep-linguistic NLU engine for questions about meetings, but in practice users do not consistently take advantage of these features. Instead they often search for keywords in meetings. When visual overviews of the available annotations are provided, users refer to those annotations in their questions, but the complexity of questions remains simple. Users search with a breadth-first approach, asking questions in sequence instead of a single complex question. We conclude that natural language interfaces to meeting archives are useful, but that more experimental work is needed to find ways to incent users to take advantage of the expressive power of natural language when asking questions about meetings.

EPFL2009

Designable Visual Markers for Mobile Human-Computer Interaction

Enrico Costanza

Visual markers are graphic symbols designed to be easily recognised by machines. They are traditionally used to track goods, but there is increasing interest in their application to mobile human-computer interaction (HCI). By scanning a visual marker through a camera phone, users can retrieve localised information and access mobile services. In particular the dissertation examines the application of visual markers to physical tagging: practices of association of digital information with physical items. One missed opportunity in current visual marker systems is that the markers themselves cannot be visually designed; they are not expressive to humans, and thus fail to convey information before being scanned. To address this limitation, this dissertation introduces the idea of designable markers, visual markers that are both machine-readable and visually communicative to humans, and presents an investigation of the ways in which they can support mobile human-computer interaction. The application of designable visual markers to the creation of mobile interfaces is explored through a variety of methods: through formal usability experiments, through the creation and analysis of example designs, as well as through the qualitative analysis of two field trials. All three approaches were enabled by the engineering and development of d-touch, an actual recognition system that supports designable visual markers and by its integration in a variety of applications and experimental probes. D-touch is based on image topology, and its markers are defined in terms of constraints on the nesting of dark and light regions. The constraints imposed by d-touch are flexible enough to allow novice users to create markers which are visually expressive and at the same time machine readable. A user study demonstrates how such system enables people to design their own functional visual markers, determining their aesthetic qualities and what they visually communicate to others. A desktop application to support users in the creation of valid markers, the d-touch analyser, is presented and its usefulness is demonstrated through the same study. A formal usability experiment comparing five variations of marker-based interfaces on keypad and touch-screen phones shows that all of them allow users to reliably select targets within, on average, less than 4 seconds. Participants of the experiment reported a strong preference for interfaces that involve only marker scanning, compared to those that require a combination of marker scanning and key-presses or touch selections. Example designs of mobile interface generated by the author as well as others are presented to expose how the d-touch recognition system can be integrated in mobile applications. The examples illustrate a variety of ways in which markers can be used to augment printed materials such as cards, books and product packages, adding to them interactive capabilities. The examples show also different approaches to marker design, ranging from simple and recognisable iconic design, to symbols that integrate cues about the interactive functionality, to making them invisible by hiding them in existing graphics. Finally, the dissertation reports and analyses two field trials conducted to study what practices of physical tagging can emerge from, and be supported by, the use of markers. The trials were centred around the use of uWiki, a functional prototype based on d-touch, that allows users to associate digital content to markers printed on physical tags that can be affixed to objects or buildings. Observations show that a variety of practices emerge around the use of this technology, indicating that they provide a rich medium that has potential to attract the interest of real users. Though the results of this work are preliminary, they serve to demonstrate the range of potential for the future of such systems.

EPFL2010