Information Services banner Edinburgh Research Archive The University of Edinburgh crest

Edinburgh Research Archive >
Informatics, School of >
Informatics thesis and dissertation collection >

Please use this identifier to cite or link to this item: http://hdl.handle.net/1842/3309

This item has been viewed 29 times in the last year. View Statistics

Files in This Item:

File Description SizeFormat
A Winterboer PhD thesis 2009.pdf1.56 MBAdobe PDFView/Open
Title: Evaluating Information Presentation Strategies for Spoken Dialogue Systems
Authors: Winterboer, Andi
Supervisor(s): Moore, Johanna D.
Issue Date: 2009
Publisher: The University of Edinburgh
Abstract: A common task for spoken dialogue systems (SDS) is to help users select a suitable option (e.g., flight, hotel, restaurant) from the set of options available. When the number of options is small, they can simply be presented sequentially. However, as the number of options increases, the system must have strategies for helping users browse the space of available options. In this thesis, I compare two approaches to information presentation in SDS: (1) the summarize and refine (SR) approach (Polifroni et al., 2003; Polifroni, 2008) in which the summaries are generated by clustering the options based on attributes that lead to the smallest number of clusters, and (2) the user-model based summarize and refine (UMSR) approach (Demberg, 2005; Demberg and Moore, 2006) which employs a user model to cluster options based on attributes that are relevant to the user and uses coherence markers (e.g., connectives, discourse cues, adverbials) to highlight the trade-offs among the presented items. Prior work has shown that users prefer approaches to information presentation that take the user’s preferences into account (e.g., Komatani et al., 2003;Walker et al., 2004; Demberg and Moore, 2006). However, due to the complexity of building a working end-to-end SDS, these studies employed an ”overhearer” evaluation methodology, in which participants read or listened to pre-prepared dialogues, thus limiting evaluation criteria to users’ perceptions (e.g., informativeness, overview of options, and so on). In order to examine whether users prefer presentations based on UMSR when they were actively interacting with a dialogue system, and to measure the effectiveness and efficiency of the two approaches, I compared them in a Wizard-of-Oz experiment. I found that in terms of both task success and dialogue efficiency the UMSR approach was superior to the SR approach. In addition, I found that users also preferred presentations based on UMSR in the interactive mode. SDS are typically developed for situations in which the user’s hands and eyes are busy. I hypothesized that the benefits of pointing out relationships among options (i.e., trade-offs) in information presentation messages outweighs the costs of processing more complex sentences. To test this hypothesis, I performed two dual task experiments comparing the two approaches to information presentation in terms of their effect on cognitive load. Again, participants performed better with presentations based on the UMSR algorithm in terms of both dialogue efficiency and task success, and I found no detrimental effect on performance of the primary task. Finally, I hypothesized that one of the main reasons why UMSR is more efficient is because it uses coherence markers to highlight relations (e.g., trade-offs) between options and attributes. To test this hypothesis, I performed an eye-tracking experiment in which participants read presentations with and without these linguistic devices, and answered evaluation and comparison questions to measure differences in item recall. In addition, I used reading times to examine comprehension differences between the two information presentation strategies. I found that the linguistic devices used in UMSR indeed facilitated item recall, with no penalty in terms of comprehension cost. Thus, in this thesis I showed that an approach to information presentation that employs a user model and uses linguistic devices such as coherence markers to highlight trade-offs among the presented items improves information browsing. User studies demonstrated that this finding also applies to situations where users are performing another demanding task simultaneously.
Description: Institute for Communicating and Collaborative Systems
Keywords: Informatics
Institute for Communicating and Collaborative Systems
URI: http://hdl.handle.net/1842/3309
Appears in Collections:Informatics thesis and dissertation collection

Items in ERA are protected by copyright, with all rights reserved, unless otherwise indicated.

 

Valid XHTML 1.0! Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh 2013, and/or the original authors. Privacy and Cookies Policy