Book
A Wizard-of-Oz System Evaluation Study.
| Title: | A Wizard-of-Oz System Evaluation Study. |
|---|---|
| Authors: | Carbonell, Jaime G., Siekmann, Jörg, Matoušek, Václav, Mautner, Pavel, Hajdinjak, Melita, Mihelič, France |
| Source: | Text, Speech & Dialogue (9783540746270); 2007, p532-539, 8p |
| Abstract: | In order to evaluate the performance of the dialogue-manager component of a developing, Slovenian and Croatian spoken dialogue system, two Wizard-of-Oz experiments were performed. The only difference between the two experiment settings was in the dialogue-management manner, i.e., while in the first experiment dialogue management was performed by a human, the wizard, in the second experiment it was performed by the newly-implemented dialogue-manager component. The data from both Wizard-of-Oz experiments was evaluated with the PARADISE evaluation framework, a potential general methodology for evaluating and comparing different versions of spoken-language dialogue systems. The study ascertains a remarkable difference in the performance functions when taking different satisfaction-measure sums or even individual scores as the target to be predicted, it proves the indispensableness of the recently introduced database parameters when evaluating information-providing dialogue systems, and it confirms the dialogue manager's cooperativity subject to the incorporated knowledge representation. [ABSTRACT FROM AUTHOR] |
| Copyright of Text, Speech & Dialogue (9783540746270) is the property of Springer eBooks and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| DOI: | 10.1007/978-3-540-74628-7_69 |
| Database: | Complementary Index |
Availability