Dialog Systems Usability

As a priority-setting exercise, we compared interactions between users and a simple spoken dialog system to interactions between users and a human operator. We observed usability events, places in which system behavior differed from human behavior, and for each we noted the impact, root causes, and prospects for improvement. We suggest some priority issues for research, involving not only such core areas as speech recognition and synthesis and language understanding and generation, but also less-studied topics such as adaptive or flexible time-outs, turn-taking and speaking rate.

The results are given in Root Causes of Lost Time and User Stress in a Simple Dialog System. Nigel Ward, Anais Rivera, Karen Ward, and David Novick. Interspeech 2005.

Full detail is given in the expanded version: Some Usability Issues and Research Priorities in Spoken Dialog Applications. Nigel Ward, Anais Rivera, Karen Ward, and David Novick. Technical Report UTEP-CS-05-23.

To guide observations we developed a Usability Event Checklist.

Observers followed the Dialog Issue (Usability Event) Labelers Guide.

Three Sample Dialogs: 1-S-H.au, 2-H-H.au, 4-H-H.au

Three initial fragments of the above: Fragment 1, Fragment 2, Fragment 3; Dialog States Exercise

Other-domain example: Persuasive dialog fragment

This material is based upon work supported by the National Science Foundation under Grant no. 045150. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Other themes in dialog systems usability

Up to Nigel's Home Page