A Study in Responsiveness in Spoken Dialog,

Nigel Ward and Wataru Tsukahara.

International Journal of Human-Computer Studies, 59 (6), pp 959-981, 2003


The future of human-computer interfaces may include systems which are human-like in abilities and behavior. One particularly interesting aspect of human-to-human communication is the ability of some conversation partners to sensitively pick up on the nuances of the other's utterances, as they shift from moment to moment, and to use this information to subtly adjust responses to express interest, supportiveness, sympathy, and the like. This paper reports a model of this ability in the context of a spoken dialog system for a tutoring-like interaction. The system used information about the user's \scare{ephemeral emotions} --- such as confidence, confusion, pleasure, and dependency --- as inferred from the prosody of his utterances and the context, and used this information to select the most appropriate acknowledgement form at each moment. Users rated the system with this ability more highly than a version without, showing that human-like, real-time sensitivity can be of value in interfaces. The paper further discusses ways to discover and quantify such rules of social interaction, using corpus-based-analysis, developer intuitions, and feedback from naive judges; and further proposes `evaluation after re-listening' as a technique for evaluating spoken dialog systems which operate at near-human levels of performance.

preprint pdf