Responding to User Emotional State by Adding Emotional Coloring to Utterances

Interspeech 2009.

Jaime C. Acosta, Nigel G. Ward
Department of Computer Science, University of Texas at El Paso

Abstract: When people speak to each other, they share a rich set of nonverbal behaviors such as varying prosody in voice. These behaviors, sometimes interpreted as demonstrations of emotions, call for appropriate responses, but today's spoken dialog systems lack the ability to do so. We collected a corpus of persuasive dialogs, specifically conversations about graduate school between a staff member and students, and had judges label all utterances with triples indicating the perceived emotions, using the three dimensions activation, evaluation, and power. We found immediate response patterns, in which the staff member colored her utterances in response to the emotion shown by the student in the immediately previous utterance, and built a predictive model suitable for use in a dialog system to persuasively discuss graduate school with students.

Full Paper


Nigel Ward's Publications