My research focuses on interactive systems, and especially (a) development methods for interfaces and their documentation and (b) models of conversation. The underlying question common to these areas is “What does it mean to interact?” With this knowledge, developers could build systems that are more intuitive, more useful, and safer, in domains ranging from consumer kiosks to modern aircraft.
The main sections of this page explain:
If you’re a student thinking about working with me on research, an M.S., or a Ph.D., here’s some information about what sort of careers this would get you, what you’d learn, and the kinds of things on which you’d get to work.
Careers: A degree in HCI, particularly focusing on usability, development methods or spoken-language interfaces, enables graduates to enter a range of interesting—and, of course, well-paying—careers. Some of my former students work in academia as researchers or teachers, and most others work in industry. One growing area for HCI careers is in developing and managing the user experience, particularly for Web sites. You can get an idea of the kinds of jobs currently available by looking at the Human-Computer Interaction Resource Network’s job bank for North America, the Usability Professionals’ Association job bank, and the International Speech Communications Association’s international job listings. The home pages of HCI-related professional organizations provide a broader look at what people in our field do. Interesting sites include ACM SIGCHI (computer-human interaction), ACM SIGDOC (design of communication), the Usability Professionals’ Association, Content Management Professionals, and the International Speech Communications Association.
Learning: Most of my students learn skills both for research and for building great user interfaces and Web sites. If their focus is spoken-language systems, they learn a great deal about how humans talk to each other and how to represent this process computationally. Whatever the specific focus, most students find the work challenging, as doing effective research means learning new ways of reading, writing and thinking. Students usually work with me one-on-one and also participate in the Interactive Systems Group. The kinds of things students are learning are reflected in some of the recent publications my students co-authored with me:
Research activities for students: Students typically develop a research question related to one of my major research areas, such as usability, development methods, and dialogue models. Depending on their particular topic, they may do an observational study of people using computers, a laboratory study of people’s perceptions of interaction, or implement a prototype system. Most recently, students have interviewed people about their frustrations with computers, observed people at work using computers, and conducted a study contrasting the way speakers of Arabic, American English and Mexican Spanish perceive animated characters in conversation. Another student is developed a system for on-the-fly writing of plays by conversational agents.
Research projects that I am currently conducting or supervising include:
Overall, my research efforts have addressed four aspects of the meaning of interaction: (a) design of communication, (b) interface development methodologies, (c) spoken-language dialogue models, and (d) mediated communications.
My most recent research has centered on issues related to design of communication. This generally involves applying insights from conversation and dialogue modeling to problems such as the authoring of documentation, developing operating procedures for safety-critical applications such as aviation, and the use of a text editor. An example of my work in this area is
Novick, D., and Ward, K. (2003). An interaction initiative model for documentation, Proceedings of SIGDOC 2003, San Francisco, CA, October 11-15, 2003, 80-85.
This paper, jointly authored with Karen Ward, proposed a model of creation and use of documentation based on the concept of mixed-initiative interaction. In our model, successful single-initiative interaction is characterized by grounding of contributions, and successful mixed-initiative interaction is characterized by both grounding and agreement. Just as in spoken conversation, achievement of actual agreement depends on the intentions of both parties; agreement is achieved when the reader follows the documentation’s instructions. In fact, readers are not obligated to—and often do not—act according to the author’s intentions. By making these dynamics explicit, the model can aid authors in developing effective documentation.
The research related to documentation and procedures for aviation was conducted at EURISCO in the COHERE project sponsored by Aerospatiale Airbus. This project, which ran from 1996 to 1999, had three principal activities, all related to the improvement of cockpit procedures and their documentation for future Airbus aircraft. The first activity was a comparative analysis of flight crew operating manuals (FCOMs) for existing Airbus aircraft. The second activity involved methodological evolution of the development processes for cockpit procedures and their documentation. The third activity involved developing ways of improving the authoring and publication of FCOMs, particularly with respect to internal consistency. The acronym COHERE stands for Common Object-oriented Hypermedia Editing and Revision Environment. Key results of this project include:
In a later paper, my students and I, building on work by Michel Beaudouin-Lafon, showed how a text editor could be extended beyond the WIMP interface to incorporate conversation-like functions through new approaches to direct-manipulation interaction. More recently, a project, conducted with help from student Brian Lowe, explored ways to generate both the textual and graphical components of documentation from a single source, so that the text and graphics are reliably consistent. Brian earned his B.S. in Computer Science in spring, 2005. Results were published as Novick, D., and Lowe, B. (2005). Co-generation of text and graphics, Proceedings of SIGDOC 2005, Coventry, UK, September 21-23, 2005, 6-11.
This area of research could be described generally as computational pragmatics. This work involves modeling and simulating conversations, especially with respect to conversational control. The first known simulation of conversation by computational agents was reported by Power (1979). In this work, a pair of agents named John and Mary conversed in simple English about opening a door and moving between rooms; conversational turn-taking was controlled by the system in which the agents were implemented.
In Novick (1988, 1991), two agents conversed using speech acts, reproducing human conversations in a laboratory-based "letter sequence" task. The main contribution of this work was to show that speech-act models enabled the agents to control turn-taking as a "meta" part of the conversation. This is the first known agent-based reproduction of real human-human conversations. Subsequently, Traum and Hinkelman (1992), both students of James Allen at Rochester, also simulated conversations with agents using what they called "conversation acts." These conversations were based on human conversations in a laboratory-based "trains" domain. An example of my work in this area is
Novick, D., and Ward, K.
(1993). Mutual beliefs of multiple conversants: A computational model of collaboration in air
traffic control. Proceedings of AAAI'93,
This paper, authored jointly with Karen Ward, extended both the technology and the theory of control in simulated conversations. In particular, it is significant for being the first simulation to replicate conversation in a non-laboratory, "real-world" domain, and for producing conversation acts that are the result of rules that take account of the mutuality of conversational beliefs. In a later paper (Novick, Hansen & Ward 1996), we showed that physical acts of changing gaze could produce realistic turn-taking behavior in a two-person dialogue.
More recently, I have been working with colleagues at USC’s Institute for Creative Technologies to produce representations of proxemics and gaze that are salient across the cultures of Levantine Arabic, Mexican Spanish, and American English. This work was done with doctoral student David Herrera.
This aspect of my research adapts and extends user interface development methodologies to new technologies, including spoken-language systems, aircraft cockpit systems and, especially, documentation. The development of both spoken language systems and operating procedures for aircraft have traditionally relied on waterfall-style models or extended hand-crafting. How could this process be speeded up and improved? Gould and Lewis (1985) suggested that usability could be produced by early focus on users and tasks, empirical measurement, and iterative design. Novick and Douglas (1992) adapted and extended Gould and Lewis's method for use with rapid prototyping systems for user interfaces.
An example of my work in this area is
Hansen, B., Novick, D., and Sutton, S. (1996). Systematic design of spoken prompts, Conference on Human Factors in Computing Systems (CHI'96), Vancouver, BC, April, 1996, 157-164.
Designers of system prompts for interactive spoken-language systems typically seek 1) to constrain users so that they say things that the system can understand accurately and 2) to produce "natural" interaction that maximizes users' satisfaction. Unfortunately, these goals are often at odds. In this paper, we presented a set of heuristics for choosing appropriate prompt styles and show that a set of dimensions can be formulated from these heuristics. A point (or region) in the space formed by these dimensions is a "style" for prompts. We developed and applied metrics for empirically testing different prompt styles.
In Novick, D., and Chater, M (1999), we showed how the methodology of development of interfaces can be adapted to development of operating procedures. In particular, the cognitive walkthrough can be adapted to account for steps and resources outside the computer's part of the system interface. Empirical evaluation suggests that a cognitive walkthrough for operating procedures (CW-OP) is reasonably efficient and can provide useful information for developers. A practical guide to applying the CW-OP was written for developers. Another paper in this area is Novick and Perez-Quiñones (1998). In this paper, jointly authored with Manuel A. Perez-Quiñones, we point out that for the design of computer interfaces, interaction can be represented as situated acts that abstract up from situated action (cf., Suchman, 1987). The situated-act representation specifies interaction independent of interface modalities, enabling specialization of interfaces based into particular modalities. We illustrate the approach through interfaces in two domains: an aircraft navigation system and a multimodal VR viewer.
The fourth area of research examines the effects of technology on interaction. The goals of this research include improving the design of mediating systems to enhance conversational effectiveness and, indeed, to enable mediating systems to improve upon face-to-face interaction. A number of computer-based systems had been developed that were intended to facilitate dyadic and multiparty interaction. These included structured messaging systems (e.g, Bowers & Churcher, 1988), computer-supported meetings (e.g., Mantei, 1988; Stefik, Foster, Bobrow, Lanning & Suchman, 1987), and multimedia communications systems such as Cruiser (Root, 1988) and the TeleCollaboration Project (Bulick, Abel, Corey, Schmidt & Coffin, 1989). However, the effectiveness of mediated communications systems was in doubt (Egido, 1988). A number of researchers conducted empirical examinations of effectiveness and efficiency of different modes of interaction (e.g., Chapanis, Ochsman, Parrish & Weeks, 1977; Oviatt & Cohen, 1991).
An example of my work in this area is
Novick, D. G., and Walpole, J. (1990). Enhancing the efficiency of multiparty communication through computer mediation, Interacting with Computers, 2(2), 227-246.
This paper, authored with Jonathan Walpole, who studies distributed systems, addressed the question of whether computer mediation could produce interaction more efficient than face-to-face conversation. In particular, the paper applied a model of conversational control to multiparty interaction, producing a new theory and form of mediated interaction. This work was the subject of a special commentary by Boyle (1990). Extensions of this work have been used in developing usability testing for Intel Corporation's ProShare software. These extensions involve understanding the relations of control acts to collaborative tasks.
Another paper in this area is (Marshall & Novick, 1995). In this paper, jointly authored with Catherine R. Marshall, we showed that situational factors, such as the type of task being performed, have a significant consequence for the effect of modality on conversational effectiveness. We were able to show that mediated communication can be as effective as face-to-face communication for such a task if the mediating system supports physical co-presence by providing (1) both an audio and a video channel, (2) remote camera control so that each party can decide what to see in the other's environment and (3) a split video display that allows each party to see what the other is seeing. In contrast, for a decision task in which linguistic and cultural co-presence provided the basis for relevant mutual knowledge, mediated communication with audio+video had no advantage over audio-only.
More recently, my student Annette Arrigucci looked at the comparative usability of mixed synthesized and recorded speech. Her M.S. thesis tested whether usability problems of spoken-language systems that mix synthetic and recorded system speech output are attributable to lack of user experience with synthesized speech. Annette earned her M.S. at UTEP in 2005.
Bowers, J., and Churcher, J.
(1988). Local and global structuring of computer mediated communication:
Developing linguistic perspective on CSCW in COSMOS, Proceedings of CSCW-88,
Boyle, T. (1990), Further discussion on increasing the efficiency of multiparty interaction, Interacting with Computers 2(2) 247-252.
Bulick, S., Abel, M., Corey, D., Schmidt, J. and Coffin, S. (1989). The U S WEST Advanced Technologies prototype multimedia system, IEEE Globecom '89, Dallas, TX.
Chapanis, A., Ochsman, R., Parrish R., and Weeks, G. (1977). Studies in interactive communication. II: The effects of four communication modes on the linguistic performance of teams during cooperative problem solving, Human Factors, 19, 487-509.
Clark, H. H., and Marshall, C. R. (1981). Definite reference and mutual knowledge. In A K. Joshi, B.
L. Webber, I. A. Sag (Eds.), Elements of discourse understanding (pp.
Clark, H. H., and Schaefer, E. F. (1987). Collaborating on contributions to conversations. Language and Cognitive Processes, 2, 19-41.
Cohen, P. (1984). The pragmatics of referring and the modality of communication. Computational Linguistics, 10(2), 97-146.
Cole, R., Novick, D., Vermeulen, P.J.E., Sutton, S., Fanty, M., Wessels, L., de Villiers, J., Schalkwyk, J., Hansen, B., and Burnett, D. (1997). Experiments with a spoken dialogue system for taking the U.S. census, Free Speech Journal, 1(3), February 25, 1997.
Egido, C. (1988). Videoconferencing as a
technology to support group work: A review of its failure, Proceedings of
Gould, J., and Lewis, C. (1985). Designing for usability: Key principles and what designers think. CACM, 28(3), 300-311.
Grosz, B. J., and Sidner, C. L. (1986). Attention, intentions, and the structure of discourse. Computational Linguistics, 12, 175-204.
Mantei, M. (1988). Capturing the Capture Lab concept: A case study in the design of computer-supported meeting environments, Proceedings of CSCW-88, Portland, OR, 257-270.
Marshall, C. R., and Novick, D. G. (1995). Conversational effectiveness in multimedia communications, Information Technology & People, 8(1), 54-79.
Novick, D. G. (1988). Proceedings of Control of mixed-initiative discourse through meta-locutionary acts: A computational model. Technical Report CIS-TR-88-18, Department of Computer and Information Science, University of Oregon.
Novick, D. (1991). Controlling interaction with meta-acts (refereed poster). Conference on Human Factors in Computing Systems (CHI'91), New Orleans, LA, May, 1991.
Oviatt, S. L., and Cohen, P. R. (1991). Discourse structure and performance efficiency in interactive and noninteractive spoken modalities. Computer Speech and Language, 5(4), 297.
Power, R. (1979). The organization of purposeful dialogues. Linguistics, 17, 107-153.
Root, R. (1988). Design of a multi-media vehicle for
social browsing. Proceedings of CSCW-88,
Stefik, M., Foster, G., Bobrow, D., Kahn, K., Lanning, S., and Suchman, L. (1987). CACM, 30, 23-27.
Suchman, L. (1987). Plans and situated actions.
Traum, D., and Hinkelman, E. (1992). Conversation acts in task-oriented spoken dialogue. Computational Intelligence, 8(3).