Analyzing Tutorial Documents

CHI 2000 Workshop on Natural Language Interfaces

The Hague, The Netherlands, April 3, 2000

Analyzing Tutorial Documents
in Terms of Dialogue Acts

Saïd Tazi

LIHS
University Toulouse 1
31042 Toulouse Cedex, France
Tel +33 (0) 5 61 63 35 64

Abstract

Learning the use of a system can be made by a tutorial. We consider a tutorial as a part of the documentation set furnished by the seller. It is built following specific procedures to achieve pedagogical goals. The communication aspect between author of a tutorial and the learners should be taken in account since the start of the tutorial design. The evolution of the system implies changes on the tutorial as a part of the documentation. A co-evolutionary design method of a system, and its documentation necessitates to take in account links that relate the system to its documentation. Links can be analyzed in terms of virtual dialogues between users and designers via the system and between users and authors of the documentation. The main idea is to simulate dialogues between users and the system at time of use (and learning) and dialogues between authors and users at time of design and development to have a framework that leads to improve communication and co-evolutionary design of the system and its documentation. The act-function-phase (AFP) framework provides a means of relating what happens between users and the system, to what happens between authors and the user in terms of dialogue acts.

Introduction

A tutorial is a part of the documentation furnished by the seller, to learn the basic concepts to use the system. This type of document include technical, methodological and pedagogical knowledge. These different aspects of tutorial knowledge are not always considered prominently. This can be explained by the complexity of the writing process. Effectively, writing is a complex process by which authors want to deliver their communicative intentions through what they put down on a paper or a computer (Hayes and Flower1998). This is worse in the case of tutorials of a systems. The author have to deal with the definitions of concepts, with the system operational procedures by taking in account of pedagogical considerations. The authors face a number of human-factors issues, including how to describe the constraints, systems and interfaces clearly, how to specify safe, effective and understandable operating procedures, and how to communicate these procedures to the users in a clear and simple manner.

What is very difficult to deal with is the evolution of the system that implies changes and modifications of the documentation. So a co-evolutionary framework is necessary to help designers and authors to ensure efficiency and coherence of the both the system and its documentation. A more direct approach would be to enable co-evolution (Fisher et al., 1995) of systems, operating procedures and documentation, where changes in one part would be readily propagated to the others.

The main idea of the research presented in this paper is to consider the AFP framework to analyze the different communicative goals of technical documentation. By considering in the whole the design of the system and its documentation, we want to enhance the communicative aspectof this kind of document. This paper considers particularly tutorials that is included in the technical documentation and experiments had been on Macromedia Director tutorial.

In the next section I will present briefly the AFP framework. I show how this framework could be used to analyze particularly a tutorial and how this analyze provide hints to improve the communication between authors and readers. We are experimenting these ideas in the environment of Macromedia Director™, as the framework of applications design. And lastly I will open a discussion that I hope will contribute to advancement of putting together people from HCI and Natural Language Processing.

The Act-Function-Phase framework

The AFP framework has been applied to the context of the development of flight crew operating manual for a commercial aircraft, (Novick and Tazi 1998). In this position paper, the AFP framework is considered in the context of development of a tutorial to learn a multimedia system to demonstrate how we can take advantage from it to enhance documentation and particularly here tutorials. We Consider the AFP framework as a new theory of documentation development to multimedia systems. Technically, AFP represents and relates differences in (1) the virtual dialogues among the users and system designers to (2) the virtual dialogues between the author(s) of the user’s manual and the users. Thus framework connects the dialogue acts in the context of use to the dialogue acts in the context of design. Specifically, the model accounts for the "domain" dialogue acts of doing things and the "meta" dialogue acts of communicating, across the contexts of prescription (i.e., operating procedures) and description (i.e., systems, functions and constraints). We claim that this model leads to useful insights about how to develop operating procedures and manuals—and even the underlying systems—in a co-evolutionary way. In a practical sense, the model serves as a kind of lens through which to view current and proposed manuals.

Figure 1: Dialogue , virtual dialogue and actions

The AFP framework is a theory that takes its resources from results and experiences from NLP pragmatics, especially speech act theory (Searl 1996), and from studies in human factors for the design of interfaces and documentation. The first project that have inspired the creation of the AFP model aimed to enhance the embarked documentation on commercial aircraft. (Novick and Tazi 98). Let first define the main concepts of the framework.

Dialogue Acts

A key extension to speech-act theory has been the development of "meta" models that treat a wider spectrum of communicative acts, particularly those dealing with control of the conversation (Carbonell, 1982; Novick, 1988). These meta-act models use the notion of act to account for communication about communication, such as resolution of reference or turn-taking. Indeed, a conversation can be viewed as a multi-layered composition (Novick, 1988) of conversation acts (Traum & Hinkelman, 1992), that contain both the domain-acts that accomplish things in the world of the parties' nominal goals and the meta-acts that accomplish things in the sphere of the communication itself. Another way of distinguishing domain from meta acts is to distinguish between what the person wants to achieve or how they want to achieve this.

It turns out that the concepts of acts, both domain and meta, can be applied not only to spoken interaction but to written materials as well (Tazi & Novick, 1998). An author, no less than a speaker, will have some set of effects that he or she wishes to produce in the intended recipient. Typically these causes and effects are along the lines of wanting the text to cause the reader to know or to believe something. Possible intended causes and effects could also include wanting the reader to do something in the future, such as adhere to a standard operating procedure during an activity. Consequently, we use the term dialogue acts to mean a generalization of the notion of speech acts to include the complex of acts associated with communicative action, whether spoken or written.

Written materials can be formalized within speech act-theory—despite the fact that this theory was mainly used in natural language dialogue studies—by assuming that (1) the act of writing is a kind of simplified dialogue between the author and his or her audience via the document, and (2) the act of reading is an other simplified dialogue between the reader and the author also via the document. While these activities might be seen as hardly constituting dialogues at all, they can be seen also as asymmetrical complements that rely on each other for a "shadow" partner in a temporally disjoint exchange. In acts of both writing and reading there is no immediate reaction from the author or from the reader. Authors construct their document as if they are speaking to their audience (future readers); they assume that the audience could (and would) react in the case of a dialogue, so they create implicit hypotheses about these reactions and write on the basis of the hypotheses. Readers also, in reading, build the knowledge they read as if they were addressed by the author in a dialogue; the act of reading is like a series of queries from the reader to the author. Written "speech" acts, just like oral ones, can cause intended effects in the world. In the case of multimedia manuals, for example, users execute operating procedures as an intended effect of the acts taken by the authors.

The dialogue-act model can also be applied to interaction with computer interfaces generally. A full treatment of this aspect is beyond the scope of this paper. We note here, however, that a symbolic act can be translated into a set of actions in an interface, regardless of whether those actions are expressed in spoken language or some other language of action suited to, for example, graphical user interfaces. And it is the case of the documentation of the Macromedia Director™.

Functions

The second component consists of the task functions intended to be achieved through the system and its documentation. Broadly speaking, parts of the tutorial present information about the system; these can be viewed as constituting a function of description. Complementary parts of the tutorial present action-oriented material such as operating procedures that explain how to do to fix some parameters depending on the platform; these can be viewed as constituting the manual's function of prescription. Other parts of the tutorial present pedagogical aspects as detailed explanation, or reference to the what could explain or illustrate a concept. This third aspect is new in the AFP framework. Exercises found in the tutorial are an example of pedagogical functions of the tutorial.

Phases

The third component consists of the contexts of use. Viewing the development and use of the documentation as interactive processes suggests that there are actually two distinct phases, with two corresponding kinds of use:

A dialogue between system and its users, specified by the system designer. We call this the operational phase.

A dialogue between the documentation and the users, created by the documentation author. We call this the referential phase.

The operational phase depends on the designer of the system while the referential phase depends in the case of tutorials, on the pedagogical skills of the authors.

Examples of Acts

From the study we made on the tutorial we give here some examples of domain and meta acts playing referential, operational and pedagogical functions. In the operating procedures context, we can identify a variety of domain acts that involve communication from the user to the system and from the system to the user. There are both meta and domains acts.

Domain

Meta

Acts that involve communication from the user to the system

-Create a new movie

-Open File

-Change a movie settings

-Choose Web216 from the default palette pop-up menu.

-Save a movie as Shockwave format.

-Choose bit-map display fixes

-Choose screen color display

-Choose dimensions of a window

(*) We notice here that these acts depend on the operating system and hardware characteristics.

Acts that involve communication from the system to the user

-Advise

-Execute a script

-Warn

-Scrolling windows

-Default choice of color

Table 1: Operational dialogues

In the referential dialogue phase, all acts are from author to reader, because the tutorial is a printed and thus static, non-interactive document. Pedagogical f

Domain

Meta

Referential acts

All prescriptive function acts basically fall into this category.

- Inform

- Do

-Title

-Enumerate

-Type Face

Pedagogical acts

This category is in fact a particular type of referential acts

Some of descriptive function acts fall into this category as:

-Explain

New acts are considered as:

-Make an exercise

-Choose an other medium to illustrate, as a video clip for example

Table 2: Referential dialogues

How the AFP framework should help to enhance the learning process ?

Experiments

In a classical course of multimedia systems at the university of Toulouse 1, we have used exclusively the tutorial distributed by Macromedia Director™ furnished by the seller. The students noticed the relations between what is written in this documentation and what they must do as actions to perform leaning of the concepts and techniques of the system.

They encountered many difficulties to learn by themselves the system from the tutorial. These difficulties are due in one hand to translation mistakes found in the tutorial, and in an other hand to the amalgam of what we called descriptive and prescriptive functions. The difficulty to learn such a system is that the student must learn the concepts and execute operations in the same time. The students do not have enough time to understand definitions of new concepts because they must manipulate procedures specified in the tutorial. Several operations are difficult to execute because the student has to know first the definition of the concept. Some explanations are presented as if they was operations to execute. Some concepts are referred to without being defined before. Some difficulties can be interpreted in our theory that the description acts and prescriptive ones are intermixed and presented as meta acts in the same manner. We noticed that the documentation does not distinguish between description context from prescriptive one.

References

Carbonell, J. G. (1982). Meta-language utterances in purposive discourse, Technical report CMU-CS-82-185, Department of Computer Science, Carnegie-Mellon University.

Fischer, G., Redmiles, D., Williams, L., Puhr, G., Aoki, A., and Nakakoji, K. (1995). Beyond object-oriented technology: Where current approaches fall short. Human-Computer Interaction, 10(1), 79-119.

Hayes, J. R., and Flower, L. S. (1980). Identifying the organisation of writing processes. In L. Gregg, and E. R. Steinberg (Eds.), Cognitive processes in writing. Hillsdale, NJ: Erlbaum, 17-35.

Novick, D. (1988). Control of mixed-initiative discourse through meta-locutionary acts: A computational model. Doctoral dissertation, available as Technical Report CIS-TR-88-18, Department of Computer and Information Science, University of Oregon.

Novick, D., and Tazi, S., (1998). Flight crew operating manuals as dialogue: The act-function-phase model, Proceedings of HCI-Aero'98, Montreal, May, 1998, 179-184.

Searle, J. (1969). Speech acts. Cambridge: Cambridge University Press.

Tazi, S., and Novick, D. Actes de discours de la communication écrite, Proceedings of Ergonomie et Informatique Avancée (Ergo-IA 98), Biarritz, FR, Novembre, 1998.

Traum, D., and Hinkelman, E. (1992). Conversation acts in task-oriented spoken dialogue, Computational Intelligence, 8(3), 575-599.