CHI 2000 Workshop on Natural Language
Interfaces
Tony
Lambie
University College, London
Carroll (1997) writes, “…the emergence of HCI is ongoing. Perhaps the most impressive current feature of the area is its fragmentation”. So the problem of a fractured and discontinuous research community is more widespread than merely among NL workers. There may well be different factors bearing on the research in the wider community of HCI work, factors different from those affecting members of the world of NL interaction. However, it seems likely that these factors which have the widest impact can only serve to exaggerate any divisive forces native to language design. It is, therefore, worth examining what these general factors may be, looking at the divisions within the NL community, and commenting on any common denominators before looking at the means by which those differences, apparently peculiar to the NL community, can be resolved or the conflicting parties reconciled.
There are two notable issues which have created rifts in the CHI/HCI community: firstly, the question of whether design should rest on academic psychology (science) or be independent – famously associated with Carroll & Campbell versus Newell & Card (Carroll & Campbell (1989) – and secondly (and relatedly), Whiteside & Wixon’s (referred to in Carroll & Campbell’s paper) stress on the importance of ‘in vivo’ rather than ‘in vitro’ studies, which is continuous with some of the concerns of sociolinguistics and poses what it regards as some of the inevitable difficulties of practicing such ‘in vivo’ studies.
Most researchers have ignored these debates and carried on with work which resulted in practical improvements or served as steps in the right direction. HCI has, for the most part, been concerned with the specification of design problems with particular requirements. Some of the design solutions have, as a matter of fact, invoked metaphorical reference to actual things or abstract descriptions of cognitive behaviour, e.g., the ‘desktop’ in the PC interface and the concept of usability to characterise the desirable quality of good interface designs. In general, however, the goal of the design process has been to satisfy a fairly clear set of plausible requirements. Now, the goals of NL design seem to bring with them the necessity of mimicking cognitive behaviour. The ultimate aim is, indeed, to provide easier interfaces which tax the user less, but the intermediate and unavoidable aim is to fulfill the goal of creating something which resembles natural cognitive behaviour. This aim raises two problems reflecting the “notable issues” above: a) how we are going to know what the properties of NL systems should be – and this raises in an explicit form the question, How can we do this without some knowledge of the science of linguistics or sociolinguistics? And b) a spectre similar to that of ‘strong AI’ appears on the horizon, since the ability to manufacture NL systems appears to presuppose a reductive view of human cognitive behaviour and this raises the corresponding objection that we can only judge human cognition in a context. I shall address these questions in what follows.
The CHI papers which serve as the background to the workshop appear to follow empirical or experimental routes, gathering data on heuristics or investigating features of actual dialogue, and identifying or attempting to satisfy some fairly well defined set of requirements. Their interests may be diverse, e.g., some address multi-modal interaction; one, evaluation by comparison; one, examination of the procedure from design to delivery. But there does not seem to be an explicit difference of a more fundamental or material kind, such as those alluded in the last section. Most of the papers do not address the big picture but get ahead with the business of design. Karat et al, at CHI 99, make a statement about the limits of NL design and they write, “While having no difference between human-human and human-computer communication might be a laudable goal, it is not one likely to be attainable in the near future. Context aids human understanding in ways that are not possible with machines (though there are ongoing efforts to provide machines with broad contextual and social knowledge).” Cassell et al (CHI 99) make loose but, I think, identifiable reference to the sociolinguistic tradition, by mentioning “conversational invitation”, “turn-taking” and “feedback”. However, one of the sets of authors, Cunningham et al, in the book on hypertext (in which the Landauer et al’s paper is to be found), does expose a radically different approach in an explicit manner.
Here, then, is the ‘locus classicus’ of conflict within the NL community, though the field of interest is hypertext. The ideas the authors expose are, of course, interesting ones. The difficulty is that it is fairly clear that they make less headway when their work is compared with that of Hammond, in the same book, which is facing the same sort of challenge. Like them, Hammond stresses the difficulty which the particularity of the application (‘situatedness’) poses, but attempts to embrace this elusive feature of design by employing experimental notions of ‘encoding specificity’ and ‘encoding variability’ to delineate the relevant features of the target situation of the design which should enhance the learning of the user. His aim, in addition, is to increase the application’s effectiveness, while Cunningham et al, who draw on a different less ‘scientific’ (better, perhaps, ‘scientistic’) tradition and who refer to, amongst others, Suchman, Winograd & Flores and Heidegger, appear to withdraw from the constraints of design commitment. So, after exposing their rich background of ideas, they consider a system called Intermedia, but express reluctance to assess its worth; and they conclude their paper writing, “The results of our study support our belief that learning is a constructive process that occurs personally but within a social domain”. Given the stated scepticism of the value of an evaluation of Intermedia, this conclusion is difficult to confirm.
While the former author, Hammond, is looking for a model which will allow him to make the transition from a scientific generalisation to a specific instantiation and is prepared to operationalise the ideas on which he draws, the authors of the latter paper believe in a constructivist epistemological framework in which the user contributes to what is known in a way which is inseparable from the object of knowledge. This inseparability seems to introduce an indeterminacy which is counterproductive of systematic or explicit design; and only such design can be discriminated, for or against.
The temptation, for people who take the view that we cannot but interfere (because of the above ‘inseparability’ of subjective and objective knowledge), for example, when collecting data as a consequence of the uncertainty of the investigator’s influence (perhaps the cognitive correlate of the ubiquitous influence of the context), is a feature of the sociolinguistic tradition. It may lead to the belief that there can be no explicit and secure method of cognitive design: that if you wish to pursue this kind of enquiry you must ‘apprentice’ yourself, in a sense, and discover your prejudices in order to minimise their effect. The business of design then becomes more of an art than a science; or at least this appears to be the danger. In effect, the constructivist approach (by emphasising the essential uncertainty of the factors which contribute to natural language and therefore to NL design) refuses to accept the possibility of the reductive view, which is thought to impoverish the image of human cognition.
This view of knowledge of the mental sphere might be said to mirror the uncertainty principle of the physical sphere, in the sense that the view introduces an unavoidable approximation to any design specification in the same way as there is an unavoidable approximation to ascertaining either the velocity or the position of a subatomic particle. However, could it not be that the profound kind of uncertainty which rests on, for example, distributed cognition, situated behaviour etc. may be negligible in practice as it appears to be in physics (if one considers the increasingly plausible prospect of quantum computing, quantum ‘ratcheting’ and so on)? However, if the community of designers in Human Factors, or Cognitive Ergonomics more generally, feel that the perspective may be undermining, it is partly because the extent of the uncertainty is not clearly limited. Is the indeterminacy, then, all-pervasive or can one manage the problem with some sort of ‘workaround’ solution?
To answer this question demands that we step back to consider what design problems are and how they are distinguished from what is often thought of as the paradigm of rigour – pure science. What if the obstacle to managing or designing cognition might be associated with the project of understanding, which is primarily the aim of science but not with that of engineering or systematic design? In quantum physics the problem of uncertainty and the rebarbative nature of the joint wave/particle picture of subatomic behaviour is that it is difficult to picture or grasp as the way things happen to be, because it contradicts our ordinary experience, or appears to be just plain contradictory (one particle going through two holes simultaneously). If this difficulty is one of understanding and yet, as a matter of fact, design in the field of quantum physical behaviour, operating on those very assumptions which are hard to grasp, is able to go forward, then, surely, the NL community can accommodate puzzling linguistic or cognitive phenomena and still exploit cognitive behaviour. Perhaps the dissolution of our puzzlement will instead follow our ability to manipulate both the physical and the mental universe: the engineering solution could precede the scientific one.
It is, of course, conventionally thought that technological or engineering achievements are based on scientific success. However, this picture is at least a simplification of the truth, e.g., steam technology before thermodynamics and the complicated relationship between pure science and technology which is evidenced by the development of the transistor (referred to by Carroll, 1995). If the separation of science and technology, including systematic design or engineering is the case, generally, the question again arises, in connection with cognitive ergonomics or HCI, how, given the need to mimic language we can do without what we know of language of a rigorous and systematic nature (embodied, say, in linguistics) – a position which seems to compromise the independent status of systematic cognitive design, giving precedence again to science.
How then, in the case of NL design, do we avoid the paradox of asserting the independence of systematic design or engineering from science, and, at the same time, acknowledge that we need descriptive/explanatory (scientific) knowledge as a basis for the project of matching the design product (NL system) to the natural phenomenon (human language)? If we took Speech Act theory as a ‘stripped down’ version of our ‘scientific’ base (Searle (1969) himself would characterise speech acts as ‘constitutive’, and perhaps, I would contend, this could describe a version neutral as between science and engineering), could it then be adapted to fit the design purpose? One way would be by introducing planning to the bare ‘constitutive’ speech acts, and something like this (though not with the same intention) was attempted in the late 1970s by researchers such as Power (1979) and Cohen & Perrault (1979). Planning would then be the component which contributed the intentional (or, more generally, goal-oriented) element, and would also bring with it a feature which any model of intentional cognitive behaviour needs – a correspondence to a hierarchy of task – a domain of application.
One important difference between the two parties – Power and Cohen & Perrault – is the manner in which planning is inserted into the design framework. For Cohen & Perrrault, planning and intention, narrowly interpreted, are intimately related, and understanding would involve inference, and traditional AI chaining backwards and forwards, to arrive at an understanding. Power, instead, starts his project by asserting that the robotic agents who will participate in the dialogue are copies of the same program and the planning system is potentially common to them both. Power’s system is very simple and the crucial difference between his and Cohen & Perrault’s is not brought out. Nevertheless, I believe it marks a fork in the road which has not been properly acknowledged, and a route which has not been followed through.
If this research direction is extrapolated, and if planning is common and not individual, then mutual knowledge (MK) is no longer a problem because, as Clark (1996) puts it: it is not ‘shared’ or ‘reflexive’ MK which causes problems but the ‘iterative’ variety of MK. Nor need there be a chasm between local and global management which distinguishes the approaches of the CA school and the traditional AI camp, since there is a hierarchy of planning corresponding with the hierarchy of tasks in the domain, as mentioned above; and this hierarchy underpins all manner of utterance bearing on the domain directly, and on the dialogue and meta-dialogue and meta-meta-dialogue etc. indirectly. Novick (1988) has gone some way towards this as his solution to what he calls the “integration problem” of reconciling the rationalistic with the sociolinguistic traditions. In general, we recognise what each other means because we are similar in the way we value things, or find things or events around us relevant. Another way of saying this is that we share the same general goals.
If this is a true and feasible way of characterising planning then it, in its turn, should allow the key conflicts within the NL community to be resolved. This extended, or deepened, concept of planning encompasses the rationalistic and the sociolinguistic traditions. Finally, this high-level account of one manner in which planning could be united with the primitive structures of language can also be regarded as a way of introducing the context systematically, since it is the goals of the situation which individuate it. However, though this establishes the semantics of a dialogue (or multi-modal interaction), it does not separate the particular domain of application from more general pragmatic determinants.
The fragmentation to which Carroll referred is not simply diversity, which is what, in the main, seems to describe the background literature. It is rather a fundamental lack of cooperation, in the sense that it leads to mutual incomprehension (lack of MK within the discipline), which is damaging and may block progress. It has been my aim on the basis of a brief and condensed argument to suggest a means of overcoming this lack of cooperation in the CHI-NL community (establishing the possibility of MK within the discipline), at the same time as suggesting a wider foundation (the extended and deepened notion of planning and cooperation) which accommodates the conflicting conceptions of cognitive design belonging to certain NL research groups. Furthermore, this aim is pursued on the basis of treating systematic design or engineering as quite distinct from scientific knowledge, without severing completely the epistemological connection.
Carroll, J. M. (1995) “Artefacts and Scenarios: an Engineering Approach” in “Perspective on HCI – Diverse Approaches”, ed. by Monk, A. F. & Gilbert, N., Academic Press
Carroll, J. M. (1997) “Human-computer interaction: psychology as a science of design”, International Journal of Human-Computer Studies, v46, pp501-522
Carroll, J. M. & Campbell, R.L. (1989) “Artefacts as psychological theories: the case of human-computer interaction”, Behaviour and Information Technology, v8, no4, pp247-256
Clark, H. H. (1996) “The Uses of Language”, Cambridge University Press
Cohen, P. R. & Perrault, C. R. (1979) “Elements of a Plan-based Theory of Speech Acts”, Cognitive Science, v3, pp177-212
Novick, D. G. “Control of Mixed-Initiative Discourse through Meta-Locutionary Acts: a Computational Model”, Technical Report CIS-TR-88-18, Dept. of Computer and Information Science, University of Oregon
Power, R. “The Organisation of Purposeful Dialogue” Linguistics, v17, pp107-152
Searle, J. R. (1969) “Speech Acts”, Cambridge University Press
(Other references are to the ‘Workshop Background’ bibliography)