When dictating using speech recognition,
most of the time is spent correcting errors.
However it is possible to predict which words are probably in error,
using various confidence measures.
Today most applicants to graduate programs cannot predict where they will be accepted. Even those departments which do give quantitative information regarding GPAs and GREs report statistics so obliquely that it is very difficult for a potential applicant to infer what they mean for him or her.
A little knowledge engineering and data mining led to a model of how graduate admissions decisions are made. I then coded this up in Javascript in the form of a calculator. Students interested in Computer Science graduate study can enter their data and get a quick estimate of where they are likely to be accepted.
Towards a Model of Computer Science Graduate Admissions Decisions. Nigel Ward. Journal of Advanced Computational Intelligence and Intelligent Informatics, 10, pp 372-383. pdf draft, final version
When dictating using speech recognition,
most of the time is spent correcting errors.
However it is possible to predict which words are probably in error,
using various confidence measures.
This suggests an editor with two new functions: display probable errors in red, and enable jumping to the next probable error with the tab key. Simple experiments suggest that these functions can be valuable.
Using a GOMS-type model of editing costs, it is possible to predict in which circumstances these function are useful, depending on three parameters: speech recognition rate, confidence measure quality, and user tolerance for uncorrected errors.
Can Confidence Scores Help Users Post-Editing Speech Recognizer Output? Taku Endo, Nigel Ward, and Minoru Terada. International Conference on Spoken Language Processing, 2002. pdf
A similar proposal was later made by Feng and Sears, in ToCHI 2004.
When you want to find out the basic meaning of some new word or abbreviation, the net is a good place, but search engines do rather poorly on single-word terminology-type queries.
It is possible to do better by adapting various classic information-retrieval techniques, together with the new technique of favoring pages which contain a region with a high local density of relevant concepts.
Searching for Explanatory Web Pages using Query Expansion.
This work won a Best Paper award at Pacling 2001, leading to an invited paper 5 years later:
Searching for Explanatory Web Pages using Query Expansion. (updated version) Manabu Tauchi and Nigel Ward. Computational Intelligence, 2006, to appear. pdf
DigitalNote is a freeware Java application which makes it easier to take class notes using your Tablet PC or other notebook computer. Notes taken with DigitalNote are superior to pencil-and-paper notes in being searchable, editable, easily sharable, and more legible.
DigitalNote may be useful for you if you type relatively quickly, you carry around a notebook PC all the time anyway, and equations and diagrams are rare in your classes ... or, if they are common, your PC is a tablet or has a touchscreen (i.e. is stylus-equipped).
DigitalNote is stable. While there are many improvements and new features that are needed, it is usable as-is. (Hajime Tatsukawa and Jabel Morales)
DigitalNote Downloads and Publications
Many speech-to-speech translation projects assume that perfect translation of arbitrary text is possible and necessary, but for many situations simpler technology is more appropriate. Our inpiration comes from phrasebooks, which contain useful, context-tailored information. They are, however, not convenient enough to use during actual interactions.
We developed a wearable device, Yak, as an aid to cross-language communication. Yak produces utterances in the native's langauge at the user's command. The use of a heads-up display allows the user to operate the device while maintaining eye contact with the native, and while sending and receiving non-verbal signals. We believe this interaction paradigm will be valuable for many face-to-face encounters, for example as travelers deal with people in service roles, such as ticket agents, waiters, clerks. In this work we developed a hardware configuration, a user interface, and a scripting language. Preliminary experiments showed that the system was usable. (Jani Patokallio)
A central issue for dialog systems is the proper handling of turn-taking, to ensure that the system responds swiftly, but without talking over the user or denying him or her time to think. For one class of dialogs, direction-giving dialogs, this is relatively easy; it can be done surprisingly well using only the information in the duration and pitch of the user's utterances, as we found in a confirmation of Schmandt's early work.
Pacing Spoken Directions to Suit the Listener. Tatsuya Iwase and Nigel Ward. 5th International Conference on Spoken Language Processing (ICSLP-98). abstract, pdf
Modern networked games generally support audio communication among players, but the way this is done is unrealistic in that the team members can hear each other perfectly, but opposing teams cannot overhear their talk at all. We have augmented Quake with an audio environment where audio is transmitted to everyone whose avatar is close to the avatar of any talker. We expect that this new functionality will make games more compelling. (Jaime C. Acosta)
Music plays an important part in the lives of many people, however the deaf are largely excluded from this aspect of society. We are developing software to present music visually, in the form of a spectrogram conveying the beat and some of the tonal information together with karaoke-style presentation of the lyrics. (Rosario Chavez)
When a new visitor comes to your e-commerce site, you usually don't know have a clue what sort of person it is, which makes customization hard.
In daily life, we easily classify people as to age, sex, confidence, and so on, based, among other things, on how they move. For example, from a person's walk, or driving style, or handwriting, it is possible to make a reasonable guess about some important properties.
It would be nice if the same were possible from users' movements as they they move the mouse across a web page.
Based on mouse trajectories from hundreds of users, it seems that there are no easy correlations. More detailed examination might be profitable.
Chikara Shimizu and Nigel Ward, 2002
Tidying up files and directories is a nuisance, and there's no software designed to help. One possible solution is to use an enhanced representation, with file properties represented with colors, shapes, and textures, and file organization represented as a 3-D tree.
This system won first prize at the "Night for Java Technology", 2001.
Three-dimensional Display of Directory Structure to Support Users Tidying up Files. (in Japanese) Keisuke Nishimoto and Nigel Ward. Interaction 2001 (87th Human Interface Workshop), Information Processing Society of Japan. 2001.
A static web page or brochure is not effective for conveying to potential new members the feeling of being and working in the lab. On the other hand, not everyone can physically come and visit.
An effective solution is to provide a chat system for/about the lab, inhabited by a gaggle of chatterbots each having the personality and speech patterns of a lab member, interacting with the user and each other.
Evaluating Information-Giving Chatterbots. (in Japanese). Tatsuya Iwase and Nigel Ward. Interaction 2000 (87th Human Interface Workshop), Information Processing Society of Japan. 2000.
Chat using standard chat systems gets confused if more than a few people are participating.
Many classic cyberspace ideas have shown up in chat systems, including a two- or three- dimensional layout of participants, and avatars' with controllable facial expressions.
Responsive Chat was a low-overhead implementation of some of these ideas, plus gaze direction transmission, which seems to be the most valuable feature of all.
Daisuke Nakamura and Nigel Ward, 2000
A Lobby Agent should be able to greet people, and provide information and entertainment appropriate for users and passer-bys level of interest, engagement, and so on.
Our system did image processing to detect position, posture and gesture, speech processing to detect prosodic features, and used this information to determine the next appropriate action.
Gathering data to use for system design |
Tracking the users's face and hands | 
Testing the system in the lobby |
9 second video (MPG video, 1.6MB)
Requirements for a Socially Aware Free-standing Agent. Nigel Ward and Takeshi Kuroda. Proceedings of the Second International Symposium on Humanoid Robotics, Waseda University, Tokyo, Japan, 1999 (pdf)
In the late 1980s many machine translation systems were under
development, but fluency of the output was generally poor.
I attacked this problem by building a new kind of natural language
generator, one in which the words and structures of the target language
were flexibly employed to generate outputs more natural
than those seen before. This generator was also incremental, parallel,
and worked without assembling syntactic structures, features which
were advantageous computationally and as a psycholinguistic model.
A Parallel Approach to Syntax for Generation. Nigel Ward. Artificial Intelligence 57, pages 183-225, 1992. abstract
A Connectionist Language Generator. Nigel Ward. Ablex 1994.
Related projects, and the students who created them, are listed at my Sanpo advisees page.
Up to Nigel Ward.