Multimodality in Language and Speech Systems

This book is based on contributions to the Seventh European Summer School on Language and Speech Communication that was held at KTH in Stockholm, Sweden, in July of 1999 under the auspices of the European Language and Speech Network (ELSNET). This work covers the topic of multimodality from a large number of different perspectives and provides the advanced student/researcher with a survey of theories of multimodal communication between people as well as reviewing many aspects of multimodal input/output in technical systems. Chapters dealing with human-human multimodal communication include speech-gesture systems, semiotics of gesture, structure and functions of face-to-face communication, emotional relations and intercultural variation, and human-human communication which is mediated by computer for the handicapped. Chapters dealing with human-machine communication and interfaces cover the technology and science of creating talking faces, technology and methods for the development of animated interface agents in intelligent multimedia systems, and the integration of multimodal input and output in the computer interface. The book also covers computer processing and understanding of signal and symbol input from speech, text, and visual images.