Ebook: Multimodality in Language and Speech Systems

This book is based on contributions to the Seventh European Summer School on Language and Speech Communication that was held at KTH in Stockholm, Sweden, in July of 1999 under the auspices of the European Language and Speech Network (ELSNET). The topic of the summer school was "Multimodality in Language and Speech Systems" (MiLaSS). The issue of multimodality in interpersonal, face-to-face communication has been an important research topic for a number of years. With the increasing sophistication of computer-based interactive systems using language and speech, the topic of multimodal interaction has received renewed interest both in terms of human-human interaction and human-machine interaction. Nine lecturers contri­ buted to the summer school with courses on specialized topics ranging from the technology and science of creating talking faces to human-human communication, which is mediated by computer for the handicapped. Eight of the nine lecturers are represented in this book. The summer school attracted more than 60 participants from Europe, Asia and North America representing not only graduate students but also senior researchers from both academia and industry.

---

This book covers the topic of multimodality from a large number of different perspectives and provides the advanced student/researcher with a current survey of theories of multimodal communication between people as well as reviewing many aspects of multimodal input/output in technical systems. Chapters dealing with human-human multimodal communication include speech-gesture systems, semiotics of gesture, structure and functions of face-to-face communication, emotional relations and intercultural variation, and human-human communication which is mediated by computer for the handicapped. Chapters dealing with human-machine communication and interfaces cover the technology and science of creating talking faces, technology and methods for the development of animated interface agents in intelligent multimedia systems, and the integration of multimodal input and output in the computer interface. The book also covers computer processing and understanding of signal and symbol input from speech, text, and visual images.

---

This book covers the topic of multimodality from a large number of different perspectives and provides the advanced student/researcher with a current survey of theories of multimodal communication between people as well as reviewing many aspects of multimodal input/output in technical systems. Chapters dealing with human-human multimodal communication include speech-gesture systems, semiotics of gesture, structure and functions of face-to-face communication, emotional relations and intercultural variation, and human-human communication which is mediated by computer for the handicapped. Chapters dealing with human-machine communication and interfaces cover the technology and science of creating talking faces, technology and methods for the development of animated interface agents in intelligent multimedia systems, and the integration of multimodal input and output in the computer interface. The book also covers computer processing and understanding of signal and symbol input from speech, text, and visual images.
Content:
Front Matter....Pages i-ix
Introduction....Pages 1-6
Bodily Communication Dimensions of Expression and Content....Pages 7-26
Dynamic Imagery in Speech and Gesture....Pages 27-44
Multimodal Speech Perception: A Paradigm for Speech Science....Pages 45-71
Multimodal Interaction and People with Disabilities....Pages 73-92
Multimodality in Language and Speech Systems — From Theory to Design Support Tool....Pages 93-148
Developing Intelligent Multimedia Applications....Pages 149-171
Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action....Pages 173-207
Speech and Gestures for Talking Faces in Conversational Dialogue Systems....Pages 209-241
Back Matter....Pages 243-243

---

This book covers the topic of multimodality from a large number of different perspectives and provides the advanced student/researcher with a current survey of theories of multimodal communication between people as well as reviewing many aspects of multimodal input/output in technical systems. Chapters dealing with human-human multimodal communication include speech-gesture systems, semiotics of gesture, structure and functions of face-to-face communication, emotional relations and intercultural variation, and human-human communication which is mediated by computer for the handicapped. Chapters dealing with human-machine communication and interfaces cover the technology and science of creating talking faces, technology and methods for the development of animated interface agents in intelligent multimedia systems, and the integration of multimodal input and output in the computer interface. The book also covers computer processing and understanding of signal and symbol input from speech, text, and visual images.
Content:
Front Matter....Pages i-ix
Introduction....Pages 1-6
Bodily Communication Dimensions of Expression and Content....Pages 7-26
Dynamic Imagery in Speech and Gesture....Pages 27-44
Multimodal Speech Perception: A Paradigm for Speech Science....Pages 45-71
Multimodal Interaction and People with Disabilities....Pages 73-92
Multimodality in Language and Speech Systems — From Theory to Design Support Tool....Pages 93-148
Developing Intelligent Multimedia Applications....Pages 149-171
Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action....Pages 173-207
Speech and Gestures for Talking Faces in Conversational Dialogue Systems....Pages 209-241
Back Matter....Pages 243-243
....