Man-machine dialogue : design and challenges

This book summarizes the main problems posed by the design of a man-machine dialogue system and offers ideas on how to continue along the path towards efficient, realistic and fluid communication between humans and machines. A culmination of ten years of research, it is based on the author's development, investigation and experimentation covering a multitude of fields, including artificial intelligence, automated language processing, man-machine interfaces and notably multimodal or multimedia interfaces.

Author Notes

Frédéric Landdragin is a computer science engineer and has a PhD from the University of Lorraine, France. He is currently in charge of linguistics research for the French National Center for Scientific Research (CNRS). His studies focus on the analysis and modeling of language interpretation. Man-machine dialogue is one of the applications of this research.

Preface	p. xi
Introduction	p. xv
Part 1 Historical and Methodological Landmarks	p. 1
Chapter 1 An Assessment of the Evolution of Research and Systems	p. 3
1.1 A few essential historical landmarks	p. 5
1.1.1 First motivations, first written systems	p. 6
1.1.2 First oral and multimodal systems	p. 11
1.1.3 Current systems: multiplicity of fields and techniques	p. 14
1.2 A list of possible abilities for a current system	p. 16
1.2.1 Recording devices and their use	p. 17
1.2.2 Analysis and reasoning abilities	p. 20
1.2.3 System reaction types and their manifestation	p. 22
1.3 The current challenges	p. 23
1.3.1 Adapting and integrating existing theories	p. 23
1.3.2 Diversifying systems' abilities	p. 25
1.3.3 Rationalizing the design	p. 26
1.3.4 Facilitating the implementation	p. 27
1.4 Conclusion	p. 27
Chapter 2 Man-Machine Dialogue Fields	p. 29
2.1 Cognitive aspects	p. 30
2.1.1 Perception, attention and memory	p. 32
2.1.2 Representation and reasoning	p. 35
2.1.3 Learning	p. 37
2.2 Linguistic aspects	p. 40
2.2.1 Levels of language analysis	p. 41
2.2.2 Automatic processing	p. 44
2.3 Computer aspects	p. 45
2.3.1 Data structures and digital resources	p. 45
2.3.2 Man-machine interfaces, plastic interfaces and ergonomics	p. 46
2.4 Conclusion	p. 46
Chapter 3 The Development Stages of a Dialogue System	p. 47
3.1 Comparing a few development progresses	p. 48
3.1.1 A scenario matching the 1980s	p. 48
3.1.2 A scenario matching the 2000s	p. 49
3.1.3 A scenario today	p. 51
3.2 Description of the main stages of development	p. 52
3.2.1 Specifying the system's task and roles	p. 52
3.2.2 Specifying covered phenomena	p. 54
3.2.3 Carrying out experiments and corpus studies	p. 55
3.2.4 Specifying the processing processes	p. 58
3.2.5 Resource writing and development	p. 59
3.2.6 Assessment and scalability	p. 61
3.3 Conclusion	p. 62
Chapter 4 Reusable System Architectures	p. 63
4.1 Run-time architectures	p. 64
4.1.1 A list of modules and resources	p. 64
4.1.2 The process flow	p. 66
4.1.3 Module interaction language	p. 68
4.2 Design-time architectures	p. 69
4.2.1 Toolkits	p. 70
4.2.2 Middleware for man-machine interaction	p. 71
4.2.3 Challenges	p. 72
4.3 Conclusion	p. 73
Part 2 Inputs Processing	p. 75
Chapter 5 Semantic Analyses and Representations	p. 77
5.1 Language in dialogue and in man-machine dialogue	p. 78
5.1.1 The main characteristics of natural language	p. 78
5.1.2 Oral and written languages	p. 82
5.1.3 Language and spontaneous dialogue	p. 83
5.1.4 Language and conversational gestures	p. 84
5.2 Computational processes: from the signal to the meaning	p. 85
5.2.1 Syntactic analyses	p. 86
5.2.2 Semantic and conceptual resources	p. 87
5.2.3 Semantic analyses	p. 89
5.3 Enriching meaning representation	p. 91
5.3.1 At the level of linguistic utterance	p. 91
5.3.2 At the level of multimodal utterance	p. 93
5.4 Conclusion	p. 94
Chapter 6 Reference Resolution	p. 95
6.1 Object reference resolution	p. 96
6.1.1 Multimodal reference domains	p. 98
6.1.2 Visual scene analysis	p. 100
6.1.3 Pointing gesture analysis	p. 101
6.1.4 Reference resolution depending on determination	p. 102
6.2 Action reference resolution	p. 105
6.2.1 Action reference and verbal semantics	p. 105
6.2.2 Analyzing the utterance "put that there"	p. 108
6.3 Anaphora and coreference processing	p. 109
6.4 Conclusion	p. 112
Chapter 7 Dialogue Acts Recognition	p. 113
7.1 Nature of dialogue acts	p. 114
7.1.1 Definitions and phenomena	p. 114
7.1.2 The issue with indirect acts	p. 116
7.1.3 The issue with composite acts	p. 118
7.2 Identification and processing of dialogue acts	p. 119
7.2.1 Act identification and classification	p. 119
7.2.2 Indirect and composite acts	p. 121
7.3 Multimodal dialogue act processing	p. 122
7.4 Conclusion	p. 124
Part 3 System Behavior and Evaluation	p. 125
Chapter 8 A Few Dialogue Strategies	p. 127
8.1 Natural and cooperative aspects of dialogue management	p. 128
8.1.1 Common goal and cooperation	p. 128
8.1.2 Speaking turns and interactive aspects	p. 130
8.1.3 Interpretation and inferences	p. 132
8.1.4 Dialogue, argumentation and coherence	p. 133
8.1.5 Choosing an answer	p. 135
8.2 Technical aspects of dialogue management	p. 136
8.2.1 Dialogue management and control	p. 136
8.2.2 Dialogue history modeling	p. 138
8.2.3 Dialogue management and multimodality management	p. 143
8.2.4 Can a dialogue system he?	p. 145
8.3 Conclusion	p. 147
Chapter 9 Multimodal Output Management	p. 149
9.1 Output management methodology	p. 151
9.1.1 General principles of output multimodality	p. 151
9.1.2 Human factors for multimedia presentation	p. 153
9.2 Multimedia presentation pragmatics	p. 156
9.2.1 Elocutionary forces and values	p. 156
9.2.2 Perlocutionary forces and values	p. 157
9.3 Processes	p. 159
9.3.1 Allocation of the information over communication channels	p. 159
9.3.2 Redundancy management and multimodal fission	p. 161
9.3.3 Generation of referring expressions	p. 162
9.3.4 Valorizing part of the information and text to speech synthesis	p. 163
9.4 Conclusion	p. 165
Chapter 10 Multimodal Dialogue System Assessment	p. 167
10.1 Dialogue system assessment feasibility	p. 168
10.1.1 A few assessment experiments	p. 170
10.1.2 Man-machine interface methodologies	p. 172
10.1.3 Oral dialogue methodologies	p. 174
10.1.4 Multimodal dialogue methodologies	p. 175
10.2 Multimodal system assessment challenges	p. 176
10.2.1 Global assessment or segmented assessment?	p. 176
10.2.2 Should a multimodal corpus be managed?	p. 178
10.2.3 Can we compare several multimodal systems?	p. 179
10.3 Methodological elements	p. 180
10.3.1 User expertise and system complexity	p. 181
10.3.2 Questionnaires for users	p. 183
10.3.3 Extending DQR and DCR to multimodal dialogue	p. 185
10.3.4 Towards other assessment methods	p. 189
10.4 Conclusion	p. 190
Conclusion	p. 191
Bibliography	p. 193
Index	p. 203

Available:*

On Order

Summary

Summary

Author Notes

Table of Contents