Multimodality and genre : a foundation for the systematic analysis of multimodal documents

The first systematic, corpus-based and theoretically rigorous approach to the description and analysis of multimodal documents. Drawing on academic research and the experience of designers and production teams, Bateman uses linguistically-based analysis to show how different modes of expression together make up a document with a recognisable genre.

Author Notes

JOHN BATEMAN is Professor of English Linguistics at the University of Bremen, Germany.

List of Tables	p. ix
List of Figures	p. xi
Acknowledgements	p. xvii
Preface	p. xix
1 Introduction: Four Whys and a How	p. 1
1.1 Learning to walk: framing issues and analytic focus	p. 3
1.1.1 Why multimodality?	p. 3
1.1.2 Why 'documents'?	p. 7
1.1.3 Why genre?	p. 9
1.1.4 Why analysis?	p. 11
1.2 How can we analyse multimodal documents?	p. 13
1.2.1 An orientation for analysis: empirical linguistics	p. 14
1.2.2 A framework for empirical analysis: the GeM model	p. 15
1.2.3 Structure of the book	p. 19
2 Multimodal Documents and their Components	p. 21
2.1 Starting points: how to find document parts?	p. 24
2.2 The page as an object of interpretation	p. 27
2.2.1 Interpretation within document design	p. 28
2.2.2 Multimodal linguistics	p. 38
2.3 The page as object of perception	p. 57
2.4 Page as signal	p. 65
2.5 The Page as object of production	p. 74
2.5.1 Describing a page for design	p. 75
2.5.2 Describing a page for rendering	p. 85
2.5.3 Producing a page from intentions: automatic document generation	p. 91
2.6 Combining viewpoints on document parts	p. 103
3 The GeM Model: Treating the Multimodal Page as a Multilayered Semiotic Artefact	p. 107
3.1 The GeM Model: the base layer	p. 110
3.2 The GeM presentation layers: the layout base	p. 115
3.2.1 Layout segmentation: identification of layout units	p. 116
3.2.2 Realisation information	p. 117
3.2.3 Layout structure	p. 121
3.3 A more complicated example of layout analysis	p. 129
3.3.1 The parts of the Louvre	p. 130
3.3.2 The layout of the Louvre	p. 134
3.4 Conclusion	p. 142
4 The Rhetorical Organisation of Multimodal Documents	p. 143
4.1 Rhetoric and multimodal documents: our starting points	p. 144
4.2 A brief introduction to Rhetorical Structure Theory	p. 146
4.2.1 The RST rhetorical relations	p. 147
4.2.2 The RST rhetorical structure	p. 150
4.3 The move to multimodal RST: the GeM rhetorical layer	p. 151
4.3.1 Andre's extension of RST	p. 152
4.3.2 Problems with traditional multimodal RST	p. 155
4.3.3 Multimodal relationals: subnuclear elaboration	p. 160
4.4 Example analyses: rhetorical relations between layout units	p. 163
4.4.1 Mismatches between layout structure and intended rhetorical structure	p. 166
4.4.2 Explaining how to use a telephone	p. 171
4.5 Conclusion	p. 174
5 Multimodal Documents and Genre	p. 177
5.1 Perspectives on genre	p. 183
5.1.1 Genre as social semiotic	p. 184
5.1.2 Genre as social action	p. 188
5.1.3 Genre: the need for fine-grained descriptions	p. 194
5.2 The move to multimodal genre	p. 196
5.2.1 Multimodal moves within linguistic and rhetorical approaches to genre	p. 197
5.2.2 Moving in on genre from the visual	p. 201
5.2.3 Cybergenres: a brief critique	p. 209
5.3 Representing genre	p. 217
5.3.1 Genre typology	p. 219
5.3.2 Genre topology	p. 223
5.4 The multimodal genre space	p. 225
5.5 Illustrations of genre: tracking change	p. 229
5.5.1 Field guides across time	p. 229
5.5.2 Wildlife fact files across time	p. 240
5.6 Discussion and conclusion	p. 246
6 Building Multimodal Document Corpora: the State of the Art	p. 249
6.1 Corpus-based linguistics	p. 250
6.2 The origin and representation of annotated corpora	p. 252
6.2.1 Annotated corpora: early days	p. 252
6.2.2 Applying XML to corpus design	p. 254
6.2.3 Annotation problems with complex data	p. 260
6.3 The move to multimodal corpora	p. 264
6.4 The GeM model as a corpus annotation scheme	p. 267
6.5 Conclusions and recommendations	p. 272
7 Conclusions and Outlook: What Next?	p. 273
Bibliography	p. 279
Author Index	p. 301
Subject Index	p. 307

Available:*

On Order

Summary

Summary

Author Notes

Table of Contents