Recognition of whiteboard notes : online, offline and combination

This book addresses the task of processing online handwritten notes acquired from an electronic whiteboard, which is a new modality in handwriting recognition research. The main motivation of this book is smart meeting rooms, aim to automate standard tasks usually performed by humans in a meeting.The book can be summarized as follows. A new online handwritten database is compiled, and four handwriting recognition systems are developed. Moreover, novel preprocessing and normalization strategies are designed especially for whiteboard notes and a new neural network based recognizer is applied. Commercial recognition systems are included in a multiple classifier system. The experimental results on the test set show a highly significant improvement of the recognition performance to more than 86%.

Preface	p. vii
List of Figures	p. xv
List of Tables	p. xix
1 Introduction	p. 1
1.1 Motivation	p. 2
1.1.1 Smart Meeting Rooms	p. 3
1.1.2 Human Performance	p. 4
1.2 Handwriting Recognition	p. 7
1.2.1 Recognition System Overview	p. 7
1.2.2 Offline Handwriting Recognition	p. 10
1.2.3 Online Handwriting Recognition	p. 12
1.3 Comparability of Recognition Results	p. 15
1.4 Related Topics	p. 17
1.5 Contribution	p. 18
1.6 Outline	p. 19
2 Classification Methods	p. 21
2.1 Hidden Markov Models	p. 21
2.1.1 Definition	p. 22
2.1.2 Training and Testing	p. 23
2.1.3 Design Parameters	p. 25
2.1.4 Adaptation	p. 27
2.2 Neural Networks	p. 28
2.2.1 Multilayer Perceptron Networks	p. 28
2.2.2 Recurrent Neural Networks	p. 30
2.2.3 Long Short-Term Memory	p. 31
2.2.4 Bidirectional Recurrent Neural Networks	p. 33
2.2.5 Connectionist Temporal Classification	p. 34
2.3 Gaussian Mixture Models	p. 44
2.3.1 Definition	p. 44
2.3.2 Training and Testing	p. 45
2.3.3 Adaptation	p. 47
2.4 Language Models	p. 47
2.4.1 N-grams	p. 48
2.4.2 Language Model Perplexity	p. 49
2.4.3 Integration into Recognition	p. 49
3 Linguistic Resources and Handwriting Databases	p. 51
3.1 Linguistic Resources	p. 51
3.1.1 Major Corpora	p. 52
3.1.2 Massive Electronic Collections	p. 54
3.2 IAM Offline Database	p. 55
3.2.1 Description	p. 55
3.2.2 Acquisition	p. 56
3.2.3 Benchmarks	p. 59
3.3 IAM Online Database	p. 60
3.3.1 Description	p. 60
3.3.2 Acquisition	p. 62
3.3.3 Benchmarks	p. 63
4 Offline Approach	p. 67
4.1 System Description	p. 67
4.1.1 Online Preprocessing	p. 68
4.1.2 Online to Offline Transformation	p. 69
4.1.3 Offline Preprocessing and Feature Extraction	p. 70
4.1.4 Recognition	p. 72
4.2 Enhancing the Training Set	p. 73
4.2.1 Enhancing the Training Set with a Different Database	p. 73
4.2.2 Enhancing the Training Set with the IAM-OnDB	p. 74
4.3 Experiments	p. 75
4.3.1 Performance Measures	p. 75
4.3.2 Testing Significance	p. 76
4.3.3 Initial Experiments	p. 78
4.3.4 Experiments on Enhanced Training Data	p. 80
4.3.5 Experiments on the IAM-OnDB-t1 Benchmark	p. 82
4.4 Word Extraction	p. 83
4.4.1 Previous Work	p. 83
4.4.2 Data Preparation	p. 84
4.4.3 Word Extraction Method	p. 86
4.4.4 Word Extraction Experiments	p. 87
4.4.5 Word Recognition Experiments	p. 89
4.5 Conclusions	p. 90
5 Online Approach	p. 93
5.1 Line Segmentation	p. 94
5.1.1 System Overview	p. 95
5.1.2 Dynamic Programming	p. 96
5.1.3 Cost Function	p. 97
5.1.4 Postprocessing	p. 100
5.1.5 Experiments and Results	p. 100
5.2 Preprocessing	p. 102
5.2.1 Online Preprocessing	p. 103
5.2.2 Normalization	p. 104
5.3 Features	p. 107
5.3.1 Online Features	p. 107
5.3.2 Pseudo Offline Features	p. 109
5.4 HMM-Based Experiments	p. 110
5.4.1 Recognition Parameters	p. 110
5.4.2 Initial Experiments	p. 111
5.4.3 Results on the Benchmarks of the IAM-OnDB	p. 112
5.4.4 Feature Subset Selection Experiments	p. 113
5.5 Experiments with Neural Networks	p. 116
5.5.1 Recognition Parameters	p. 118
5.5.2 Results on the IAM-OnDB-t2 Benchmark	p. 119
5.5.3 Analysis	p. 120
5.5.4 Influence of the Vocabulary	p. 121
5.6 Conclusions and Discussion	p. 123
6 Multiple Classifier Combination	p. 129
6.1 Methodology	p. 130
6.1.1 Alignment	p. 131
6.1.2 Voting Strategies	p. 132
6.2 Recognition Systems	p. 133
6.2.1 Hidden Markov Models and Neural Networks	p. 133
6.2.2 Microsoft© Handwriting Recognition Engine	p. 134
6.2.3 Vision Objects© Recognizer	p. 138
6.3 Initial Experiments	p. 140
6.3.1 Combination of HMM-Based Recognizers	p. 140
6.3.2 Combination of Neural Networks	p. 141
6.4 Experiments with All Recognition Systems	p. 143
6.4.1 Individual Recognition Results	p. 144
6.4.2 Optimization on the Validation Set	p. 145
6.4.3 Test Set Results	p. 148
6.5 Advanced Confidence Measures	p. 149
6.5.1 Voting Strategies	p. 149
6.5.2 Experiments	p. 150
6.6 Conclusions	p. 151
7 Writer-Dependent Recognition	p. 155
7.1 Writer Identification	p. 156
7.1.1 System Overview	p. 156
7.1.2 Feature Sets for Whiteboard Writer Identification	p. 158
7.1.3 Experiments	p. 160
7.2 Writer-Dependent Experiments	p. 165
7.2.1 Experimental Setup	p. 167
7.2.2 Results	p. 168
7.3 Automatic Handwriting Classification	p. 169
7.3.1 Classification Systems	p. 170
7.3.2 Combination	p. 171
7.3.3 Experiments and Results	p. 172
7.4 Conclusions	p. 176
8 Conclusions	p. 181
8.1 Overview of Recognition Systems	p. 183
8.2 Overview of Experimental Results	p. 186
8.3 Concluding Remarks	p. 188
8.4 Outlook	p. 189
Bibliography	p. 191
Index	p. 205

Available:*

On Order

Summary

Summary

Table of Contents