Unsupervised learning : a dynamic approach

Evolving technologies have brought about an explosion of information in recent years, but the question of how such information might be effectively harvested, archived, and analyzed remains a monumental challenge--for the processing of such information is often fraught with the need for conceptual interpretation: a relatively simple task for humans, yet an arduous one for computers.

Inspired by the relative success of existing popular research on self-organizing neural networks for data clustering and feature extraction, Unsupervised Learning: A Dynamic Approach presents information within the family of generative, self-organizing maps, such as the self-organizing tree map (SOTM) and the more advanced self-organizing hierarchical variance map (SOHVM). It covers a series of pertinent, real-world applications with regard to the processing of multimedia data--from its role in generic image processing techniques, such as the automated modeling and removal of impulse noise in digital images, to problems in digital asset management and its various roles in feature extraction, visual enhancement, segmentation, and analysis of microbiological image data.

Self-organization concepts and applications discussed include:

Distance metrics for unsupervised clustering Synaptic self-amplification and competition Image retrieval Impulse noise removal Microbiological image analysis

Unsupervised Learning: A Dynamic Approach introduces a new family of unsupervised algorithms that have a basis in self-organization, making it an invaluable resource for researchers, engineers, and scientists who want to create systems that effectively model oppressive volumes of data with little or no user intervention.

Author Notes

Matthew Kyan received his Ph.D. in Electrical Engineering in 2007 from the University of Sydney. Australia, winning the Siemens National Prize for innovation for his work with 3-D confocal imaging. He is currently an Assistant Professor at Ryerson University, Toronto, Canada.
Paisarn Muneesawang received his Ph.D. from the school of Electrical and Information Engineering at the University of Sydney in 2002. He is currently an Associate Professor at Naresuan University.
Kambiz Jarrah received his B.Eng. (with honors) in 2004 and M.A.Sc. in 2006, both in Electrical Engineering, from Ryerson University.
Ling Guan is a Canada Research Chair in Multimedia and Computer Technology and a Professor in Electrical and Computer Engineering at Ryerson University, Canada.

Acknowledgments	p. xi
1 Introduction	p. 1
1.1 Part I: The Self-Organizing Method	p. 1
1.2 Part II: Dynamic Self-Organization for Image Filtering and Multimedia Retrieval	p. 2
1.3 Part III: Dynamic Self-Organization for Image Segmentation and Visualization	p. 5
1.4 Future Directions	p. 7
2 Unsupervised Learning	p. 9
2.1 Introduction	p. 9
2.2 Unsupervised Clustering	p. 9
2.3 Distance Metrics for Unsupervised Clustering	p. 11
2.4 Unsupervised Learning Approaches	p. 13
2.4.1 Partitioning and Cluster Membership	p. 13
2.4.2 Iterative Mean-Squared Error Approaches	p. 15
2.4.3 Mixture Decomposition Approaches	p. 17
2.4.4 Agglomerative Hierarchical Approaches	p. 18
2.4.5 Graph-Theoretic Approaches	p. 20
2.4.6 Evolutionary Approaches	p. 20
2.4.7 Neural Network Approaches	p. 21
2.5 Assessing Cluster Quality and Validity	p. 21
2.5.1 Cost Function-Based Cluster Validity Indices	p. 22
2.5.2 Density-Based Cluster Validity Indices	p. 23
2.5.3 Geometric-Based Cluster Validity Indices	p. 24
3 Self-Organization	p. 27
3.1 Introduction	p. 27
3.2 Principles of Self-Organization	p. 27
3.2.1 Synaptic Self-Amplification and Competition	p. 27
3.2.2 Cooperation	p. 28
3.2.3 Knowledge Through Redundancy	p. 29
3.3 Fundamental Architectures	p. 29
3.3.1 Adaptive Resonance Theory	p. 29
3.3.2 Self-Organizing Map	p. 37
3.4 Other Fixed Architectures for Self-Organization	p. 43
3.4.1 Neural Gas	p. 44
3.4.2 Hierarchical Feature Map	p. 45
3.5 Emerging Architecture for Self-Organization	p. 46
3.5.1 Dynamic Hierarchical Architectures	p. 47
3.5.2 Nonstationary Architectures	p. 48
3.5.3 Hybrid Architectures	p. 50
3.6 Conclusion	p. 50
4 Self-Organizing Tree Map	p. 53
4.1 Introduction	p. 53
4.2 Architecture	p. 54
4.3 Competitive Learning	p. 55
4.4 Algorithm	p. 57
4.4 Evolution	p. 61
4.5.1 Dynamic Topology	p. 61
4.5.2 Classification Capability	p. 64
4.6 Practical Considerations, Extensions, and Refinements	p. 68
4.6.1 The Hierarchical Control Function	p. 68
4.6.2 Learning, Timing, and Convergence	p. 71
4.6.3 Feature Normalization	p. 73
4.6.4 Stop Criteria	p. 73
4.7 Conclusions	p. 74
5 Self-Organization in Impulse Noise Removal	p. 75
5.1 Introduction	p. 75
5.2 Review of Traditional Median-Type Filters	p. 76
5.3 The Noise-Exclusive Adaptive Filtering	p. 82
5.3.1 Feature Selection and Impulse Detection	p. 82
5.3.2 Noise Removal Filters	p. 84
5.4 Experiment Results	p. 86
5.5 Detection-Guided Restoration and Real-Time Processing	p. 99
5.5.1 Introduction	p. 99
5.5.2 Iterative Filtering	p. 101
5.5.3 Recursive Filtering	p. 104
5.5.4 Real-Time Processing of Impulse Corrupted TV Pictures	p. 105
5.5.5 Analysis of the Processing Time	p. 109
5.6 Conclusions	p. 115
6 Self-Organizations in Image Retrieval	p. 119
6.1 Retrieval of Visual Information	p. 120
6.2 Visual Feature Descriptor	p. 122
6.2.1 Color Histogram and Color Moment Descriptors	p. 122
6.2.2 Wavelet Moment and Gabor Texture Descriptors	p. 123
6.2.3 Fourier and Moment-based Shape Descriptors	p. 125
6.2.4 Feature Normalization and Selection	p. 127
6.3 User-Assisted Retrieval	p. 130
6.3.1 Radial Basis Function Method	p. 132
6.4 Self-Organization for Pseudo Relevance Feedback	p. 136
6.5 Directed Self-Organization	p. 140
6.5.1 Algorithm	p. 142
6.6 Optimizing Self-Organization for Retrieval	p. 146
6.6.1 Genetic Principles	p. 147
6.6.2 System Architecture	p. 149
6.6.3 Genetic Algorithm for Feature Weight Detection	p. 150
6.7 Retrieval Performance	p. 153
6.7.1 Directed Self-Organization	p. 153
6.7.2 Genetic Algorithm Weight Detection	p. 155
6.8 Summary	p. 157
7 The Self-Organizing Hierarchical Variance Map	p. 159
7.1 An Intuitive Basis	p. 160
7.2 Model Formulation and Breakdown	p. 162
7.2.1 Topology Extraction via Competitive Hebbian Learning	p. 163
7.2.2 Local Variance via Hebbian Maximal Eigenfilters	p. 165
7.2.3 Global and Local Variance Interplay for Map Growth and Termination	p. 170
7.3 Algorithm	p. 173
7.3.1 Initialization, Continuation, and Presentation	p. 173
7.3.2 Updating Network Parameters	p. 175
7.3.3 Vigilance Evaluation and Map Growth	p. 175
7.3.4 Topology Adaptation	p. 176
7.3.5 Node Adaptation	p. 177
7.3.6 Optional Tuning Stage	p. 177
7.4 Simulations and Evaluation	p. 177
7.4.1 Observations of Evolution and Partitioning	p. 178
7.4.2 Visual Comparisons with Popular Mean-Squared Error ARchitectures	p. 181
7.4.3 Visual Comparison Against Growing Neural Gas	p. 183
7.4.4 Comparing Hierarchical with Tree-Based Methods	p. 183
7.5 Tests on Self-Determination and the Optional Tuning Stage	p. 187
7.6 Cluster Validity Analysis on Synthetic and UCI Data	p. 187
7.6.1 Performance vs. Popular Clustering Methods	p. 190
7.6.2 IRIS Dataset	p. 192
7.6.3 WINE Dataset	p. 195
7.7 Summary	p. 195
8 Microbiological Image Analysis Using Self-Organization	p. 197
8.1 Image Analysis in the Biosciences	p. 197
8.1.1 Segmentation: The Common Denominator	p. 198
8.1.2 Semi-supervised versus Unsupervised Analysis	p. 199
8.1.3 Confocal Microscopy and Its Modalities	p. 200
8.2 Image Analysis Tasks Considered	p. 202
8.2.1 Visualising Chromosomes During Mitosis	p. 202
8.2.2 Segmenting Heterogeneous Biofilms	p. 204
8.3 Microbiological Image Segmentation	p. 205
8.3.1 Effects of Feature Space Definition	p. 207
8.3.2 Fixed Weighting of Feature Space	p. 209
8.3.3 Dynamic Feature Fusion During Learning	p. 213
8.4 Image Segmentation Using Hierarchical Self-Organization	p. 215
8.4.1 Gray-Level Segmentation of Chromosomes	p. 215
8.4.2 Automated Multilevel Thresholding of Biofilm	p. 220
8.4.3 Multidimensional Feature Segmentation	p. 221
8.5 Harvesting Topologies to Facilitate Visualization	p. 226
8.5.1 Topology Aware Opacity and Gray-Level Assignment	p. 227
8.5.2 Visualization of Chromosomes During Mitosis	p. 228
8.6 Summary	p. 233
9 Closing Remarks and Future Directions	p. 237
9.1 Summary of Main Findings	p. 237
9.1.1 Dynamic Self-Organization: Effective Models for Efficient Feature Space Parsing	p. 237
9.1.2 Improved Stability, Integrity, and Efficiency	p. 238
9.1.3 Adaptive Topologies Promote Consistency and Uncover Relationships	p. 239
9.1.4 Online Selection of Class Number	p. 239
9.1.5 Topologies Represent a Useful Backbone for Visualization or Analysis	p. 240
9.2 Future Directions	p. 240
9.2.1 Dynamic Navigation for Information Repositories	p. 241
9.2.2 Interactive Knowledge-Assisted Visualization	p. 243
9.2.3 Temporal Data Analysis Using Trajectories	p. 245
Appendix A p. 249
A.1 Global and Local Consistency Error	p. 249
References	p. 251
Index	p. 269

Available:*

On Order

Summary

Summary

Author Notes

Table of Contents