Neural networks and computing : learning algorithms and applications

This book covers neural networks with special emphasis on advanced learning methodologies and applications. It includes practical issues of weight initializations, stalling of learning, and escape from a local minima, which have not been covered by many existing books in this area. Additionally, the book highlights the important feature selection problem, which baffles many neural networks practitioners because of the difficulties handling large datasets. It also contains several interesting IT, engineering and bioinformatics applications.

Preface	p. V
1 Introduction	p. 1
1.1 Background	p. 1
1.2 Neuron Model	p. 2
1.3 Historical Remarks	p. 4
1.4 Network architecture	p. 6
1.4.1 Supervised Neural Networks	p. 6
1.4.1.1 McCulloh and Pitts Model	p. 7
1.4.1.2 The Perceptron Model	p. 11
1.4.1.3 Multi-layer Feedforward Network	p. 14
1.4.1.4 Recurrent Networks	p. 15
1.4.2 Unsupervised Neural Networks	p. 17
1.5 Modeling and Learning Mechanism	p. 19
1.5.1 Determination of Parameters	p. 20
1.5.2 Gradient Descent Searching Method	p. 26
Exercises	p. 28
2 Learning Performance and Enhancement	p. 31
2.1 Fundamental of Gradient Descent Optimization	p. 32
2.2 Conventional Backpropagation Algorithm	p. 35
2.3 Convergence Enhancement	p. 42
2.3.1 Extended Backpropagation Algorithm	p. 44
2.3.2 Least Squares Based Training Algorithm	p. 47
2.3.3 Extended Least Squares Based Algorithm	p. 55
2.4 Initialization Consideration	p. 59
2.4.1 Weight Initialization Algorithm I	p. 61
2.4.2 Weight Initialization Algorithm II	p. 64
2.4.3 Weight Initialization Algorithm III	p. 67
2.5 Global Learning Algorithms	p. 69
2.5.1 Simulated Annealing Algorithm	p. 70
2.5.2 Alopex Algorithm	p. 71
2.5.3 Reactive Tabu Search	p. 72
2.5.4 The NOVEL Algorithm	p. 73
2.5.5 The Heuristic Hybrid Global Learning Algorithm	p. 74
2.6 Concluding Remarks	p. 82
2.6.1 Fast Learning Algorithms	p. 82
2.6.2 Weight Initialization Methods	p. 83
2.6.3 Global Learning Algorithms	p. 84
Appendix 2.1 p. 85
Exercises	p. 87
3 Generalization and Performance Enhancement	p. 91
3.1 Cost Function and Performance Surface	p. 93
3.1.1 Maximum Likelihood Estimation	p. 94
3.1.2 The Least-Square Cost Function	p. 95
3.2 Higher-Order Statistic Generalization	p. 98
3.2.1 Definitions and Properties of Higher-Order Statistics	p. 99
3.2.2 The Higher-Order Cumulants based Cost Function	p. 101
3.2.3 Property of the Higher-Order Cumulant Cost Function	p. 105
3.2.4 Learning and Generalization Performance	p. 108
3.2.4.1 Experiment one: Henon Attractor	p. 109
3.2.4.2 Experiment Two: Sunspot time-series	p. 116
3.3 Regularization for Generalization Enhancement	p. 117
3.3.1 Adaptive Regularization Parameter Selection (ARPS) Method	p. 120
3.3.1.1 Stalling Identification Method	p. 121
3.3.1.2 [lambda] Selection Schemes	p. 122
3.3.2 Synthetic Function Mapping	p. 124
3.4 Concluding Remarks	p. 126
3.4.1 Objective function selection	p. 128
3.4.2 Regularization selection	p. 129
Appendix 3.1 Confidence Upper Bound of Approximation Error	p. 131
Appendix 3.2 Proof of the Property of the HOC Cost Function	p. 133
Appendix 3.3 The Derivation of the Sufficient Conditions of the Regularization Parameter	p. 136
Exercises	p. 137
4 Basis Function Networks for Classification	p. 139
4.1 Linear Separation and Perceptions	p. 140
4.2 Basis Function Model for Parametric Smoothing	p. 142
4.3 Radial Basis Function Network	p. 144
4.3.1 RBF Networks Architecture	p. 144
4.3.2 Universal Approximation	p. 146
4.3.3 Initialization and Clustering	p. 149
4.3.4 Learning Algorithms	p. 152
4.3.4.1 Linear Weights Optimization	p. 152
4.3.4.2 Gradient Descent Optimization	p. 154
4.3.4.3 Hybrid of Least Squares and Penalized Optimization	p. 155
4.3.5 Regularization Networks	p. 157
4.4 Advanced Radial Basis Function Networks	p. 159
4.4.1 Support Vector Machine	p. 159
4.4.2 Wavelet Network	p. 161
4.4.3 Fuzzy RBF Controllers	p. 164
4.4.4 Probabilistic Neural Networks	p. 167
4.5 Concluding Remarks	p. 169
Exercises	p. 170
5 Self-organizing Maps	p. 173
5.1 Introduction	p. 173
5.2 Self-Organizing Maps	p. 177
5.2.1 Learning Algorithm	p. 178
5.3 Growing SOMs	p. 182
5.3.1 Cell Splitting Grid	p. 182
5.3.2 Growing Hierarchical Self-Organizing Quadtree Map	p. 185
5.4 Probabilistic SOMs	p. 188
5.4.1 Cellular Probabilistic SOM	p. 188
5.4.2 Probabilistic Regularized SOM	p. 193
5.5 Clustering of SOM	p. 202
5.6 Multi-Layer SOM for Tree-Structured data	p. 205
5.6.1 SOM Input Representation	p. 207
5.6.2 MLSOM Training	p. 210
5.6.3 MLSOM visualization and classification	p. 212
Exercises	p. 216
6 Classification and Feature Selection	p. 219
6.1 Introduction	p. 219
6.2 Support Vector Machines (SVM)	p. 223
6.2.1 Support Vector Machine Visualization (SVMV)	p. 224
6.3 Cost Function	p. 229
6.3.1 MSE and MCE Cost Functions	p. 230
6.3.2 Hybrid MCE-MSE Cost Function	p. 232
6.3.3 Implementing MCE-MSE	p. 236
6.4 Feature Selection	p. 239
6.4.1 Information Theory	p. 241
6.4.1.1 Mutual Information	p. 241
6.4.1.2 Probability density function (pdf) estimation	p. 243
6.4.2 MI Based Forward Feature Selection	p. 245
6.4.2.1 MIFS and MIFS-U	p. 247
6.4.2.2 Using quadratic MI	p. 248
Exercises	p. 253
7 Engineering Applications	p. 255
7.1 Electric Load Forecasting	p. 255
7.1.1 Nonlinear Autoregressive Integrated Neural Network Model	p. 257
7.1.2 Case Studies	p. 261
7.2 Content-based Image Retrieval Using SOM	p. 266
7.2.1 GHSOQM Based CBIR Systems	p. 267
7.2.1.1 Overall Architecture of GHSOQM-Based CBIR System	p. 267
7.2.1.2 Image Segmentation, Feature Extraction and Region-Based Feature Matrices	p. 268
7.2.1.3 Image Distance	p. 269
7.2.1.4 GHSOQM and Relevance Feedback in the CBIR System	p. 270
7.2.2 Performance Evaluation	p. 274
7.3 Feature Selection for cDNA Microarray	p. 278
7.3.1 MI Based Forward Feature Selection Scheme	p. 279
7.3.2 The Supervised Grid Based Redundancy Elimination	p. 280
7.3.3 The Forward Gene Selection Process Using MIIO and MISF	p. 281
7.3.4 Results	p. 282
7.3.4.1 Prostate Cancer Classification Dataset	p. 284
7.3.4.2 Subtype of ALL Classification Dataset	p. 288
7.3.5 Remarks	p. 294
Bibliography	p. 291
Index	p. 305

Available:*

On Order

Summary

Summary

Table of Contents