Cloete / Shi Sensitivity Analysis for Neural Networks

1. Auflage 2009
ISBN: 978-3-642-02532-7
Verlag: Springer
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 86 Seiten

Artificial neural networks are used to model systems that receive inputs and produce outputs. The relationships between the inputs and outputs and the representation parameters are critical issues in the design of related engineering systems, and sensitivity analysis concerns methods for analyzing these relationships. Perturbations of neural networks are caused by machine imprecision, and they can be simulated by embedding disturbances in the original inputs or connection weights, allowing us to study the characteristics of a function under small perturbations of its parameters. This is the first book to present a systematic description of sensitivity analysis methods for artificial neural networks. It covers sensitivity analysis of multilayer perceptron neural networks and radial basis function neural networks, two widely used models in the machine learning field. The authors examine the applications of such analysis in tasks such as feature selection, sample reduction, and network optimization. The book will be useful for engineers applying neural network sensitivity analysis to solve practical problems, and for researchers interested in foundational problems in neural networks.

Autoren/Hrsg.

Cloete, Ian

Shi, Daming

Inhaltsverzeichnis

1;Preface;5
2;Contents;7
3;1 Introduction to Neural Networks;9
3.1;1.1 Properties of Neural Networks;11
3.2;1.2 Neural Network Learning;12
3.2.1;1.2.1 Supervised Learning;13
3.2.2;1.2.2 Unsupervised Learning;13
3.3;1.3 Perceptron;14
3.4;1.4 Adaline and Least Mean Square Algorithm;16
3.5;1.5 Multilayer Perceptron and Backpropagation Algorithm;17
3.5.1;1.5.1 Output Layer Learning;19
3.5.2;1.5.2 Hidden Layer Learning;19
3.6;1.6 Radial Basis Function Networks;20
3.7;1.7 Support Vector Machines;21
4;2 Principles of Sensitivity Analysis;24
4.1;2.1 Perturbations in Neural Networks;24
4.2;2.2 Neural Network Sensitivity Analysis;25
4.3;2.3 Fundamental Methods of Sensitivity Analysis;28
4.3.1;2.3.1 Geometrical Approach;28
4.3.2;2.3.2 Statistical Approach;30
4.4;2.4 Summary;31
5;3 Hyper-Rectangle Model;32
5.1;3.1 Hyper-Rectangle Model for Input Space of MLP;32
5.2;3.2 Sensitivity Measure of MLP;33
5.3;3.3 Discussion;34
6;4 Sensitivity Analysis with Parameterized Activation Function;35
6.1;4.1 Parameterized Antisymmetric Squashing Function;35
6.2;4.2 Sensitivity Measure;36
6.3;4.3 Summary;37
7;5 Localized Generalization Error Model;38
7.1;5.1 Introduction;38
7.2;5.2 The Localized Generalization Error Model;40
7.2.1;5.2.1 The Q-Neighborhood and Q-Union;41
7.2.2;5.2.2 The Localized Generalization Error Bound;41
7.2.3;5.2.3 Stochastic Sensitivity Measure for RBFNN;43
7.2.4;5.2.4 Characteristics of the Error Bound;45
7.2.5;5.2.5 Comparing Two Classifiers Using the Error Bound;47
7.3;5.3 Architecture Selection Using the Error Bound;47
7.3.1;5.3.1 Parameters for MC2SG;49
7.3.2;5.3.2 RBFNN Architecture Selection Algorithm for MC2SG;49
7.3.3;5.3.3 A Heuristic Method to Reduce the Computational Time for MC2SG;50
7.4;5.4 Summary;50
8;6 Critical Vector Learning for RBF Networks;52
8.1;6.1 Related Work;52
8.2;6.2 Construction of RBF Networks with Sensitivity Analysis;53
8.2.1;6.2.1 RBF Classifiers' Sensitivity to the Kernel Function Centers;54
8.2.2;6.2.2 Orthogonal Least Square Transform;56
8.2.3;6.2.3 Critical Vector Selection;57
8.3;6.3 Summary;57
9;7 Sensitivity Analysis of Prior Knowledge;59
9.1;7.1 KBANNs;59
9.2;7.2 Inductive Bias;60
9.3;7.3 Sensitivity Analysis and Measures;63
9.3.1;7.3.1 Output-Pattern Sensitivity;63
9.3.2;7.3.2 Output-Weight Sensitivity;64
9.3.3;7.3.3 Output-H Sensitivity;65
9.3.4;7.3.4 Euclidean Distance;65
9.4;7.4 Promoter Recognition;65
9.4.1;7.4.1 Data and Initial Domain Theory;66
9.4.2;7.4.2 Experimental Methodology;67
9.5;7.5 Discussion and Conclusion;68
10;8 Applications;72
10.1;8.1 Input Dimension Reduction;72
10.1.1;8.1.1 Sensitivity Matrix;73
10.1.2;8.1.2 Criteria for Pruning Inputs;73
10.2;8.2 Network Optimization;74
10.3;8.3 Selective Learning;77
10.4;8.4 Hardware Robustness;78
10.5;8.5 Measure of Nonlinearity;80
10.6;8.6 Parameter Tuning for Neocognitron;81
10.6.1;8.6.1 Receptive Field;82
10.6.2;8.6.2 Selectivity;83
10.6.3;8.6.3 Sensitivity Analysis of the Neocognitron;83
11;Bibliography;86