Palit / Popovic Computational Intelligence in Time Series Forecasting

Theory and Engineering Applications
1. Auflage 2006
ISBN: 978-1-84628-184-6
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 372 Seiten

Reihe: Advances in Industrial Control

Foresight in an engineering business can make the difference between success and failure, and can be vital to the effective control of industrial systems. The authors of this book harness the power of intelligent technologies individually and in combination.

Autoren/Hrsg.

Palit, Ajoy K.

Popovic, Dobrivoje

Inhaltsverzeichnis

1;Series Editors’ Foreword;9
2;Preface;11
3;Contents;15
4;Part I Introduction;22
4.1;1 Computational Intelligence: An Introduction;24
4.1.1;1.1 Introduction;24
4.1.2;1.2 Soft Computing;24
4.1.3;1.3 Probabilistic Reasoning;25
4.1.4;1.4 Evolutionary Computation;27
4.1.5;1.5 Computational Intelligence;29
4.1.6;1.6 Hybrid Computational Technology;30
4.1.7;1.7 Application Areas;31
4.1.8;1.8 Applications in Industry;32
4.1.9;References;33
4.2;2 Traditional Problem Definition;38
4.2.1;2.1 Introduction to Time Series Analysis;38
4.2.2;2.2 Traditional Problem Definition;39
4.2.2.1;2.2.1 Characteristic Features;39
4.2.2.1.1;2.2.1.1 Stationarity;39
4.2.2.1.2;2.2.1.2 Linearity;41
4.2.2.1.3;2.2.1.3 Trend;41
4.2.2.1.4;2.2.1.4 Seasonality;42
4.2.2.1.5;2.2.1.5 Estimation and Elimination of Trend and Seasonality;42
4.2.3;2.3 Classification of Time Series;43
4.2.3.1;2.3.1 Linear Time Series;44
4.2.3.2;2.3.2 Nonlinear Time Series;44
4.2.3.3;2.3.3 Univariate Time Series;44
4.2.3.4;2.3.4 Multivariate Time Series;45
4.2.3.5;2.3.5 Chaotic Time Series;45
4.2.4;2.4 Time Series Analysis;46
4.2.4.1;2.4.1 Objectives of Analysis;46
4.2.4.2;2.4.2 Time Series Modelling;47
4.2.4.3;2.4.3 Time Series Models;47
4.2.5;2.5 Regressive Models;48
4.2.5.1;2.5.1 Autoregression Model;48
4.2.5.2;2.5.2 Moving-average Model;49
4.2.5.3;2.5.3 ARMA Model;49
4.2.5.4;2.5.4 ARIMA Model;50
4.2.5.5;2.5.5 CARMAX Model;53
4.2.5.6;2.5.6 Multivariate Time Series Models;54
4.2.5.7;2.5.7 Linear Time Series Models;56
4.2.5.8;2.5.8 Nonlinear Time Series Models;56
4.2.5.9;2.5.9 Chaotic Time Series Models;57
4.2.6;2.6 Time-domain Models;58
4.2.6.1;2.6.1 Transfer-function Models;58
4.2.6.2;2.6.2 State-space Models;59
4.2.7;2.7 Frequency-domain Models;60
4.2.8;2.8 Model Building;63
4.2.8.1;2.8.1 Model Identification;64
4.2.8.2;2.8.2 Model Estimation;66
4.2.8.3;2.8.3 Model Validation and Diagnostic Check;69
4.2.9;2.9 Forecasting Methods;70
4.2.9.1;2.9.1 Some Forecasting Issues;71
4.2.9.2;2.9.2 Forecasting Using Trend Analysis;72
4.2.9.3;2.9.3 Forecasting Using Regression Approaches;72
4.2.9.4;2.9.4 Forecasting Using the Box-Jenkins Method;74
4.2.9.5;2.9.5 Forecasting Using Smoothing;78
4.2.10;2.10 Application Examples;87
4.2.10.1;2.10.1 Forecasting Nonstationary Processes;87
4.2.10.2;2.10.2 Quality Prediction of Crude Oil;88
4.2.10.3;2.10.3 Production Monitoring and Failure Diagnosis;89
4.2.10.4;2.10.4 Tool Wear Monitoring;89
4.2.10.5;2.10.5 Minimum Variance Control;90
4.2.10.6;2.10.6 General Predictive Control;92
4.2.11;References;95
4.2.12;Selected Reading;95
5;Part II Basic Intelligent Computational Technologies;98
5.1;3 Neural Networks Approach;100
5.1.1;3.1 Introduction;100
5.1.2;3.2 Basic Network Architectures;101
5.1.3;3.3 Networks Used for Forecasting;105
5.1.3.1;3.3.1 Multilayer Perceptron Networks;105
5.1.3.2;3.3.2 Radial Basis Function Networks;106
5.1.3.3;3.3.3 Recurrent Networks;108
5.1.3.4;3.3.4 Counterpropagation Networks;113
5.1.3.5;3.3.5 Probabilistic Neural Networks;115
5.1.4;3.4 Network Training Methods;116
5.1.4.1;3.4.1 Accelerated Backpropagation Algorithm;120
5.1.5;3.5 Forecasting Methodology;124
5.1.5.1;3.5.1 Data Preparation for Forecasting;125
5.1.5.2;3.5.2 Determination of Network Architecture;127
5.1.5.3;3.5.3 Network Training Strategy;133
5.1.5.4;3.5.4 Training, Stopping and Evaluation;137
5.1.6;3.6 Forecasting Using Neural Networks;150
5.1.6.1;3.6.1 Neural Networks versus Traditional Forecasting;150
5.1.6.2;3.6.2. Combining Neural Networks and Traditional Approaches;152
5.1.6.3;3.6.3 Nonlinear Combination of Forecasts Using Neural Networks;153
5.1.6.4;3.6.4 Forecasting of Multivariate Time Series;157
5.1.7;References;158
5.1.8;Selected Reading;163
5.2;4 Fuzzy Logic Approach;164
5.2.1;4.1 Introduction;164
5.2.2;4.2 Fuzzy Sets and Membership Functions;165
5.2.3;4.3 Fuzzy Logic Systems;167
5.2.3.1;4.3.1 Mamdani Type of Fuzzy Logic Systems;169
5.2.3.2;4.3.2 Takagi-Sugeno Type of Fuzzy Logic Systems;169
5.2.3.3;4.3.3 Relational Fuzzy Logic System of Pedrycz;170
5.2.4;4.4 Inferencing the Fuzzy Logic System;171
5.2.4.1;4.4.1 Inferencing a Mamdani-type Fuzzy Model;171
5.2.4.2;4.4.2 Inferencing a Takagi-Sugeno type Fuzzy Model;174
5.2.4.3;4.4.3 Inferencing a (Pedrycz) Relational Fuzzy Model;175
5.2.5;4.5 Automated Generation of Fuzzy Rule Base;178
5.2.5.1;4.5.1 The Rules Generation Algorithm;178
5.2.5.2;4.5.2 Modifications Proposed for Automated Rules Generation;183
5.2.5.3;4.5.3 Estimation of Takagi-Sugeno Rule’s Consequent Parameters;187
5.2.6;4.6 Forecasting Time Series Using the Fuzzy Logic Approach;190
5.2.6.1;4.6.1 Forecasting Chaotic Time Series: An Example;190
5.2.7;4.7 Rules Generation by Clustering;194
5.2.7.1;4.7.1 Fuzzy Clustering Algorithms for Rules Generation;194
5.2.7.2;4.7.2 Fuzzy c-means Clustering;199
5.2.7.2.1;4.7.2.1 Fuzzy c-means Algorithm;200
5.2.7.2.1.1;4.7.2.1.1 Parameters of Fuzzy c-means Algorithm;201
5.2.7.3;4.7.3 Gustafson - Kessel Algorithm;204
5.2.7.3.1;4.7.3.1 Gustafson-Kessel Clustering Algorithm;205
5.2.7.3.1.1;4.7.3.1.1 Parameters of Gustafson-Kessel Algorithm;206
5.2.7.3.1.2;4.7.3.1.2 Interpretation of Cluster Covariance Matrix;206
5.2.7.4;4.7.4 Identification of Antecedent Parameters by Fuzzy Clustering;206
5.2.7.5;4.7.5 Modelling of a Nonlinear Plant;208
5.2.8;4.8 Fuzzy Model as Nonlinear Forecasts Combiner;211
5.2.9;4.9 Concluding Remarks;214
5.2.10;References;214
5.3;5 Evolutionary Computation;216
5.3.1;5.1 Introduction;216
5.3.1.1;5.1.1 The Mechanisms of Evolution;217
5.3.1.2;5.1.2 Evolutionary Algorithms;217
5.3.2;5.2 Genetic Algorithms;218
5.3.2.1;5.2.1 Genetic Operators;219
5.3.2.1.1;5.2.1.1 Selection;220
5.3.2.1.2;5.2.1.2 Reproduction;220
5.3.2.1.3;5.2.1.3 Mutation;220
5.3.2.1.4;5.2.1.4 Crossover;222
5.3.2.2;5.2.2 Auxiliary Genetic Operators;222
5.3.2.2.1;5.2.2.1 Fitness Windowing or Scaling;222
5.3.2.3;5.2.3 Real-coded Genetic Algorithms;224
5.3.2.3.1;5.2.3.1 Real Genetic Operators;225
5.3.2.3.1.1;5.2.3.1.1 Selection Function;225
5.3.2.3.1.2;5.2.3.1.2 Crossover Operators for Real-coded Genetic Algorithms;226
5.3.2.3.1.3;5.2.3.1.3 Mutation Operators;226
5.3.2.4;5.2.4 Forecasting Example;227
5.3.3;5.3 Genetic Programming;230
5.3.3.1;5.3.1 Initialization;231
5.3.3.2;5.3.2 Execution of Algorithm;232
5.3.3.3;5.3.3 Fitness Measure;232
5.3.3.4;5.3.4 Improved Genetic Versions;232
5.3.3.5;5.3.5 Applications;233
5.3.4;5.4 Evolutionary Strategies;233
5.3.4.1;5.4.1 Applications to Real-world Problems;234
5.3.5;5.5 Evolutionary Programming;235
5.3.5.1;5.5.1 Evolutionary Programming Mechanism;236
5.3.6;5.6 Differential Evolution;236
5.3.6.1;5.6.1 First Variant of Differential Evolution (DE1);237
5.3.6.2;5.6.2 Second Variant of Differential Evolution (DE2);239
5.3.7;References;239
6;Part III Hybrid Computational Technologies;242
6.1;6 Neuro-fuzzy Approach;244
6.1.1;6.1 Motivation for Technology Merging;244
6.1.2;6.2 Neuro-fuzzy Modelling;245
6.1.2.1;6.2.1 Fuzzy Neurons;248
6.1.2.1.1;6.2.1.1 AND Fuzzy Neuron;249
6.1.2.1.2;6.2.1.2 OR Fuzzy Neuron;250
6.1.3;6.3 Neuro-fuzzy System Selection for Forecasting;251
6.1.4;6.4 Takagi-Sugeno-type Neuro-fuzzy Network;253
6.1.4.1;6.4.1 Neural Network Representation of Fuzzy Logic Systems;254
6.1.4.2;6.4.2 Training Algorithm for Neuro-fuzzy Network;255
6.1.4.2.1;6.4.2.1 Backpropagation Training of Takagi-Sugeno-type Neuro-fuzzy Network;255
6.1.4.2.2;6.4.2.2 Improved Backpropagation Training Algorithm;259
6.1.4.2.3;6.4.2.3 Levenberg-Marquardt Training Algorithm;260
6.1.4.2.3.1;6.4.2.3.1 Computation of Jacobian Matrix;262
6.1.4.2.4;6.4.2.4 Adaptive Learning Rate and Oscillation Control;267
6.1.5;6.5 Comparison of Radial Basis Function Network and Neurofuzzy Network;268
6.1.6;6.6 Comparison of Neural Network and Neuro-fuzzy Network Training;269
6.1.7;6.7 Modelling and Identification of Nonlinear Dynamics;270
6.1.7.1;6.7.1 Short-term Forecasting of Electrical Load;270
6.1.7.2;6.7.2 Prediction of Chaotic Time Series;274
6.1.7.3;6.7.3 Modelling and Prediction of Wang Data;279
6.1.8;6.8 Other Engineering Application Examples;285
6.1.8.1;6.8.1 Application of Neuro-fuzzy Modelling to Material Property Prediction;286
6.1.8.2;6.8.2 Correction of Pyrometer Reading;287
6.1.8.3;6.8.3 Application for Tool Wear Monitoring;289
6.1.9;6.9 Concluding Remarks;291
6.1.10;References;292
6.2;7 Transparent Fuzzy/Neuro-fuzzy Modelling;296
6.2.1;7.1 Introduction;296
6.2.2;7.2 Model Transparency and Compactness;297
6.2.3;7.3 Fuzzy Modelling with Enhanced Transparency;298
6.2.3.1;7.3.1 Redundancy in Numerical Data-driven Modelling;298
6.2.3.2;7.3.2 Compact and Transparent Modelling Scheme;300
6.2.4;7.4 Similarity Between Fuzzy Sets;302
6.2.4.1;7.4.1 Similarity Measure;303
6.2.4.2;7.4.2 Similarity-based Rule Base Simplification;303
6.2.5;7.5 Simplification of Rule Base;306
6.2.5.1;7.5.1 Merging Similar Fuzzy Sets;308
6.2.5.2;7.5.2 Removing Irrelevant Fuzzy Sets;310
6.2.5.3;7.5.3 Removing Redundant Inputs;311
6.2.5.4;7.5.4 Merging Rules;311
6.2.6;7.6 Rule Base Simplification Algorithms;312
6.2.6.1;7.6.1 Iterative Merging;313
6.2.6.2;7.6.2 Similarity Relations;315
6.2.7;7.7 Model Competitive Issues: Accuracy versus Complexity;317
6.2.8;7.8 Application Examples;320
6.2.9;7.9 Concluding Remarks;323
6.2.10;References;323
6.3;8 Evolving Neural and Fuzzy Systems;326
6.3.1;8.1 Introduction;326
6.3.1.1;8.1.1 Evolving Neural Networks;326
6.3.1.1.1;8.1.1.1 Evolving Connection Weights;327
6.3.1.1.2;8.1.1.2 Evolving the Network Architecture;330
6.3.1.1.3;8.1.1.3 Evolving the Pure Network Architecture;331
6.3.1.1.4;8.1.1.4 Evolving Complete Network;332
6.3.1.1.5;8.1.1.5 Evolving the Activation Function;333
6.3.1.1.6;8.1.1.6 Application Examples;334
6.3.1.2;8.1.2 Evolving Fuzzy Logic Systems;334
6.3.2;References;338
6.4;9 Adaptive Genetic Algorithms;342
6.4.1;9.1 Introduction;342
6.4.2;9.2 Genetic Algorithms Parameters to Be Adapted;343
6.4.3;9.3 Probabilistic Control of Genetic Algorithms Parameters;344
6.4.4;9.4 Adaptation of Population Size;348
6.4.5;9.5 Fuzzy Logic Controlled Genetic Algoithms;350
6.4.6;9.6 Concluding Remarks;351
6.4.7;References;351
7;Part IV Recent Developments;354
7.1;10 State of the Art and Development Trend;355
7.1.1;10.1 Introduction;355
7.1.2;10.2 Support Vector Machines;358
7.1.2.1;10.2.1 Data-dependent Representation;363
7.1.2.2;10.2.2 Machine Implementation;364
7.1.2.3;10.2.3 Applications;365
7.1.3;10.3 Wavelet Networks;366
7.1.3.1;10.3.1 Wavelet Theory;366
7.1.3.2;10.3.2 Wavelet Neural Networks;367
7.1.3.3;10.3.3 Applications;370
7.1.4;10.4 Fractally Configured Neural Networks;371
7.1.5;10.5 Fuzzy Clustering;373
7.1.5.1;10.5.1 Fuzzy Clustering Using Kohonen Networks;374
7.1.5.2;10.5.2 Entropy-based Fuzzy Clustering;376
7.1.5.2.1;10.5.2.1 Entropy Measure for Cluster Estimation;377
7.1.5.2.1.1;10.5.2.1.1 The Entropy Measure;377
7.1.5.2.2;10.5.2.2 Fuzzy Clustering Based on Entropy Measure;379
7.1.5.2.3;10.5.2.3 Fuzzy Model Identification Using Entropy-based Fuzzy Clustering;380
7.1.6;References;381
8;Index;384

Leseproben

6 Neuro-fuzzy Approach (p.223)

6.1 Motivation for Technology Merging

Contemporary intelligent technologies have various characteristic features that can be used to implement systems that mimic the behaviour of human beings. For example, expert systems are capable of reasoning about the facts and situations using the rules out of a specific domain, etc. The outstanding feature of neural networks is their capability of learning, which can help in building artificial systems for pattern recognition, classification, etc. Fuzzy logic systems, again, are capable of interpreting the imprecise data that can be helpful in making possible decisions. On the other hand, genetic algorithms provide implementation of random, parallel solution search procedures within a large search space.

Therefore, in fact, the complementary features of individual categories of intelligent technologies make them ideal for isolated use in solving some specific problems, but not well suited for solving other kinds of intelligent problem. For example, the black-box modelling approach through neural networks is evidently well suited for process modelling or for intelligent control, but less suitable for decision making. On the other hand, the fuzzy logic systems can easily handle imprecise data, and explain their decisions in the context of the available facts in linguistic form; however, they cannot automatically acquire the linguistic rules to make those decisions. Such capabilities and restrictions of individual intelligent technologies have actually been a central driving force behind their fusion for creation of hybrid intelligent systems capable of solving many complex problems.

The permanent growing interest in intelligent technology merging, particularly in merging of neural and fuzzy technology, the two technologies that complement each other (Bezdek, 1993), to create neuro-fuzzy or fuzzy-neural structures, has largely extended the capabilities of both technologies in hybrid intelligent systems. The advantages of neural networks in learning and adaptation and those of fuzzy logic systems in dealing with the issues of human-like reasoning on a linguistic level, transparency and interpretability of the generated model, and handling of uncertain or imprecise data, enable building of higher level intelligent systems. The synergism of integrating neural networks with fuzzy logic technology into a hybrid functional system with low-level learning and high-level reasoning transforms the burden of the tedious design problems of the fuzzy logic decision systems to the learning of connectionist neural networks. In this way the approximation capability and the overall performance of the resulting system are enhanced.

A number of different schemes and architectures of this hybrid system have been proposed, such as fuzzy-logic-based neurons (Pedrycz, 1995), fuzzy neurons (Gupta, 1994), neural networks with fuzzy weights (Buckley and Hayashi, 1994), neuro-fuzzy adaptive models (Brown and Harris, 1994), etc. The proposed architectures have been successful in solving various engineering and real-world problems, such as in applications like system identification and modelling, process control, systems diagnosis, cognitive simulation, classification, pattern recognition, image processing, engineering design, financial trading, signal processing, time series prediction and forecasting, etc.