Sethi / Thompson Optimal Control Theory

Applications to Management Science and Economics
2. Auflage 2000
ISBN: 978-0-387-29903-7
Verlag: Springer US
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 504 Seiten, eBook

Optimal control methods are used to determine optimal ways to control a dynamic system. The theoretical work in this field serves as a foundation for the book, which the authors have applied to business management problems developed from their research and classroom instruction.Sethi and Thompson have provided management science and economics communities with a thoroughly revised edition of their classic text on Optimal Control Theory. The new edition has been completely refined with careful attention to the text and graphic material presentation. Chapters cover a range of topics including finance, production and inventory problems, marketing problems, machine maintenance and replacement, problems of optimal consumption of natural resources, and applications of control theory to economics. The book contains new results that were not available when the first edition was published, as well as an expansion of the material on stochastic optimal control theory.

Zielgruppe

Professional/practitioner

Autoren/Hrsg.

Sethi, Suresh P.

Thompson, Gerald L.

Inhaltsverzeichnis

1;Contents;7
2;Preface to First Edition;13
3;Preface to Second Edition;15
4;1 What is Optimal Control Theory?;18
4.1;1.1 Basic Concepts and Definitions;19
4.2;1.2 Formulation of Simple Control Models;21
4.3;1.3 History of Optimal Control Theory;24
4.4;1.4 Notation and Concepts Used;27
5;2 The Maximum Principle: Continuous Time;40
5.1;2.1 Statement of the Problem;40
5.1.1;2.1.1 The Mathematical Model;41
5.1.2;2.1.2 Constraints;41
5.1.3;2.1.3 The Objective Function;42
5.1.4;2.1.4 The Optimal Control Problem;42
5.2;2.2 Dynamic Programming and the Maximum Principle;44
5.2.1;2.2.1 The Hamilton-Jacobi-Bellman Equation;44
5.2.2;2.2.2 Derivation of the Adjoint Equation;48
5.2.3;2.2.3 The Maximum Principle;50
5.2.4;2.2.4 Economic Interpretations of the Maximum Principle;51
5.3;2.3 Elementary Examples;53
5.4;2.4 Sufficiency Conditions;61
5.5;2.5 Solving a TPBVP by Using Spreadsheet Software;65
6;3 The Maximum Principle: Mixed Inequality Constraints;74
6.1;3.1 A Maximum Principle for Problem.s with Mixed Inequality Constraints;75
6.2;3.2 Sufficiency Conditions;81
6.3;3.3 Current-Value Formulation;82
6.4;3.4 Terminal Conditions;86
6.4.1;3.4.1 Examples Illustrating Terminal Conditions;91
6.5;3.5 Infinite Horizon and Stationarity;97
6.6;3.6 Model Types;100
7;4 The Maximum Principle: General Inequality Constraints;114
7.1;4.1 Pure State Variable Inequality Constraints: Indirect Method;115
7.1.1;4.1.1 Jump Conditions;120
7.2;4.2 A Maximum Principle: Indirect Method;121
7.3;4.3 Current-Value Maximum Principle: Indirect Method;128
7.4;4.4 Sufficiency Conditions;130
8;5 Applications to Finance;136
8.1;5.1 The Simple Cash Balance Problem;137
8.1.1;5.1.1 The Model;137
8.1.2;5.1.2 Solution by the Maximum Principle;138
8.1.3;5.1.3 An Extension DisalloAving Overdraft and Short-Selling;141
8.2;5.2 Optimal Financing Model;146
8.2.1;5.2.1 The Model;146
8.2.2;5.2.2 Application of the Maximum Principle;148
8.2.3;5.2.3 Synthesis of Optimal Control Paths;150
8.2.4;5.2.4 Solution for the Infinite Horizon Problem;161
9;6 Applications to Production and Inventory;170
9.1;6.1 A Production-Inventory System;171
9.1.1;6.1.1 The Production-Inventory Model;171
9.1.2;6.1.2 Solution by the Maximum Principle;173
9.1.3;6.1.3 The Infinite Horizon Solution;176
9.1.4;6.1.4 A Complete Analysis of the Constant Positive S Case with Infinite Horizon;177
9.1.5;6.1.5 Special Cases of Time Varying Demands;179
9.2;6.2 Continuous Wheat Trading Model;181
9.2.1;6.2.1 The Model;182
9.2.2;6.2.2 Solution by the Maximum Principle;183
9.2.3;6.2.3 Complete Solution of a Special Case;184
9.2.4;6.2.4 The Wheat Trading Model with No Short-SeUing;187
9.3;6.3 Decision Horizons and Forecast Horizons;190
9.3.1;6.3.1 Horizons for the Wheat Trading Model;191
9.3.2;6.3.2 Horizons for the Wheat Trading Model with Warehousing Constraint;192
10;7 Applications to Marketing;202
10.1;7.1 The Nerlove-Arrow Advertising Model;203
10.1.1;7.1.1 The Model;203
10.1.2;7.1.2 Solution by the Maximum Principle;205
10.1.3;7.1.3 A Nonlinear Extension;208
10.2;7.2 The Vidale-Wolfe Advertising Model;211
10.2.1;7.2.1 Optimal Control Formulation for the Vidale-Wolfe Model;212
10.2.2;7.2.2 Solution Using Green's Theorem when Q is Large;213
10.2.3;7.2.3 Solution When Q Is Small;222
10.2.4;7.2.4 Solution When T Is Infinite;223
11;8 The Maximum Principle: Discrete Time;234
11.1;8.1 Nonlinear Programming Problems;234
11.1.1;8.1.1 Lagrange Multipliers;235
11.1.2;8.1.2 Inequality Constraints;237
11.1.3;8.1.3 Theorems from Nonlinear Programming;244
11.2;8.2 A Discrete Maximum Principle;245
11.2.1;8.2.1 A Discrete-Time Optimal Control Problem;245
11.2.2;8.2.2 A Discrete Maximum Principle;246
11.2.3;8.2.3 Examples;248
11.3;8.3 A General Discrete Maximum Principle;251
12;9 Maintenance and Replacement;258
12.1;9.1 A Simple Maintenance and Replacement Model;259
12.1.1;9.1.1 The Model;259
12.1.2;9.1.2 Solution by the Maximum Principle;260
12.1.3;9.1.3 A Numerical Example;262
12.1.4;9.1.4 An Extension;264
12.2;9.2 Maintenance and Replacement for a Machine Subject to Failure;265
12.2.1;9.2.1 The Model;266
12.2.2;9.2.2 Optimal Policy;268
12.2.3;9.2.3 Determination of the Sale Date;270
12.3;9.3 Chain of Machines;271
12.3.1;9.3.1 The Model;271
12.3.2;9.3.2 Solution by the Discrete Maximum Principle;273
12.3.3;9.3.3 Special Case of Bang-Being Control;274
12.3.4;9.3.4 Incorporation into the Wagner-Whitin Framework for a Complete Solution;275
12.3.5;9.3.5 A Numerical Example;276
13;10 Applications to Natural Resources;284
13.1;10.1 The Sole Owner Fishery Resource Model;285
13.1.1;10.1.1 The Dynamics of Fishery Models;285
13.1.2;10.1.2 The Sole Owner Model;286
13.1.3;10.1.3 Solution by Green's Theorem;287
13.2;10.2 An Optimal Forest Thinning Model;290
13.2.1;10.2.1 The Forestry Model;290
13.2.2;10.2.2 Determination of Optimal Thinning;291
13.2.3;10.2.3 A Chain of Forests Model;293
13.3;10.3 An Exhaustible Resource Model;296
13.3.1;10.3.1 Formulation of the Model;296
13.3.2;10.3.2 Solution by the Maximum Principle;299
14;11 Economic Applications;306
14.1;11.1 Models of Optimal Economic Growth;306
14.1.1;11.1.1 An Optimal Capital Accumulation Model;307
14.1.2;11.1.2 Solution by the Maximum Principle;307
14.1.3;11.1.3 A One-Sector Model with a Growing Labor Force;308
14.1.4;11.1.4 Solution by the Maximum Principle;309
14.2;11.2 A Model of Optimal Epidemic Control;312
14.2.1;11.2.1 Formulation of the Model;312
14.2.2;11.2.2 Solution by Green's Theorem;313
14.3;11.3 A Pollution Control Model;316
14.3.1;11.3.1 Model Formulation;316
14.3.2;11.3.2 Solution by the Maximum Principle;317
14.3.3;11.3.3 Phase Diagram Analysis;318
14.4;11.4 Miscellaneous Applications;320
15;12 Differential Gaines, Distributed Systems, and Impulse Control;324
15.1;12.1 Differential Gaines;325
15.1.1;12.1.1 Two Person Zero-Sum Differential Games;325
15.1.2;12.1.2 Nonzero-Sum Differential Games;327
15.1.3;12.1.3 An Application to t h e Common-Property Fishery Resources;329
15.2;12.2 Distributed Parameter Systems;332
15.2.1;12.2.1 The Distributed Parameter Maximum Principle;334
15.2.2;12.2.2 The Cattle Ranching Problem;335
15.2.3;12.2.3 Interpretation of the Adjoint Function;339
15.3;12.3 Impulse Control;339
15.3.1;12.3.1 The Oil Driller's Problem;341
15.3.2;12.3.2 The Maximum Principle for Impulse Optimal Control;342
15.3.3;12.3.3 Solution of the Oil Driller's Problem;344
15.3.4;12.3.4 Machine Maintenance and Replacement;348
15.3.5;12.3.5 Application of the Impulse Maximum Principle;349
16;13 Stochastic Optimal Control;356
16.1;13.1 The Kalman Filter;357
16.2;13.2 Stochastic Optimal Control;362
16.3;13.3 A Stochastic Production Planning Model;364
16.3.1;13.3.1 Solution for the Production Planning Problem;367
16.4;13.4 A Stochastic Advertising Problem;369
16.5;13.5 An Optimal Consumption-Investment Problem;372
16.6;13.6 Concluding Remarks;377
17;A Solutions of Linear Differential Equations;380
17.1;A.l Linear Differential Equations with Constant Coefficients;380
17.2;A.2 Homogeneous Equations of Order One;381
17.3;A.3 Homogeneous Equations of Order Two;381
17.4;A.4 Homogeneous Equations of Order n;382
17.5;A.5 Particular Solutions of Linear D.E. with Constant Coefficients;383
17.6;A.6 Integrating Factor;385
17.7;A.7 Reduction of Higher-Order Linear Equations to Systems of First-Order Linear Equations;386
17.8;A.8 Solution of Linear Two-Point Boundary Value Problems;389
17.9;A.9 Homogeneous Partial Differential Equations;389
17.10;A. 10 Inhomogeneous Partial Differential Equations;391
17.11;A. 11 Solutions of Finite Difference Equations;392
17.11.1;A. 11.1 Changing Polynomials in Powers of k into Factorial Powers of k;393
17.11.2;A. 11.2 Changing Factorial Powers of k into Ordinary Powers of k;394
18;B Calculus of Variations and Optimal Control Theory;396
18.1;B.l The Simplest Variational Problem;396
18.2;B.2 The Euler Equation;397
18.3;B.3 The Shortest Distance Between Two Points on the Plane;400
18.4;B.4 The Brachistochrone Problem;401
18.5;B.5 The Weierstrass-Erdmann Corner Conditions;403
18.6;B.6 Legendre's Conditions: The Second Variation;405
18.7;B.7 Necessary Condition for a Strong Maximum;406
18.8;B.8 Relation to the Optimal Control Theory;407
19;C An Alternative Derivation of the Maximum Principle;410
19.1;C.l Needle-Shaped Variation;411
19.2;C.2 Derivation of the Adjoint Equation and the Maximum Principle;413
20;D Special Topics in Optimal Control;418
20.1;D.l Linear-Quadratic Problems;418
20.2;D.2 Second-Order Variations;422
20.3;D.3 Singular Control;424
21;E Answers to Selected Exercises;426
22;Bibliography;434
23;Index;500
24;List of Figures;518
25;List of Tables;522

Leseproben

Chapter 13 Stochastic Optimal Control (p.341)

In previous chapters we assumed that the state variables of the system were known with certainty. If this were not the case, the state of the system over time would be a stochastic process. In addition, it might not be possible to measure the value of the state variables at time t. In this case, one would have to measure functions of the state variables. Moreover, the measurements are usually noisy, i.e., they are subject to errors. Thus, a decision maker is faced with the problem of making good estimates of these state variables from noisy measurements on functions of them.

The process of estimating the values of the state variables is called optimal filtering, In Section 13.1, we will discuss one particular filter, called the Kalman filter and its continuous-time analogue caUed the Kalman- Bucy filter. It should be noted that while optimal filtering provides optimal estimates of the value of the state variables from noisy measurements of related quantities, no control is involved.

When a control is involved, we are faced with a stochastic optimal control problem. Here, the state of the system is represented by a controlled stochastic process. In Section 13.2, we shall formulate a stochastic optimal control problem which is governed by stochastic differential equations. We shall only consider stochastic differential equations of a type known as Ito equations. These equations arise when the state equar tions, such as those we have seen in the previous chapters, are perturbed by Markov diffusion processes. Our goal in Section 13.2 will be to synthesize optimal feedback controls for systems subject to Ito equations in a way that maximizes the expected value of a given objective function. In Section 13.3, we shall extend the production planning model Chapter 6 to allow for some uncertain disturbances. We shall obtain an optimal production policy for the stochastic production planning problem thus formulated.

In Section 13.4, we solve an optimal stochastic advertising problem explicitly. The problem is a modification as well as a stochastic extension of the optimal control problem of the Vidale-Wolfe advertising model treated in Section 7.2.4.

In Section 13.5, we wiU introduce investment decisions in the consumption model of Example 1.3. We will consider both risk-free and risky investments. Our goal will be to find optimal consumption and investment policies in order to maximize the discounted value of the utility of consumption over time. In Section 13.6, we shall conclude the chapter by mentioning other types of stochastic optimal control problems that arise in practice. In particular, production planning problems where production is done by machines that are unreliable or failure-prone, can be formulated as stochastic optimal control problems involving jimip Markov processes. Such problems are treated in Sethi and Zhang (1994a, 1994c). Karatzas and Shreve (1998) address stochastic optimal control problems in finance involving more general stochastic processes including jimip processes.