With Applications to Machine Learning
Buch, Englisch, 672 Seiten, Format (B × H): 183 mm x 260 mm, Gewicht: 1437 g
ISBN: 978-1-119-87763-9
Verlag: John Wiley & Sons Inc
Accessible introductory textbook on optimization theory and methods, with an emphasis on engineering design, featuring MATLAB® exercises and worked examples
Fully updated to reflect modern developments in the field, the Fifth Edition of An Introduction to Optimization fills the need for an accessible, yet rigorous, introduction to optimization theory and methods, featuring innovative coverage and a straightforward approach. The book begins with a review of basic definitions and notations while also providing the related fundamental background of linear algebra, geometry, and calculus.
With this foundation, the authors explore the essential topics of unconstrained optimization problems, linear programming problems, and nonlinear constrained optimization. In addition, the book includes an introduction to artificial neural networks, convex optimization, multi-objective optimization, and applications of optimization in machine learning.
Numerous diagrams and figures found throughout the book complement the written presentation of key concepts, and each chapter is followed by MATLAB® exercises and practice problems that reinforce the discussed theory and algorithms.
The Fifth Edition features a new chapter on Lagrangian (nonlinear) duality, expanded coverage on matrix games, projected gradient algorithms, machine learning, and numerous new exercises at the end of each chapter.
An Introduction to Optimization includes information on: - The mathematical definitions, notations, and relations from linear algebra, geometry, and calculus used in optimization
- Optimization algorithms, covering one-dimensional search, randomized search, and gradient, Newton, conjugate direction, and quasi-Newton methods
- Linear programming methods, covering the simplex algorithm, interior point methods, and duality - Nonlinear constrained optimization, covering theory and algorithms, convex optimization, and Lagrangian duality
- Applications of optimization in machine learning, including neural network training, classification, stochastic gradient descent, linear regression, logistic regression, support vector machines, and clustering.
An Introduction to Optimization is an ideal textbook for a one- or two-semester senior undergraduate or beginning graduate course in optimization theory and methods. The text is also of value for researchers and professionals in mathematics, operations research, electrical engineering, economics, statistics, and business.
Autoren/Hrsg.
Fachgebiete
- Wirtschaftswissenschaften Betriebswirtschaft Betriebswirtschaft: Theorie & Allgemeines
- Wirtschaftswissenschaften Betriebswirtschaft Unternehmensforschung
- Mathematik | Informatik EDV | Informatik Informatik Künstliche Intelligenz Neuronale Netzwerke
- Mathematik | Informatik Mathematik Numerik und Wissenschaftliches Rechnen Optimierung
Weitere Infos & Material
Preface xv
About the Companion Website xviii
Part I Mathematical Review 1
1 Methods of Proof and Some Notation 3
1.1 Methods of Proof 3
1.2 Notation 5
Exercises 5
2 Vector Spaces and Matrices 7
2.1 Vector and Matrix 7
2.2 Rank of a Matrix 11
2.3 Linear Equations 16
2.4 Inner Products and Norms 18
Exercises 20
3 Transformations 23
3.1 Linear Transformations 23
3.2 Eigenvalues and Eigenvectors 24
3.3 Orthogonal Projections 26
3.4 Quadratic Forms 27
3.5 Matrix Norms 32
Exercises 35
4 Concepts from Geometry 39
4.1 Line Segments 39
4.2 Hyperplanes and Linear Varieties 39
4.3 Convex Sets 41
4.4 Neighborhoods 43
4.5 Polytopes and Polyhedra 44
Exercises 45
5 Elements of Calculus 47
5.1 Sequences and Limits 47
5.2 Differentiability 52
5.3 The Derivative Matrix 54
5.4 Differentiation Rules 57
5.5 Level Sets and Gradients 58
5.6 Taylor Series 61
Exercises 65
Part II Unconstrained Optimization 67
6 Basics of Set-Constrained and Unconstrained Optimization 69
6.1 Introduction 69
6.2 Conditions for Local Minimizers 70
Exercises 78
7 One-Dimensional Search Methods 87
7.1 Introduction 87
7.2 Golden Section Search 87
7.3 Fibonacci Method 91
7.4 Bisection Method 97
7.5 Newton’s Method 98
7.6 Secant Method 101
7.7 Bracketing 103
7.8 Line Search in Multidimensional Optimization 103
Exercises 105
8 Gradient Methods 109
8.1 Introduction 109
8.2 Steepest Descent Method 110
8.3 Analysis of Gradient Methods 117
Exercises 126
9 Newton’s Method 133
9.1 Introduction 133
9.2 Analysis of Newton’s Method 135
9.3 Levenberg–Marquardt Modification 138
9.4 Newton’s Method for Nonlinear Least Squares 139
Exercises 142
10 Conjugate Direction Methods 145
10.1 Introduction 145
10.2 Conjugate Direction Algorithm 146
10.2.1 Basic Conjugate Direction Algorithm 146
10.3 Conjugate Gradient Algorithm 151
10.4 Conjugate Gradient Algorithm for Nonquadratic Problems 154
Exercises 156
11 Quasi-Newton Methods 159
11.1 Introduction 159
11.2 Approximating the Inverse Hessian 160
11.3 Rank One Correction Formula 162
11.4 DFP Algorithm 166
11.5 BFGS Algorithm 170
Exercises 173
12 Solving Linear Equations 179
12.1 Least-Squares Analysis 179
12.2 Recursive Least-Squares Algorithm 187
12.3 Solution to a Linear Equation with Minimum Norm 190
12.4 Kaczmarz’s Algorithm 191
12.5 Solving Linear Equations in General 194
Exercises 201
13 Unconstrained Optimization and Neural Networks 209
13.1 Introduction 209
13.2 Single-Neuron Training 211
13.3 Backpropagation Algorithm 213
Exercises 222
14 Global Search Algorithms 225
14.1 Introduction 225
14.2 Nelder–Mead Simplex Algorithm 225
14.3 Simulated Annealing 229
14.3.1 Randomized Search 229
14.3.2 Simulated Annealing Algorithm 229
14.4 Particle Swarm Optimization 231
14.4.1 Basic PSO Algorithm 232
14.4.2 Variations 233
14.5 Genetic Algorithms 233
14.5.1 Basic Description 233
14.5.1.1 Chromosomes and Representation Schemes 234
14.5.1.2 Selection and Evolution 234
14.5.2 Analysis of Genetic Algorithms 238
14.5.3 Real-Number Genetic Algorithms 243
Exercises 244
Part III Linear Programming 247
15 Introduction to Linear Programming 249
15.1 Brief History of Linear Programming 249
15.2 Simple Examples of Linear Programs 250
15.3 Two-Dimensional Linear Programs 256
15.4 Convex Polyhedra and Linear Programming 258
15.5 Standard Form Linear Programs 260
15.6 Basic Solutions 264
15.7 Properties of Basic Solutions 267
15.8 Geometric View of Linear Programs 269
Exercises 273
16 Simplex Method 277
16.1 Solving Linear Equations Using Row Operations 277
16.2 The Canonical Augmented Matrix 283
16.3 Updating the Augmented Matrix 284
16.4 The Simplex Algorithm 285
16.5 Matrix Form of the Simplex Method 291
16.6 Two-Phase Simplex Method 294
16.7 Revised Simplex Method 297
Exercises 301
17 Duality 309
17.1 Dual Linear Programs 309
17.2 Properties of Dual Problems 316
17.3 Matrix Games 321
Exercises 324
18 Nonsimplex Methods 331
18.1 Introduction 331
18.2 Khachiyan’s Method 332
18.3 Affine Scaling Method 334
18.3.1 Basic Algorithm 334
18.3.2 Two-Phase Method 337
18.4 Karmarkar’s Method 339
18.4.1 Basic Ideas 339
18.4.2 Karmarkar’s Canonical Form 339
18.4.3 Karmarkar’s Restricted Problem 341
18.4.4 From General Form to Karmarkar’s Canonical Form 342
18.4.5 The Algorithm 345
Exercises 349
19 Integer Linear Programming 351
19.1 Introduction 351
19.2 Unimodular Matrices 351
19.3 The Gomory Cutting-Plane Method 358
Exercises 366
Part IV Nonlinear Constrained Optimization 369
20 Problems with Equality Constraints 371
20.1 Introduction 371
20.2 Problem Formulation 373
20.3 Tangent and Normal Spaces 374
20.4 Lagrange Condition 379
20.5 Second-Order Conditions 387
20.6 Minimizing Quadratics Subject to Linear Constraints 390
Exercises 394
21 Problems with Inequality Constraints 399
21.1 Karush–Kuhn–Tucker Condition 399
21.2 Second-Order Conditions 406
Exercises 410
22 Convex Optimization Problems 417
