Understanding Markov Chains

Examples and Applications
2013
ISBN: 978-981-4451-50-5
Verlag: Springer Singapore

Buch, Englisch, 354 Seiten, Book, Format (B × H): 155 mm x 235 mm, Gewicht: 557 g

Reihe: Springer Undergraduate Mathematics Series

This book provides an undergraduate introduction to discrete and continuous-time Markov chains and their applications. A large focus is placed on the first step analysis technique and its applications to average hitting times and ruin probabilities. Classical topics such as recurrence and transience, stationary and limiting distributions, as well as branching processes, are also covered. Two major examples (gambling processes and random walks) are treated in detail from the beginning, before the general theory itself is presented in the subsequent chapters. An introduction to discrete-time martingales and their relation to ruin probabilities and mean exit times is also provided, and the book includes a chapter on spatial Poisson processes with some recent results on moment identities and deviation inequalities for Poisson stochastic integrals. The concepts presented are illustrated by examples and by 72 exercises and their complete solutions.

Zielgruppe

Upper undergraduate

Autoren/Hrsg.

Privault, Nicolas

Inhaltsverzeichnis

Introduction1 Probability Background1.1 Probability Spaces and Events1.2 Probability Measures1.3 Conditional Probabilities and Independence1.4 Random Variables1.5 Probability Distributions1.6 Expectation of a Random Variable1.7 Conditional Expectation1.8 Moment and Probability Generating FunctionsExercises2 Gambling Problems2.1 Constrained Random Walk2.2 Ruin Probabilities2.3 Mean Game DurationExercises3 Random Walk3.1 Unrestricted Random Walk3.2 Mean and Variance3.3 Distribution3.4 First Return to ZeroExercises4 Discrete-Time Markov Chains4.1 Markov Property4.2 Transition matrix4.3 Examples of Markov Chains4.4 Higher Order Transition Probabilities4.5 The Two-State Discrete-Time Markov ChainExercises5 First Step Analysis5.1 Hitting Probabilities5.2 Mean Hitting and Absorption Times5.3 First Return Times5.4 Number of ReturnsExercises6 Classication of States6.1 Communicating States6.2 Recurrent States6.3 Transient States6.4 Positive and Null Recurrence6.5 Periodicity and AperiodicityExercises7 Long-Run Behavior of Markov Chains7.1 Limiting Distributions7.2 Stationary Distributions7.3 Markov Chain Monte CarloExercises8 Branching Processes8.1 Defnition and Examples8.2 Probability Generating Functions8.3 Extinction ProbabilitiesExercises9 Continuous-Time Markov Chains9.1 The Poisson Process9.2 Continuous-Time Chains9.3 Transition Semigroup9.4 Infinitesimal Generator9.5 The Two-State Continuous-Time Markov Chain9.6 Limiting and Stationary Distributions9.7 The Discrete-Time Embedded Chain9.8 Mean Absorption Time and ProbabilitiesExercises10 Discrete-Time Martingales10.1 Filtrations and Conditional Expectations10.2 Martingales - Definition and Properties10.3 Ruin Probabilities10.4 Mean Game DurationExercises11 Spatial Poisson Processes11.1 Spatial Poisson (1781-1840) Processes11.2 Poisson Stochastic Integrals11.3 Transformations of Poisson Measures11.4 Moments of Poisson Stochastic Integrals11.5 Deviation InequalitiesExercises12 Reliability Theory12.1 Survival Probabilities12.2 Poisson Process with Time-DependentIntensity12.3 Mean Time to FailureExercisesSome Useful IdentitiesSolutions to the ExercisesReferencesIndex

Über Autor(innen)

Nicolas Privault is an associate professor from the Nanyang Technological University (NTU) and is well-established in the field of stochastic processes and a highly respected probabilist. He has authored the book, Stochastic Analysis in Discrete and Continuous Settings: With Normal Martingales, Lecture Notes in Mathematics, Springer, 2009 and was a co-editor for the book, Stochastic Analysis with Financial Applications, Progress in Probability, Vol. 65, Springer Basel, 2011. Aside from these two Springer titles, he has authored several others. He is currently teaching the course M27004-Probability Theory and Stochastic Processes at NTU. The manuscript has been developed over the years from his courses on Stochastic Processes.