Markov chains and stochastic stability

Select an Action

Place Hold(s)
Add to My Lists
Email
Print

Title:

Personal Author:

Meyn, Sean

Series Title:

Communications and control engineering series

Series:

Communications and control engineering series

Edition:

2nd ed.

Publication Information:

New York : Cambridge University Press, 2009

Physical Description:

xxviii, 594 p. : ill. ; 24 cm.

ISBN:

9780521731829

Subject Term:

Markov processes

Added Author:

Tweedie, R. L. (Richard L.)

Available:*

Library	Item Barcode	Call Number	Material Type	Item Category 1	Status
Searching... PSZ JB	30000010229734	QA274.7 M49 2009	Open Access Book	Book	Searching... Unknown

Meyn and Tweedie is back! The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

List of figures	p. xi
Prologue to the Second edition, Peter W. Glynn	p. xiii
Preface to the second edition, Sean Meyn	p. xvii
Preface to the first edition	p. xxi
I Communication and Regeneration	p. 1
1 Heuristics	p. 3
1.1 A range of Markovian environments	p. 3
1.2 Basic models in practice	p. 6
1.3 Stochastic stability for Markov models	p. 13
1.4 Commentary	p. 19
2 Markov models	p. 21
2.1 Markov models in time series	p. 22
2.2 Nonlinear state space models	p. *26
2.3 Models in control and systems theory	p. 33
2.4 Markov models with regeneration times	p. 38
2.5 Commentary	p. *46
3 Transition probabilities	p. 48
3.1 Defining a Markovian Process	p. 49
3.2 Foundations on a countable space	p. 51
3.3 Specific transition matrices	p. 54
3.4 Foundations for general state space chains	p. 59
3.5 Building transition kernels for specific models	p. 67
3.6 Commentary	p. 72
4 Irreducibility	p. 75
4.1 Communication and irreducibility: Countable spaces	p. 76
4.2 ¿-Irreducibility	p. 81
4.3 ¿-Irreducibility for random walk models	p. 87
4.4 ¿-Irreducible linear models	p. 89
4.5 Commentary	p. 93
5 Pseudo-atoms	p. 96
5.1 Splitting ¿-irreducible chains	p. 97
5.2 Small sets	p. 102
5.3 Small sets for specific models	p. 106
5.4 Cyclic behavior	p. 110
5.5 Petite sets and sampled chains	p. 115
5.6 Commentary	p. 121
6 Topology and continuity	p. 123
6.1 Feller properties and forms of stability	p. 125
6.2 T-chains	p. 130
6.3 Continuous components for specific models	p. 134
6.4 e-Chains	p. 139
6.5 Commentary	p. 144
7 The nonlinear state space model	p. 146
7.1 Forward accessibility and continuous components	p. 147
7.2 Minimal sets and irreducibility	p. 154
7.3 Periodicity for nonlinear state space models	p. 157
7.4 Forward accessible examples	p. 161
7.5 Equicontinuity and the nonlinear state space model	p. 163
7.6 Commentary	p. *165
II Stability Structures	p. 169
8 Transience and recurrence	p. 171
8.1 Classifying chains on countable spaces	p. 173
8.2 Classifying ¿-irreducible chains	p. 177
8.3 Recurrence and transience relationships	p. 182
8.4 Classification using drift criteria	p. 187
8.5 Classifying random walk on R+	p. 193
8.6 Commentary	p. *197
9 Harris and topological recurrence	p. 199
9.1 Harris recurrence	p. 201
9.2 Non-evanescent and recurrent chains	p. 206
9.3 Topologically recurrent and transient states	p. 208
9.4 Criteria for stability on a topological space	p. 213
9.5 Stochastic comparison and increment analysis	p. 218
9.6 Commentary	p. 228
10 The existence of ¿	p. 229
10.1 Stationarity and invariance	p. 230
10.2 The existence of ¿: chains with atoms	p. 234
10.3 Invariant measures for countable space models	p. *236
10.4 The existence of ¿: ¿-irreducible chains	p. 241
10.5 Invariant measures for general models	p. 247
10.6 Commentary	p. 253
11 Drift and regularity	p. 256
11.1 Regular chains	p. 258
11.2 Drift, hitting times and deterministic models	p. 261
11.3 Drift, criteria for regularity	p. 263
11.4 Using the regularity criteria	p. 272
11.5 Evaluating non-positivity	p. 278
11.6 Commentary	p. 285
12 Invariance and tightness	p. 288
12.1 Chains bounded in probability	p. 289
12.2 Generalized sampling and invariant measures	p. 292
12.3 The existence of a ¿-finite invariant measure	p. 298
12.4 Invariant measures for e-chains	p. 300
12.5 Establishing boundedness in probability	p. 305
12.6 Commentary	p. 308
III Convergence	p. 311
13 Ergodicity	p. 313
13.1 Ergodic chains on countable spaces	p. 316
13.2 Renewal and regeneration	p. 320
13.3 Ergodicity of positive Harris chains	p. 326
13.4 Sums of transition probabilities	p. 329
13.5 Commentary	p. *334
14 f-Ergodicity and f-regularity	p. 336
14.1 f-Properties: chains with atoms	p. 338
14.2 f-Regularity and drift	p. 342
14.3 f-Ergodicity for general chains	p. 349
14.4 f-Ergodicity of specific models	p. 352
14.5 A key renewal theorem	p. 354
14.6 Commentary	p. 359
15 Geometric ergodicity	p. 362
15.1 Geometric properties: chains with atoms	p. 364
15.2 Kendall sets and drift criteria	p. 372
15.3 f-Geometric regularity of ¿ and its skeleton	p. 380
15.4 f-Geometric ergodicity for general chains	p. 384
15.5 Simple random walk and linear models	p. 388
15.6 Commentary	p. *390
16 V-Uniform ergodicity	p. 392
16.1 Operator norm convergence	p. 395
16.2 Uniform ergodicity	p. 400
16.3 Geometric ergodicity and increment analysis	p. 407
16.4 Models from queueing theory	p. 411
16.5 Autoregressive and state space models	p. 414
16.6 Commentary	p. *418
17 Sample paths and limit theorems	p. 421
17.1 Invariant ¿-fields and the LLN	p. 423
17.2 Ergodic theorems for chains possessing an atom	p. 428
17.3 General Harris chains	p. 433
17.4 The functional CLT	p. 443
17.5 Criteria for the CLT and the LIL	p. 450
17.6 Applications	p. 454
17.7 Commentary	p. *456
18 Positivity	p. 462
18.1 Null recurrent chains	p. 464
18.2 Characterizing positivity using Pn	p. 469
18.3 Positivity and T-chains	p. 471
18.4 Positivity and e-chains	p. 473
18.5 The LLN for e-chains	p. 477
18.6 Commentary	p. *480
19 Generalized classification criteria	p. 482
19.1 State-dependent drifts	p. 483
19.2 History-dependent drift criteria	p. 491
19.3 Mixed drift conditions	p. 498
19.4 Commentary	p. *508
20 Epilogue to the second edition	p. 510
20.1 Geometric ergodicity and spectral theory	p. 510
20.2 Simulation and MCMC	p. 521
20.3 Continuous time models	p. 523
IV Appendices	p. 529
A Mud maps	p. 532
A.1 Recurrence versus transience	p. 532
A.2 Positivity versus nullity	p. 534
A.3 Convergence properties	p. 536
B Testing for stability	p. 538
B.1 Glossary of drift conditions	p. 538
B.2 The Scalar SETAR model: a complete classification	p. 540
C Glossary of models assumptions	p. 543
C.1 Regenerative models	p. 543
C.2 State space models	p. 546
D Some mathematical background	p. 552
D.1 Some measure theory	p. 552
D.2 Some probability theory	p. 555
D.3 Some topology	p. 556
D.4 Some real analysis	p. 557
D.5 Convergence concepts for measures	p. 558
D.6 Some martingale theory	p. 561
D.7 Some results on sequences and numbers	p. 563
Bibliography	p. 567
Indexes	p. 587
General index	p. 587
Symbols	p. 593

Available:*

On Order

Summary

Summary

Table of Contents