Ebook: Markov Chains and Stochastic Stability

Markov Chains and Stochastic Stability is part of the Communications and Control Engineering Series (CCES) edited by Professors B.W. Dickinson, E.D. Sontag, M. Thoma, A. Fettweis, J.L. Massey and J.W. Modestino. The area of Markov chain theory and application has matured over the past 20 years into something more accessible and complete. It is of increasing interest and importance. This publication deals with the action of Markov chains on general state spaces. It discusses the theories and the use to be gained, concentrating on the areas of engineering, operations research and control theory. Throughout, the theme of stochastic stability and the search for practical methods of verifying such stability, provide a new and powerful technique. This does not only affect applications but also the development of the theory itself. The impact of the theory on specific models is discussed in detail, in order to provide examples as well as to demonstrate the importance of these models. Markov Chains and Stochastic Stability can be used as a textbook on applied Markov chain theory, provided that one concentrates on the main aspects only. It is also of benefit to graduate students with a standard background in countable space stochastic models. Finally, the book can serve as a research resource and active tool for practitioners.

---

---

Content:
Front Matter....Pages i-xvi
Front Matter....Pages 1-1
Heuristics....Pages 3-22
Markov Models....Pages 23-53
Transition Probabilities....Pages 54-79
Irreducibility....Pages 80-99
Pseudo-Atoms....Pages 100-125
Topology and Continuity....Pages 126-148
The Nonlinear State Space Model....Pages 149-169
Front Matter....Pages 171-171
Transience and Recurrence....Pages 173-199
Harris and Topological Recurrence....Pages 200-228
The Existence of ?....Pages 229-254
Drift and Regularity....Pages 255-284
Invariance and Tightness....Pages 285-305
Front Matter....Pages 307-307
Ergodicity....Pages 309-329
f-Ergodicity and f-Regularity....Pages 330-353
Geometric Ergodicity....Pages 354-381
V-Uniform Ergodicity....Pages 382-409
Sample Paths and Limit Theorems....Pages 410-445
Positivity....Pages 446-464
Generalized Classification Criteria....Pages 465-491
Back Matter....Pages 493-552

---

Content:
Front Matter....Pages i-xvi
Front Matter....Pages 1-1
Heuristics....Pages 3-22
Markov Models....Pages 23-53
Transition Probabilities....Pages 54-79
Irreducibility....Pages 80-99
Pseudo-Atoms....Pages 100-125
Topology and Continuity....Pages 126-148
The Nonlinear State Space Model....Pages 149-169
Front Matter....Pages 171-171
Transience and Recurrence....Pages 173-199
Harris and Topological Recurrence....Pages 200-228
The Existence of ?....Pages 229-254
Drift and Regularity....Pages 255-284
Invariance and Tightness....Pages 285-305
Front Matter....Pages 307-307
Ergodicity....Pages 309-329
f-Ergodicity and f-Regularity....Pages 330-353
Geometric Ergodicity....Pages 354-381
V-Uniform Ergodicity....Pages 382-409
Sample Paths and Limit Theorems....Pages 410-445
Positivity....Pages 446-464
Generalized Classification Criteria....Pages 465-491
Back Matter....Pages 493-552
....