Ebook: Competitive Markov Decision Processes

This book is intended as a text covering the central concepts and techniques of Competitive Markov Decision Processes. It is an attempt to present a rig­ orous treatment that combines two significant research topics: Stochastic Games and Markov Decision Processes, which have been studied exten­ sively, and at times quite independently, by mathematicians, operations researchers, engineers, and economists. Since Markov decision processes can be viewed as a special noncompeti­ tive case of stochastic games, we introduce the new terminology Competi­ tive Markov Decision Processes that emphasizes the importance of the link between these two topics and of the properties of the underlying Markov processes. The book is designed to be used either in a classroom or for self-study by a mathematically mature reader. In the Introduction (Chapter 1) we outline a number of advanced undergraduate and graduate courses for which this book could usefully serve as a text. A characteristic feature of competitive Markov decision processes - and one that inspired our long-standing interest - is that they can serve as an "orchestra" containing the "instruments" of much of modern applied (and at times even pure) mathematics. They constitute a topic where the instruments of linear algebra, applied probability, mathematical program­ ming, analysis, and even algebraic geometry can be "played" sometimes solo and sometimes in harmony to produce either beautifully simple or equally beautiful, but baroque, melodies, that is, theorems.

---

This book is devoted to a unified treatment of Competitive Markov Decision Processes. It examines these processes from the standpoints of modeling and of optimization, providing newcomers to the field with an accessible account of algorithms, theory, and applications, while also supplying specialists with a comprehensive survey of recent developments. The treatment is self-contained, requiring only some knowledge of linear algebra and real analysis. Topics covered include: Mathematical programming: Markov decision processes (the non-competitive case), and stochastic games via mathematical programming.- Existence, structure and applications: Summable stochastic games, average-reward stochastic games and applications and special classes of stochastic games.- Appendices on: matrix games, bimatrix games and nonlinear programming; a theorem of Hardy and Littlewood; Markov chains; and complex varieties and the limit discount equation.

---

This book is devoted to a unified treatment of Competitive Markov Decision Processes. It examines these processes from the standpoints of modeling and of optimization, providing newcomers to the field with an accessible account of algorithms, theory, and applications, while also supplying specialists with a comprehensive survey of recent developments. The treatment is self-contained, requiring only some knowledge of linear algebra and real analysis. Topics covered include: Mathematical programming: Markov decision processes (the non-competitive case), and stochastic games via mathematical programming.- Existence, structure and applications: Summable stochastic games, average-reward stochastic games and applications and special classes of stochastic games.- Appendices on: matrix games, bimatrix games and nonlinear programming; a theorem of Hardy and Littlewood; Markov chains; and complex varieties and the limit discount equation.
Content:
Front Matter....Pages i-xii
Introduction....Pages 1-6
Front Matter....Pages 7-7
Markov Decision Processes: The Noncompetitive Case....Pages 9-84
Stochastic Games via Mathematical Programming....Pages 85-151
Front Matter....Pages 153-153
Summable Stochastic Games....Pages 155-234
Average Reward Stochastic Games....Pages 235-300
Applications and Special Classes of Stochastic Games....Pages 301-341
Back Matter....Pages 343-393

---

This book is devoted to a unified treatment of Competitive Markov Decision Processes. It examines these processes from the standpoints of modeling and of optimization, providing newcomers to the field with an accessible account of algorithms, theory, and applications, while also supplying specialists with a comprehensive survey of recent developments. The treatment is self-contained, requiring only some knowledge of linear algebra and real analysis. Topics covered include: Mathematical programming: Markov decision processes (the non-competitive case), and stochastic games via mathematical programming.- Existence, structure and applications: Summable stochastic games, average-reward stochastic games and applications and special classes of stochastic games.- Appendices on: matrix games, bimatrix games and nonlinear programming; a theorem of Hardy and Littlewood; Markov chains; and complex varieties and the limit discount equation.
Content:
Front Matter....Pages i-xii
Introduction....Pages 1-6
Front Matter....Pages 7-7
Markov Decision Processes: The Noncompetitive Case....Pages 9-84
Stochastic Games via Mathematical Programming....Pages 85-151
Front Matter....Pages 153-153
Summable Stochastic Games....Pages 155-234
Average Reward Stochastic Games....Pages 235-300
Applications and Special Classes of Stochastic Games....Pages 301-341
Back Matter....Pages 343-393
....