Partially Observed Markov Decision Processes: Filtering, Learning and Controlled

The Nile on FREE SHIPPING UK WIDE
Partially Observed Markov Decision Processes by Vikram Krishnamurthy

This survey of formulation, algorithms, and structural results in POMDPs focuses on underlying concepts and connections to real-world applications in controlled sensing, keeping technical machinery to a minimum. The new edition includes inverse reinforcement learning, non-parametric Bayesian inference, variational Bayes and conformal prediction.

FORMAT
Hardcover CONDITION
Brand New Publisher Description

Covering formulation, algorithms and structural results and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. In light of major advances in machine learning over the past decade, this edition includes a new Part V on inverse reinforcement learning as well as a new chapter on non-parametric Bayesian inference (for Dirichlet processes and Gaussian processes), variational Bayes and conformal prediction.

Table of Contents

Preface to revised edition; Notation; 1. Introduction; I. Stochastic Models and Bayesian Filtering: 2. Stochastic state space model; 3. Optimal filtering; 4. Algorithms for maximum likelihood parameter estimation; 5. Multi-agent sensing: social learning and data incest; 6. Nonparametric Bayesian inference; II. POMDPs: Models and Applications: 7. Fully observed Markov decision processes; 8. Partially observed Markov decision processes; 9. POMDPs in controlled sensing and sensor scheduling; III. POMDP Structural Results: 10. Structural results for Markov decision processes; 11. Structural results for optimal filters; 12. Monotonicity of value function for POMDPs; 13. Structural results for stopping-time POMDPs; 14. Stopping-Time POMDPs for quickest detection; 15. Myopic policy bounds for POMDPs and sensitivity to model parameters; IV. Stochastic Gradient Algorithms and Reinforcement Learning: 16. Stochastic optimization and gradient estimation; 17. Reinforcement learning; 18. Stochastic gradient algorithms: convergence analysis; 19. Discrete stochastic optimization; V. Inverse Reinforcement Learning: 20. Revealed preferences for inverse reinforcement learning; 21. Bayesian inverse reinforcement learning; Appendix A. Short primer on stochastic stimulation; Appendix B. Continuous-time HMM filters; Appendix C. Discrete-time Martingales; Appendix D. Markov processes; Appendix E. Some limit theorems in statistics; Appendix F. Summary of POMDP algorithms; Bibliography; Index.

Review

'This book uniquely offers a comprehensive treatment of structural results for Partially Observable Markov Decision Processes (POMDPs), utilizing submodularity and stochastic orders. The new edition expands its scope by introducing essential results in nonparametric Bayes, stochastic optimization, and inverse reinforcement learning, making it an invaluable resource as both a textbook and reference.' Bo Wahlberg, KTH Royal Institute of Technology, Sweden
'This book is a tour-de-force on POMDPs and controlled sensing, featuring insightful treatment of foundational concepts in optimal filtering, stochastic control, and stochastic optimization. The new edition introduces innovative methods for detecting cognitive sensors through inverse reinforcement learning from a microeconomic perspective-critical for radar systems, signal processing, and control researchers.' Muralidhar Rangaswamy, Air Force Research Laboratory, U.S.
'An outstanding advanced graduate-level introduction to the increasingly important topic of partially observed Markov decision processes. The book is a delight to read - comprehensive, clear, up-to-date and insightful while preserving rigor. An essential resource for both researchers seeking to further advance the field and practitioners wishing to implement stochastic control in real engineering systems.' Rob Evans, University of Melbourne

Promotional

This new edition includes inverse reinforcement learning, non-parametric Bayesian inference, variational Bayes and conformal prediction.

Details ISBN1009449435 Author Vikram Krishnamurthy Pages 651 Publisher Cambridge University Press Edition Description 2nd Revised edition Year 2025 Edition 2nd ISBN-13 9781009449434 Format Hardcover Imprint Cambridge University Press Subtitle Filtering, Learning and Controlled Sensing Place of Publication Cambridge Country of Publication United Kingdom Alternative 9781009449441 Illustrations Worked examples or Exercises ISBN-10 1009449435 Audience Professional & Vocational DEWEY 519.233 Publication Date 2025-06-05 UK Release Date 2025-06-05 Country of Origin US Product Class Description Mathematics
We\'ve got this

At The Nile, if you\'re looking for it, we\'ve got it.
With fast shipping, low prices, friendly service and well over a million items - you\'re bound to find what you want, at a price you\'ll love!

30 DAY RETURN POLICY

No questions asked, 30 day returns!

FREE DELIVERY

No matter where you are in the UK, delivery is free.

SECURE PAYMENT

Peace of mind by paying through PayPal and Buyer Protection TheNile_Item_ID:171924420;