         
|
GRASP Lab Seminar 2003-2004
January 23, 11:00 AM, Levine Hall 307, hosted by Lawrence Saul.
Satinder Singh
University of Michigan
Predictive State Representations (A new class of models for discrete-time controlled dynamical systems)
Abstract: The use of Markov decision process (MDP) models to represent controlled dynamical systems has been very fruitful for reinforcement learning and for artificial intelligence in general. After briefly reviewing some of these "fruits" I will discuss the limitations of MDP models and the need to go beyond them. The standard extension of MDPs to partially-observable MDPs, or POMDPs, haven't served us well, at least so far. In this talk, I will present predictive state representations, or PSRs, a new class of predictive models for discrete-time controlled dynamical systems. The key idea in PSRs is to use predictions of observable outcomes of tests or experiments the agent can do in its environment to represent the state of the environment. I will show that PSRs are more general than POMDPs and yet are at least as, and often more, compact than POMDPs. I will also present some results on learning PSR models from data and conclude with some reasons for optimism about PSR models as well as with directions for future work on PSRs.
Biography: Satinder Singh is an Associate Professor of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor. Prior to this he has been a Principal Research Scientist at AT&T Labs, an Assistant Professor of Computer Science at the University of Colorado, Boulder, and a Postdoctoral Fellow at MIT's Brain and Cognitive Science department. His research focus is on developing the theory, algorithms and practice of building agents that can learn from interaction in complex, dynamic, and uncertain environments, including environments with other agents in them. His main contributions have been to the areas of reinforcement learning and more recently multi-agent learning. He edited a special issue on reinforcement learning for the Machine Learning journal in 2002, has coauthored more than 60 refereed papers in journals and conferences and has served on many program committees (AAAI, ICML, NIPS, UAI, COLT) and on journal editorial boards (Machine Learning, JAIR, JMLR).
