Computing That Serves

Using Nonparametric Bayesian Models to Learn Dynamical Systems


Thursday, January 15, 2009 - 11:00am


David Wingate
Postdoctoral research associate, Computational Cognitive Science Group, MIT

A challenge for designing RL agents capable of learning in highly structured but partially observable domains is representing uncertainty in both the state and the model.  Such agents must often cope with rich observations (such as images), a possibly unknown number of objects, their properties/types and their dynamical interactions.  The agent must be able to generalize radically to new situations and flexibly incorporate prior knowledge in the form of naturally occurring structures, such as trees, rings, and manifolds.

In this talk, I will discuss how hierarchical Bayesian models with structured nonparametric priors can be used to begin to capture these rich dynamical systems.  I'll present the Infinite Latent Events Model, which can be used to learn a factored, causal model of the world.  Time permitting, I will discuss a central challenge to using such models as part of a reinforcement learning agent, which is planning in partially observable domains where distributions over possible worlds do not have finite dimensional sufficient statistics.

This is joint work with Josh Tenenbaum and Noah Goodman.


David Wingate is a postdoctoral research associate in the  Computational Cognitive Science Group at MIT working with Josh Tenenbaum.  His research interests lie at the intersection of perception, control and cognition, and how all three have synergistic effects on learning. Specific interests include reinforcement learning, unsupervised learning of useful knowledge representations (including predictive representations of state and structured nonparametric Bayesian distributions), information theory, manifold learning, kernel methods, massively parallel processing, visual perception, and optimal control.  David holds a Ph.D. from the University of Michigan and M.S. and B.S. degrees in Computer Science from BYU.