Abstract

Bayesian learning assumes a prior probability distribution over the ways the world might be. At a fundamental philosophical level, however, it is not clear in what sense such prior probability distributions can be said to be valid. The PAC-Bayesian approach to learning theory also uses a prior probability distribution but provides guarantees on generalization performance that hold independent of the validity of the prior. This talk will present the fundamental PAC-Bayesian theorem as well as applications to a variety of concrete learning algorithms.

David McAllester is currently the chief academic officer of the Toyota Technological Institute at Chicago. He received his B.S., M.S., and Ph.D. degrees from the Massachusetts Institute of Technology in 1978, 1979, and 1987 respectively. He has served on the faculty of Cornell University and of MIT. He was a member of technical staff at AT&T Labs-Research from 1995 to 2002. He has been a fellow of the American Association of Artificial Intelligence since 1997.
Prof. McAllester's research areas include machine learning theory, the theory of programming languages, automated reasoning, AI planning, computer game playing, and computational linguistics. A 1991 paper on AI planning proved to be one of the most influential papers of the decade in that area. A 1993 paper on computer game algorithms influenced the design of the algorithms used in the Deep Blue system that defeated Gary Kasparov. A 1998 paper on machine learning theory introduced PAC-Bayesian theorems which combine Bayesian and nonBayesian methods and whose influence continues to grow rapidly. His plans for future research are focused on the integration of semantics into statistical approaches to computational linguistics.