Bayesian Methods in Machine Learning

Catalog description

This course will cover modern machine learning techniques from a Bayesian probabilistic perspective. Bayesian probability allows us to quantify, model and reason about all types of uncertainty. The result is a powerful, internally-consistent framework for approaching many problems that arise in machine learning, including parameter estimation, model comparison, and decision making. We will begin with a high-level introduction to Bayesian inference and show how it can be applied to familiar machine learning tasks, such as regression and classification. We will also introduce the workhouse model of the course: Gaussian processes, a flexible non-parametric model with many convenient properties. The second half of the course will cover more-advanced topics, with a heavy focus on probabilistic numerics. The field of probabilistic numerics seeks to apply probabilistic or statistical methods to find numerical solutions for intractable tasks, such as quadrature, global optimization and solving differential equations. Along the way, we will also consider how the techniques presented in class can be a) scaled to handle larger/higher-dimensional datasets and b) combined with active learning techniques to build adaptive models that can determine which data points should be observed next in order to most improve their performance. The course is designed for students who have completed an introductory course in machine learning.