Mathematics of Deep Learning

Catalog description

Empirical observation of deep neural networks shows surprising phenomena that classical statistical theory fails to fully describe. For example, bounds on generalization error from classical statistics grow with the flexibility of the model class but neural networks often exhibit low generalization error despite being extremely flexible. Also, training deep neural networks with stochastic gradient descent produces accurate models despite non-convexity of the loss landscape. A recently emerged literature is developing new theory to explain these and other mysteries. After presenting relevant theoretical results from classical statistics and a brief refresher on deep learning, the course will present theoretical results from recent research articles, complementing them with empirical evidence, emphasizing what is unknown along with what is known.