Optimization for AI

Catalog description

This course introduces the theory, algorithms, and applications of nonlinear optimization, which is at the core of many fundamental algorithmic challenges in AI, such as the training of models like deep neural nets and transformers, and is used at massive scales. We will study unconstrained and constrained optimization problems, focusing on convex and nonconvex settings. Topics include optimality conditions, convexity, gradient-based and Proximal-type methods, second-order methods, line-search strategies, and duality theory. Emphasis will be placed on both the mathematical foundations and the practical implementation of algorithms.