Optimization for Machine Learning

Catalog description

Nearly every problem in machine learning can be formulated as the optimization of some function, possibly under some set of constraints. This universal reduction may seem to suggest that such optimization tasks are intractable. Fortunately, many real world problems have special structure, such as convexity, smoothness, separability, etc., which allow us to formulate optimization problems that can often be solved efficiently. This course is designed to give a graduate-level student a thorough grounding in the formulation of optimization problems that exploit such structure, and in efficient solution methods for these problems. The main focus is on the formulation and solution of convex optimization problems, though we will discuss some recent advances in nonconvex optimization. These general concepts will also be illustrated through applications in machine learning and statistics. Students entering the class should have a pre-existing working knowledge of algorithms, though the class has been designed to allow students with a strong numerate background to catch up and fully participate. Though not required, having taken 10-701 or an equivalent machine learning or statistical modeling class is strongly encouraged, as we will use applications in machine learning and statistics to demonstrate the concepts we cover in class. Students will work on an extensive optimization-based project throughout the semester. Prerequisites: ( 21-240 Min. grade C or 10-606 Min. grade C or 21-671 Min. grade C or 21-341 Min. grade C or 21-242 Min. grade C) and ( 21-268 Min. grade C or 21-259 Min. grade C or 21-254 Min. grade C) and ( 15-259 Min. grade C or 36-217 Min. grade C or 15-359 Min. grade C or 21-325 Min. grade C or 36-225 Min. grade C or 36-219 Min. grade C or 36-218 Min. grade C) Course Website: https://sites.google.com/view/convexopt-10725-2023f