Deep Learning Systems: Algorithms and Implementation

Catalog description

The goal of this course is to provide students an understanding and overview of the "full stack" of deep learning systems, ranging from the high-level modeling design of modern deep learning systems, to the basic implementation of automatic differentiation tools, to the underlying device-level implementation of efficient algorithms. Throughout the course, students will design and build from scratch a complete deep learning library, capable of efficient GPU-based operations, automatic differentiation of all implemented functions, and the necessary modules to support parameterized layers, loss functions, data loaders, and optimizers. Using these tools, students will then build several state-of-the-art modeling methods, including convolutional networks for image classification and segmentation, recurrent networks and self-attention models for sequential tasks such as language modeling, and generative models for image generation. Prerequisites: (15-513 Min. grade C or 15-213 Min. grade C) and ( 21-240 Min. grade C or 21-241 Min. grade C) and ( 21-128 Min. grade C or 21-127 Min. grade C or 15-151 Min. grade C) and ( 10-715 Min. grade C or 10-701 Min. grade C or 10-601 Min. grade C or 10-315 Min. grade C or 07-280 Min. grade C or 10-301 Min. grade C)