Foundations of Deep Learning

Catalog description

Deep Neural Networks (DNNs) have revolutionized artificial intelligence, enabling machines to perceive, understand, and interact with the world in ways that were once thought impossible, from defeating world champions in complex problems to generating human-like text and creating stunning artwork. This graduate level course covers fundamental concepts in DNNs such as the expressiveness of neural networks, optimization techniques, and generalization principles. Students will also learn foundations of various neural network architectures, including feedforward networks, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Graph Neural Networks (GNNs), Transformers, and generative models such as Generative Adversarial Network (GANs) and diffusion models. The course also addresses crucial topics like scalability, efficient training and fine-tunning methods, accelerated inference, and advanced concepts and modern paradigms such as transfer learning, mixture of experts, and model merging. A strong background in machine learning and linear algebra is required. Through a combination of theoretical lectures and hands-on programming assignments, students will gain a deep understanding of the core principles driving deep learning, enabling them to analyze, implement, and improve state-of-the-art AI models.