Neural Networks: From Theory To Practice

Catalog description

This course explores the fundamental and advanced methodologies of neural networks and deep learning theoretically and practically. It emphasizes practical, hands-on learning through the extensive use of industry-standard deep learning libraries and frameworks, such as TensorFlow and Pytorch. Students are expected to engage in comprehensive projects, culminating in a team project and presentation towards the end of the term. Throughout the course, participants will handle large datasets with complex features, enhancing their ability to work with real-world data. The course includes a thorough examination of the statistical foundations underlying neural networks, emphasizing the maximum likelihood estimation technique. Key optimization strategies such as gradient descent, stochastic gradient descent, and advanced momentum methods including RMSprop, ADAM, and NADAM are explored in detail. The course also reviews a range of regularization methods—from traditional L2 and L1 regularization to dropout and early stopping—to enhance model performance and prevent overfitting. An in-depth exploration of the backpropagation algorithm equips students with a deep understanding of how neural networks learn. The course covers a diverse array of neural network models and their applications, including sequential dense feedforward networks, autoencoders, convolutional networks, recurrent networks, Transformers, variational autoencoders, and Generative Adversarial Networks. Students will gain proficiency in both supervised techniques, for tasks like regression and classification, and unsupervised methods, such as feature extraction and the use of autoencoders. Prerequisite(s): ENGR 15100 FOR LEVEL UG WITH MIN. GRADE OF D- AND ENGR 15200 FOR LEVEL UG WITH MIN. GRADE OF D- AND MA 26100 FOR LEVEL UG WITH MIN. GRADE OF D- Course Learning Outcomes 1. Understand Fundamental Concepts: Grasp the basic principles of neural networks, including the structure and function of neurons, linear regression, and key algorithms such as backpropagation and stochastic gradient descent. 2. Apply Machine Learning Techniques: Implement supervised learning methods for regression and classification and analyze data using various evaluation metrics and loss functions. 3. Develop and Optimize Neural Networks: Build and optimize dense neural networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs) for various applications, including image and timeseries analysis. 4. Utilize Advanced Techniques: Apply regularization strategies, advanced optimization methods, and understand cutting-edge topics such as Generative Adversarial Networks (GANs) and Graph Neural Networks. 5. Implement Practical Projects: Design and execute individual and team-based projects that demonstrate the application of neural network theories and practices in real-world scenarios. 6. Engage with Industry Practices: Gain insights from industry professionals and apply practical skills using contemporary tools and frameworks like PyTorch. View Class Schedule