Cloud Data Engineering and MLOps

Catalog description

This course addresses the critical challenge of moving machine learning models from isolated experiments into scalable, resilient production environments. Students will explore the convergence of Data Engineering and Machine Learning Operations (MLOps), focusing on the automation of the entire machine learning lifecycle within cloud-native ecosystems. The course covers the design of robust ETL/ELT pipelines using cloud data warehouses (e.g., Snowflake, BigQuery) and the implementation of CI/CD (Continuous Integration/Continuous Deployment) specifically tailored for ML. Students will analyze strategies for versioning data and models, managing feature stores, and deploying containerized models using Docker and Kubernetes. A significant portion of the course is dedicated to post-deployment challenges, including model monitoring, detecting "data drift," and establishing automated retraining loops to ensure long-term model performance and reliability. DATA 831: Data Governance, Privacy, and Responsible AI at Scale (3 credit hours) As AI systems become more autonomous and pervasive, the responsibility to ensure they are ethical, transparent, and compliant becomes a primary leadership function. This course examines the frameworks and technologies necessary to manage data assets and AI models responsibly within a global landscape. Students will evaluate the principles of Data Governance, including data lineage, quality, and metadata management, and how these foundations support Responsible AI. The course dives into the technical and legal aspects of Data Privacy, covering global regulations such as GDPR, CCPA, and the EU AI Act. Crucially, doctoral students will investigate the "Black Box" problem, exploring Explainable AI (XAI) techniques to ensure algorithmic transparency. The course also addresses the detection and mitigation of algorithmic bias, ensuring that large-scale AI deployments do not reinforce systemic inequities. DATA 840 Cloud Data Visualization This course provides an in-depth exploration of data visualization and exploratory data analysis within cloud environments. Students will utilize Python-based data visualization tools to analyze datasets, beginning with foundational examples and progressing to case studies focused on global health, economics, and infectious disease trends in the United States. Emphasis is placed on recognizing and addressing mistakes, biases, systematic errors, and other data quality issues that can impact analyses. The course highlights the critical role of data visualization in uncovering insights, identifying flaws, and effectively communicating findings. By mastering these skills, students will be prepared to leverage data to drive informed decisions and advance their careers in data science and analytics.