Scalable Data Systems and Architecture

Catalog description

This course provides an in-depth exploration of the architectural principles and engineering frameworks required to manage and process data at a massive scale. As organizations move beyond traditional database limits, the ability to design systems that handle high-volume and high-velocity data becomes critical. Students will analyze the evolution of distributed computing, focusing on the Hadoop ecosystem and Apache Spark for distributed storage and processing. The course bridges the gap between theoretical distributed systems and practical implementation on major cloud platforms (AWS, Azure, or GCP). Special emphasis is placed on the construction of resilient data pipelines, the transition to "data lakes" and "lakehouses," and the evaluation of scalable architectures that support real-time analytics and batch processing in production environments.