Big Data

Catalog description

Big Data provides unprecedented opportunities to gain insights and to train models. However, this is a double-edged sword, as the scale of truly big datasets also poses new challenges. This class is designed to address problems of data scaling by introducing principles and tools related to distributed computing and distributed storage as well as algorithms and storage schemes that scale well with larger samples. We will do so by using frameworks like Hadoop, Spark and Dask to wrangle big datasets found in the wild.