INFORMATION THEORY FOR DATA SCIENCE.

Catalog description

This course introduces fundamental concepts of information theory and their applications in modern data science and machine learning. Topics include entropy, mutual information, KL divergence, channel capacity, rate-distortion theory, and information-theoretic bounds. Students will learn how these principles connect to practical data science workflows, including feature selection, representation learning, compression, communication-efficient learning, and querying/processing large-scale datasets with SQL for information-theoretic analysis (e.g., aggregations, empirical distributions, entropy/MI estimation from relational data). The course emphasizes hands-on assignments and projects using real-world datasets, combining mathematical foundations with implementation skills.