Introduction to Information Theory

Catalog description

This course is intended as an introduction to information and coding theory with emphasis on the mathematical aspects. It is suitable for advanced undergraduate and graduate students in mathematics, applied mathematics, statistics, physics, computer science and electrical engineering. Course content: Information measures-entropy, relative entropy, mutual information, and their properties. Typical sets and sequences, asymptotic equipartition property, data compression. Channel coding and capacity: channel coding theorem. Differential entropy, Gaussian channel, Shannon-Nyquist theorem. Information theory inequalities (400 level). Additional topics, which may include compressed sensing and elements of quantum information theory. Recommended preparation: MATH 201 or MATH 307 . Offered as MATH 394 , CSDS 394 , ECSE 394 , MATH 494 , CSDS 494 and ECSE 494 .