Information Theory

Catalog description

This introduction to information theory will address fundamental concepts, such as information, entropy, relative entropy, and mutual information. In addition to giving precise definitions of these concepts, the course will include a probabilistic approach based on equipartitions. Many of the applications of information will be discussed, including Shannon's basic theorems on channel capacity and related coding theorems. In addition to channels and channel capacity, the course will discuss applications of information theory to mathematics, statistics, and computer science.