Information Theory

Catalog description

Deterministic and probabilistic information. The pigeon-hole principle. Entropy, relative entropy, and mutual information. Random processes and entropy rate. The asymptotic equipartition property. Optimal codes and data compression. Channel capacity. Source channel coding. The ubiquitous nature of the theory will be illustrated with a selection of applications drawn from among: universal source coding, vector quantization, network communication, the stock market, hypothesis testing, algorithmic computation and kolmogorov complexity, and thermodynamics.