Information Theory

Catalog description

This course introduces the student to the fundamental concepts and results of information theory. This is a very important course for students who want to specialize in signal processing, image processing, or digital communication. Topics include definition of information, mutual information, average information or entropy, entropy as a measure of average uncertainty, information sources and source coding, Huffman codes, run-length constraints, discrete memoryless channels, channel coding theorem, channel capacity andShannon's theorem, noisy channels, continuous sources and channels, coding in the presence of noise, performance bounds for data transmission, rate distortion theory.