Information Theory
Subject code: SIP 7001

---

Lecturer
Prof Charles Pearce
The University of Adelaide

Mode of delivery
On-line.

Assumed knowledge
A basic understanding of probability theory is required. Knowledge of communication theory would be advantageous. Assignment work will assume knowledge of MATLAB.

Aims/Learning Objectives
The course aims to introduce the basic theoretical techniques of information theory. Illustrative examples are used to show how these techniques are employed and exercises given to help familiarise the student with their use. The various parts of a Shannon communications system are examined, including coding and decoding with or without the presence of noise and with data compression and efficient transmission in mind.

Content
Information Measures: entropy, relative entropy and mutual information.
Source coding: Discrete memoryless sources, Shannon's first (noiseless) coding theorem, Shannon-Fano-Elias coding, Huffman coding. Sources with memory. Universal source coding theorem. Ziv-Lempel Coding.
Channel coding: Discrete memoryless channels, channel capacity, Shannon's second (noisy) coding theorem, error control coding, performance bounds.
Advanced topics: multiple-user information theory, fading channels, multiple-antenna channels.

Assessment
60% assignments and 40% exam, however these percentages are indicative only and may be varied at the lecturer's discretion. Details of the actual assessment used in a given year can be found in the study guide provided at the start of the semester.

Resources
All the materials necessary for the course will be availabe on-line. The lecture notes also include an extended bibliography for further reading on the subject.