ECE 4400:693 – Syllabus

Basics of Information Theory

  • Source and Channel Models
  • Entropy, Relative Entropy, Mutual Information, Asymptotic Equipartition Property, Entropy Rates

Data Compression

Channel Capacity

  • Discrete Channels and Random Coding Bound
  • Gaussian Channels: AWGN, Wireless Fading Channels and Water Filling, MIMO Channels

Practical Coding Schemes

  • Block Codes
  • Convolutional Codes and Turbo Codes

Network Information Theory
Multiple-Access and Broadcast Channels
Other topics as time permits

Main References

  1. Lecture Slides
  2. T. Cover and J. Thomas, Elements of Information Theory, Second Edition, John Wiley & Sons, Inc., 2006.

Other References

  1. D. Tse and P. Viswanath, Fundamentals of Wireless Communication, Cambridge University Press, 2005, available online
  2. D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003, available online


  • Final grades will be determined on the combination of homework assignments (30%), a project (25%), and a comprehensive final examination (45%).
  • Note on the projects: The goal of the project is to help you get exposed to some interesting research topics in Information Theory and its applications not covered in the class. It is your choice to select the topic. You can also pick up a topic from a list of suggested projects provided here. Each student is required to submit a project report (of about 5-8 pages) and provide an oral presentation (no more than 20 minutes) at the end of the term.
  • Note on final exam: It is an open-book exam