The instructor for this class is Asst Prof. Vincent Y. F. Tan.
ˆ Office: E4-06-10 (or S17-05-20)
ˆ
Office Hours: 4pm-5pm each Monday
ˆ
Email: vtan@nus.edu.sg
ˆ
Class time: 6pm-9pm each Wednesday
This is a course on information theory, first developed by the great mathematician and engineering Claude
E. Shannon. We will study the fundamentals of communications such as hypothesis testing, source coding, channel coding, all of which lead to the source-channel separation theorem.
All the information for this class can be found on the class website: http://www.ece.nus.edu.sg/stfpage/vtan/ee5139/index.htm
ˆ Homeworks (10%)
ˆ
Two quizzes (15% + 15%)
ˆ
Project (10%)
ˆ Final Exam (50%)
There will be weekly homework assignments. They will be due on the following week. No extensions will be allowed unless there are extenuating circumstances. Solutions will be posted once the homeworks are due.
You are strongly encouraged to collaborate with each other in solving the homework problems but you must write up the solutions on your own. These are meant to help improve your understanding of the subject matter, and you should treat the homework very seriously in order to gain the most out of this course. There will be many problems labelled as optional . You don’t have to do these problems but they are meant as a vehicle for learning.
Each homework set will be graded coarsely. You will receive one of four numerical grades–0, 1, 2, 3. 0 means that the homework was not turned in and 3 means that the homework was done almost perfectly. 1 and 2 interpolate in between.
You’re allowed to drop the 2 homework sets that have the worst numerical scores in the final computation of your homework grade. Thus, I’ll generally not entertain any requests to turn in the homeworks late.
1

You are required to complete a project consisting of a critical review of a research paper in the area of information theory. You are expected to have a thorough investigation of the paper and come up with a detailed report. In the report, you need to include the background of the work, the proposed major ideas, the understanding of how information and coding theory is applied to solve a practical problem, and any possible extension of the work. This project paper is due on 28/10/2015.
There will be two quizzes during the course of the semester. The exam is scheduled on the finals week and will be comprehensive, i.e., covering all topics in the course.
For each of the two quizzes, you are allowed 1 A4-cheat sheet (double sided). For the final, you are allowed 3 A4-cheat sheets.
You are expected to have a firm grasp of probability and statistics. You should know what is a random variable, probability mass functions, joint and conditional distributions, Markov chains, probability bounds
(such as Markov and Chebyshev inequalities), laws of large numbers, conditional independence, etc. We will have a thorough review of the material in the first class.
All students are to have a copy of Cover and Thomas as assignments will be taken from the problems therein.
The second reference is very much optional. Other references, such as papers, will be posted on the course website.
ˆ
T. M. Cover and J. A. Thomas: Elements of Information Theory. Wiley-Interscience, 2nd edition,
2006.
ˆ
I. Csisz´ orner. Information Theory: Coding Theorems for Discrete Memoryless Systems.
Cambridge University Press, 2011.
2