PhD Program in Information Science
Department of Informatics, College of Computing & Information
IINF 723 Information & Computing (Spring, 2008)
J Gangolly / S Goel
Welcome
Catalog Description:
Development of theories and concepts that underlie the operation of information processing and
retrieval systems; consequences derived from these theories that should be considered in designing
such systems; theoretical foundations of information and computation; technologies and application
areas.
A more Detailed Description:
 Information Theory (3 weeks) (Shannon entropy, Relative entropy or Kullbach-Leibler
divergence, Maximum entropy, Information & inference)
 Coding theory (2 weeks) (data compression, error correcting codes, noisy channel coding,
Message passing)
 Bayesian Decision Theory (3 weeks) (utility theory, Bayesian inference & networks)
 Algorithms & Complexity (3 weeks) (algorithms, time and space complexity, NPcompleteness, Big O notation)
 Logic & Information (2 weeks) (Deduction/induction/abduction, Completeness, soundness,
consistency, satisfiability, Descriptive logics, Model/Proof)
Course Grading:
A short paper (5-10 double-spaced pages) and a long paper (15-25 double-spaced pages) are due on
May 12, 2008. The topics should come from any two of the three areas covered in the course:
Information Theory/entropy, Coding theory/Bayesian Decision Theory, and Locic &
Information/Algorithms & Complexity.
Additional readings will be added to the list in the tentative schedule below.
The tentative schedule
_____________________________________________________
Jan 28 Introduction & Information Theory I (J Gangolly / S Goel)
Topics: Introduction to the course
Feb 4 Introduction & Information Theory II (J Gangolly)
Topics:
Concepts of information and entropy.
Readings:
What is Information Anyway? Blog by Peter Van Garderen
http://archivemati.ca/2007/01/29/what-is-information-anyway/
What is Information? by David Israel & John Perry
http://www-csli.stanford.edu/~jperry/PHILPAPERS/whatisinfo.pdf
Entropy-Based Link Analysis for Mining Web Informative Structures, by Hung-Yu Kao,
Shian-Hua Lin, Jan-Ming Ho, and Ming-Syan Chen
Conference on Information and Knowledge Management, November 4-9, 2002,
McLean,Virginia, USA.
Fundamental Forms of Information, by Marcia J. Bates
JOURNALOF THE AMERICAN SOCIETYFOR INFORMATION SCIENCE AND
TECHNOLOGY, 57(8):1033–1045, 2006
Feb 11 Introduction & Information Theory III (J Gangolly)
Topics: Belief functions and entropy
Readings:
Information Content of Evidence, by Philippe Smets
International Journal of Man-Machine Studies, 19:33-43, 1983
Entropy of dialogues creates coherent structures in e-mail traffic, Jean-Pierre Eckmann, Elisha Moses, and
Danilo Sergi
Proceedings of the National Academy of Sciences, October 5, 2004 vol. 101 no. 40 pp.14333–
14337
Feb 18 Winter Break (No Class)
Feb 25 Introduction to Computation (J Gangolly)
Topics: Fundamentals of Logic & Computation, Computational Complexity.
Readings:
THE BRITISH NATIONALITY ACT AS A LOGIC PROGRAM, by M. J. SERGOT, F. SADRI,
R. A. KOWALSKI, F. KRIWACZEK, P. HAMMOND, and H. T. CORY
Communications of the ACM May 1986 Volume 29 Number 5, pp. 370-386
Principles of Problem Solving, by Peter Wegner and Dina Goldin
COMMUNICATIONSOF THE ACM July 2006/Vol. 49, No. 7, pp. 27-29.
The Wikipedia article on Computational Complexity Theory
http://en.wikipedia.org/wiki/Computational_complexity_theory
Mar 3 Algorithms & Complexity I (S Ravi)
Topics: TBD
Readings: TBD
Mar 10 Float
Topics: TBD
Readings: TBD
Mar 17 Logic & Computation (N Murray)
Topics: TBD
Readings: TBD
Mar 24 Spring Break No Class
Mar 31 Maximum Entropy methods (?) (K Knuth)
Topics: TBD
Readings: TBD
Apr 7 Coding Theory- Data Compression (S Goel)
Topics:
The problem of reliable communication through a noisy channel. We discuss the
error correcting strategies of repetition and hamming code. We also discuss
Shannon's Noisy Channel. Coding theorem as well as definitions of entropy,
conditional entropy, and mutual information.
Readings: TBD
Apr 14 Coding Theory - Error Correction Codes (S Goel)
The concept of entropy in detail and Shannon's source coding theorem. We also
look at compression algorithms including block codes, uniquely decodable codes,
optimum code lengths and Huffman codes. Kolmogorov Complexity as well as
estimation techniques for complexity will also be covered in the class.
Apr 21 No Class
Apr 28 Bayesian Decision Theory & Information Theory I (S Goel)
The basics of probability theory and solve decision analysis problems using decision
tables. Students learn both deterministic and probabilistic approaches for decision
analysis as well as value of perfect information.
Readings: TBD
May 5 Bayesian Decision Theory & Information Theory II (S Goel)
Review of Bayesian Probability Theory and problems in multistage decision analysis.
Students will also learn the use of a decision tree tool that they can use for solving
complex problems. Simple decision trees will also be covered in the class.
Readings: TBD
May 12 Bayesian Decision Theory & Information Theory III (S Goel)
The basics of utility theory and its application to decision analysis. Students will also
understand the issues related to information security risk analysis.
Readings: TBD
_____________________________________________________