Sheet 2
1. Given an alphabet A = {a1, a2, a3, a4}, Find the Entropy for the following cases
(a) P(a1) = P(a2) =, P (a3) = P(a4) = 0.25
(b) P(a1) = 0.15, P(a2) = 0.04, P (a3) = 0.26, P(a4) = 0.05
2.
Determine whether the following codes are uniquely decodable:
(a) {0,01,11,111}
(b) {0,01,110,111}
(c) {0,10,110,111}
(d) {1,10,110,111}
3. (a) For a binary source with probabilities P(0) = 0.9, P(l) = 0.1, design a Huffman code for
the source obtained by blocking m bits together, m = 1, 2, 3.
Plot the average lengths versus m. Comment on your result.
(b) Repeat with P(0) = 0.99, P(l) = 0.01.
4. While the variance of lengths is an important consideration when choosing between two
Huffman codes that have the same average lengths, it is not the only consideration. Another
consideration is the ability to recover from errors in the channel. In this problem we will
explore the effect of error on two equivalent Huffman codes.
(a) For the source and Huffman code of the table below,
Letter Probability Code
a2
0.4
1
a1
0.2
01
a3
0.2
000
a4
0.1
0010
a5
0,1
0011
Encode the sequence a2 a1 a3 a1 a2
5. Suppose the length of the window is 13, the size of the look-ahead buffer is six, and we have decoded the
sequence cabraca using lz77, the triples (0, 0, C(d)) , (7, 4, C(r)), and (3, 5, C(d)).
6. Suppose we want to encode the string wabbaswabbaswabbaswabbaswooswooswoo using LZW, with
Page: 1/2
Part: 1/1
7. Encode the following sequence using the LZ78 approach: wabbaswabbaswabbaswabbaswooswoosbwoo
8. Design a Huffman code for a source that puts out letters from an alphabet {a1,a2,a3,a4,a5} with P(a1)= P(a3)= 0.2 ,
P(a3)=0.4, P(a4)= P(a5)= 0.1.
(a) Illustrate the types of data compression techniques?
(b) Illustrate the two phases used in any data compression techniques?
(c) Illustrate the types of models used in data compression?
(d) Illustrate how can we evaluate or measure the efficiency of data compression techniques?
(e) Illustrate how the Huffman technique works?
(f) Illustrate what is meant by Entropy and Redundancy?
(g) What are the concerns of LZ77 and how to overcome them?
(h) (b) What are the features of Good code
Page: 2/2
Part: 1/1