Image Processing and Computer Vision: 9
1
Image and Video Coding
Compressing data to a smaller volume without
losing (too much) information
Why Code Data?
To reduce storage volume
 To reduce transmission time


One colour image




760 by 580 pixels
3 channels, each 8 bits
1.3 Mbyte
Video data



Same resolution
25 frames per second
33 Mbyte/second
Image Processing and Computer Vision: 9
3
Redundancy

Spatial


Chromatic


Correlation between colour channels
Temporal


Correlation between adjacent pixels
Correlation between adjacent frames
Perceptual

Unnoticed losses
Image Processing and Computer Vision: 9
4
Example
Image Processing and Computer Vision: 9
5
Redundancy - Consequences

Data exceeds information


More data than content justifies
Can lose data without losing
information
Image Processing and Computer Vision: 9
6
Compression Ratio
volumeof uncompressed image
C
volumeof compressedimage
Image Processing and Computer Vision: 9
7
Lossy vs Lossless Compression



Can lose data without losing
information
Lossily compressed images can look
similar to the original
Lossy compression has greater C
Image Processing and Computer Vision: 9
8
Quality of Decoded Images

Measure differences between



Original
Coded/decoded images
Options



Maximum difference
Average difference
Subjective scales
Image Processing and Computer Vision: 9
9
Example
Image Processing and Computer Vision: 9
10
Subjective Quality Measurement
Scales - Example
0
1
2
3
4
Unusable
Annoying degradation
Adequate images
Barely perceptible degradation
No observable degradation
Image Processing and Computer Vision: 9
11
Difference coding

Adjacent pixels are similar

Difference is small

Uncompressed:
s , s , s , s , s ,
0
1
2
3
4
Compressed:

s , s  s , s  s , s  s , s  s ,
0
0
1
1
2
Image Processing and Computer Vision: 9
2
3
3
4
12
Coding






Assign 4 bits to a difference
Can code –7, … , +8
Overflow?
Use –7 and +8 to show larger differences
Code –6, … , +7 directly
Use overflow codes to indicate shift of codes
Image Processing and Computer Vision: 9
13
Image Processing and Computer Vision: 9
14
Predictive Coding

If signals are well sampled




Adjacent samples are correlated
They have similar values
Differences are small
Can guess next sample from value of
current
Image Processing and Computer Vision: 9
15
Iˆt 1   I t  



Constants are correlation coefficient and mean
grey value
Difference between real and predicted
values are smaller
Code as for difference coding
Image Processing and Computer Vision: 9
16
Run Length Coding

Replace runs of equal brightness values by

(length of run, value)
12233444565
(1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5)

More use when few brightness values

e.g. fax
Image Processing and Computer Vision: 9
17
Huffman Coding



Uses variable length codes
Most frequently occurring grey value
has shortest code
Least frequently occurring values have
longest codes
Image Processing and Computer Vision: 9
18
Example
Symbol Probability
A
B
C
D
E
F
0.4
0.3
0.1
0.1
0.06
0.04
1
0.4
0.3
0.1
0.1
0.1
2
3
0.4
0.3
0.2
0.1
0.4
0.3
0.3
Image Processing and Computer Vision: 9
4
0.6
0.4
19
Symbol Probability
A
B
C
D
E
F
0.4 1
0.3 00
0.1 011
0.1 0100
0.06 01010
0.04 01011
1
0.4
0.3
0.1
0.1
0.1
2
1
00
011
0100
0101
Image Processing and Computer Vision: 9
0.4
0.3
0.2
0.1
3
1
00
010
011
0.4 1
0.3 00
0.3 01
4
0.6 0
0.4 1
20
GIF



Applicable to images with 256 colours
Replace sequences of bytes with shorter
codes
Most common sequences use shortest
codes
Image Processing and Computer Vision: 9
21
Wavelet coders


Wavelet transform organises image
content efficiently
Can easily select data to be discarded
Image Processing and Computer Vision: 9
22
Wavelet coders
Image Processing and Computer Vision: 9
23
JPEG standard




A subcommittee of ISO
Optimised for a range of image subject
matter
Compression rates can be defined
Quality inversely proportional to C
Image Processing and Computer Vision: 9
24
Block Transform Coding
original image
decompose
Sequence of 8 by 8 blocks - different planes treated
separately (RGB, YUV etc.)
transform
Transformed blocks reduce redundancy and
concentrate signal energy into a few coefficients
discrete cosine transformation (DCT)
quantise
Blocks with discarded information - goal is to smooth
picture and discard information that will not be
missed, e.g. high frequencies
entropy code
Image Processing and Computer Vision: 9
25
Block Transform Encoding
DCT
Zig-zag
run
length
code
Image Processing and Computer Vision: 9
quantise
entropy
code
010111000111…..
26
Block Encoding
Original image
139 144 149 153
144 151 153 156
150 155 160 163
159 161 162 160
DCT
1260 -1 -12 -5
-23 -17 -6 -3
-11 -9 -2 2
-7 -2 0 1
Zig-zag
79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0
run
length
code
0 79
1 -2
0 -1
0 -1
0 -1
2 -1
0 0
Image Processing and Computer Vision: 9
Huffman
code
quantise
79
-2
-1
0
0 -1
-1 0
-1 0
0 0
0
0
0
0
10011011100011….
27
Block Transform Decoding
DCT
Zig-zag
run
length
code
Image Processing and Computer Vision: 9
quantise
entropy
code
010111000111…..
28
Result of Coding and Decoding
139 144 149 153
144 151 153 156
150 155 160 163
159 161 162 160
144 146 149 152
148 150 152 154
155 156 157 158
160 161 161 162
Original block
Reconstructed block
-5
-4
-5
-1
-2
1
-1
0
0
1
3
1
1
2
5
-2
errors
Image Processing and Computer Vision: 9
29
Discrete Cosine Transform
4C(u)C(v)
F[u,v] =
n2
with
n-1 n-1
S S
f(j,k) cos
j=0 k=0
1
2
For w = 0
1
otherwise
(2j+1) u p
2n
cos
(2k+1) v p
2n
C(w) =
Image Processing and Computer Vision: 9
30
Discussion

Where are lossy steps?


Quatisation and subsampling before coding
How is quantisation matrix chosen?

Its predefined by the standard after much
experimentation
Image Processing and Computer Vision: 9
31
Video coding

Many specific standards


AVS, MOV, QT, …
One universal standard

MPEG
Image Processing and Computer Vision: 9
32
MPEG Standards




Standard specifies audio, video and
system layers
MPEG-1: low data rates, poor quality:
VHS quality at 1.5Mbits-1
MPEG-2: high quality hence high data
rates: studio quality, 15Mbits-1
MPEG-4: low data rates, small images,
64 kbits-1
Image Processing and Computer Vision: 9
33
MPEG-1

Audio and video designed to work at CD
ROM speeds: 1.5Mbits-1



Video 1.150Mbits-1
Audio 0.256Mbits-1
System 0.094Mbits-1
Image Processing and Computer Vision: 9
34
MPEG-2



Released in 1994
Aimed at digital TV, ATM.
Additions for




Interlaced video
Scalable video coding
Graceful degradation with noise
Implementation of full standard impractical

Varying profiles/levels of conformity
Image Processing and Computer Vision: 9
35
MPEG-4

Coding specifically for multimedia
objects
Image Processing and Computer Vision: 9
36
Coding Algorithms



Frame sequence
Motion compensation
Frame coding
Image Processing and Computer Vision: 9
37
Frame Sequence

I frames (Intraframes)


P frames (Predicted frames)


Coded independently of any other frame
Derived from previous I frame by motion
prediction
B frames (Bidirectionally interpolated)

Interpolate motion compensated blocks
between I and P frames
Image Processing and Computer Vision: 9
38
Motion Compensation



Image is divided into macroblocks (16 x
16 pixels)
Matching macroblocks are found by
minimising differences
Code differences and macroblock
displacement
Image Processing and Computer Vision: 9
39
Frame Coding

Use JPEG algorithms
Image Processing and Computer Vision: 9
40
Summary





Why code data?
Redundancy
Assessment of compression
Lossy vs. lossless compression
Algorithms

JPEG, MPEG
Image Processing and Computer Vision: 9
41
But what … is it good for?
Engineer at the Advanced Computing
Systems Division of IBM, commenting on
the microchip in 1968
Image Processing and Computer Vision: 9
42