Image Processing
in the block DCT Space
Jayanta Mukhopadhyay
Department of Computer Science & Engineering
Indian Institute of Technology, Kharagpur, 721302, India
jay@cse.iitkgp.ernet.in
1
Processing in the spatial
domain
2
Processing in the compressed
domain
3
Motivations



Computation with reduced storage.
Avoid overhead of inverse and
forward transform.
Exploit spectral factorization for
improving the quality of result and
speed of computation.
4
Typical Applications




Resizing.
Transcoding.
Enhancement.
Retrieval.
5
DCT Definitions
7
Different types of DCTs
Let x (n), n=0,1,2,...N+1 be a sequence of length N+1.
Then N-point type-I DCT is defined as:
N
2
 n k 
( N )
C1e{x(n)}  X I (k )   (k ) x(n)cos  ,
N
 N 
n 0
0k  N
Note:- Type-I N-point DCT is defined over N+1 data points.
8
Different types of DCTs
Similarly, x (n), n=0,1,2,...N, be a sequence of length N. Its
N-point type-II DCT is defined as:
N 1
2
 (2n  1) k 
( N )
C2e{x(n)}  X II (k )   (k ) x(n)cos 
,
N
 2N 
n 0
0  k  N 1
Note:- Type-II N-point DCT is defined over N data points.
9
Sub-band Relationship
11
Subband DCT

DCT of a 2D images x(m,n) of size N x N

C k , l   m 0 n 0 4 xm, n  cos
N 1
N 1

 2 m 1k
2K
cos
 2 n 1l
2N

0  k, l  N 1
Jung, Mitra and Mukherjee , IEEE CSVT (1996)
12
Low-low subband
x LL ( m, n)  14 {x( 2m,2n)  x( 2m  1,2n)
 ( 2m,2n  1)  x( 2m  1,2n  1)},
0  m, n 
N
2
 1.
Sub-band approximation
4 cos( 2Nk ) cos( 2Nl )C LL (k , l ),
C (k , l )  
0,

k , l  0,1,......, N2  1
otherwise
13
Truncated approximation
4C LL (k , l ),
C (k , l )  
0,

k , l  0,1,......, N2  1
otherwise
Generalized Truncated approximation
C′ (k,l) = √(L.M).C(k,l),
=0
k,l=0,1,...., (N-1)
otherwise
Mukherjee and Mitra (2005), IEE VISP
14
Spatial Relationship
15
Need for Analysis of the Relation between
DCT blocks and sub-blocks
 Direct conversion of DCT blocks of one size
into another possible in the compressed
domain itself
 Saving of computation complexity and
memory required
 Decompression and re-compression steps
reduced
 DCT matrices being sparse number of
multiplications and additions reduced
 DCT blocks of different sizes required for
different standards
Jiang and Feng (2002), IEEE SP
16
Spatial Relationship of DCT coefficients
17
Basis function b(k,t) constructed from b1(k,t)
19
Equations to find the transform matrices for
composition and decomposition
20
EXAMPLES OF TRANSFORM MATRICES
A(2,4) =
0.7071
0
0.6407 0.2040
0
0.7071
-0.2250 0.5594
0
0
0.1503 –0.2492
0
0
-0.1274 0.1964
0
–0.0528
0
0.3629
0.7071
0.5432
0
–0.2654
0
0.7071
0
0.0182 -0.6407 0.2040
0
0
0.7071
–0.0690 0.225 0.5594
0
0
0
0.3468 –0.1503 -0.2492
0.7071
0
0
0.6122 0.1274 0.1964
0
0
0.0528 0.0182
0
0
-0.3629 –0.0690
0.7071
0
-0.5432 0.3468
0
-0.7071
0.2654 0.6122
A(2,2)=
0.7071
0
0.7071
0
0.6533 0.2706 –0.6533
0.2706
0
0.7071
0
-0.7071
-0.2706 0.6533 0.2706
0.6533
21
Block Composition and Decomposition in 2D
.
.
C0, M - 1 
C00 C01 .
C10 C11 .

.
.
C
1, M - 1


.
 T
.
.
.
.
.
C  A ( L, N ) 
 A (M, N)
.
.
.
.
.
.





CL  1,0 CL  1,1 .............. CL - 1, M - 1
Composition of L x M adjacent
N x N DCT blocks to a single
LN x MN DCT block:
C 00 C 01 . . . . C0, M - 1 
C10 C11 . . . . C1, M - 1 


. . . . . . .
  A 1 ( L, N ) C A 1 ( M , N ) T


Decomposition of LN x MN
.
. . . .
.
.


size DCT block to L x M
CL  1,0 CL  1,1 .......... .... CL - 1, M - 1
adjacent N x N DCT blocks
22
DCT domain Filtering
24
Type-I Symmetric Extension

For type-I DCT symmetric extension of N+1
sample take place in the following:
xˆ (n)  x(n)
n  0,1,
 2,..., N
 x(2 N  n) n  N  1,...,2 N  1
25
Type-II Symmetric Extension

For type-II DCT the symmetric extension of
input sequence is done in the following
manner:
xˆ (n)  x(n)
n  0,1,
 2,...,N  1
 x(2 N  n) n  N ,N  1,...,2 N  1
26
Symmetric Convolution
Let type –II symmetric extension of x(n) be
denoted by xˆ (n) and type –I symmetric
extension of h(n) be denoted as hˆ(n) .
Symmetric convolution of x(n) and h(n) defined as
follows:
27
CONVOLUTION-MULTIPLICATION PROPERTY
x(n)
h(n)
y(n)=h(n)*x(n)
•Symmetric convolution
operation
XII(k)
HI(k)
YII(k)=HI(k).XII(k)
Martucci (1994), IEEE SP
28
Filtering Approaches
1. Based on Symmetrical Convolution
2. Linear filtering with DCT of filter matrices
29
1. Filtering with Symmetric Convolution
Type-I
DCT block
of
Filter Impulse
Response
Type-II
DCT block
of
Signal/image
DCT
domain
Filtering
Type-II
DCT block
of
Filtered
Signal/image
Note:- JPEG compression scheme uses type-II DCT,
transform domain filtering directly gives the type-II DCT.
Martucci and Mersereau, IEEE ASSSP 1993,
Martucci, IEEE SP 1994,
Mukherjee and Mitra LNCS 2006
30
BOUNDARY EFFECT IN
BLOCK DCT DOMAIN
Linear Convolution
Block Symmetric
Convolution
31
Computation with larger blocks: The
BFCD filtering Algorithm
Filtered image
Larger DCT blocks
Convolution
Multiplication
Filtering
Operations
Block
Composition
Original Image
8x8 blocks
Original image
Larger DCT blocks
Block
Decomposition
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)
Filtered Image
8x8 blocks
32
Exact Computation: The Overlapping
BFCD (OBFCD) filtering Algorithm
Filtered image
Larger DCT blocks
Convolution
Multiplication
Filtering
Operations
Block
Composition
Original Image
8x8 blocks
Original image
Larger DCT blocks
Accept
Central
Blocks
Block
Decomposition
Filtered Image
8x8 blocks
33
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)
Filtered Images
BFCD Algorithm (5x5)
OBFCD Algorithm
34
Computational Cost: BFCD
L(=M)
Per Pixel
Computation
Equivalent
Operations
2
28 M + 32 A
116
3
46.17 M + 53.33 A
191.84
4
71.5 M + 96 A
310.5
35
Computational Cost: OBFCD
L(=M)
Per Pixel
Computation
Equivalent
Operations
3
103.88 M + 120 A
431.64
4
127.11 M + 170.62 A
552
5
144.92 M + 180 A
614.76
36
Removal of Blocking Artifacts
Blocking artifacts
Compressed with quality
factor 10
Artifacts removed
By filtering in DCT
domain
37
Image Sharpening in DCT domain
Lena
Peppers
38
2. Linear Filtering in the block DCT domain
Here, both filter matrices and input both in Type-II DCT space.
Spatial Domain:
DCT Domain:
Merhav and Bhaskaran, HPL-95-56, 1996, Kresch and Merhav, IEEE IP 1999,
Yim, IEEE CSVT 20004
39
Filtering in the block DCT domain
 DCT convolution-multiplication property reduces the cost of
filtering.
 Type-I DCT of the filter and the Type-II DCT data are used.
 To avoid boundary effect, Filtering is performed on the larger DCT
block.
 The filtering technique by Mukherjee and Mitra
 Three separate steps: composition, filtering and decomposition of
the DCT blocks.
 Cost of filtering is high.
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)
 In this work, three operations are computed in a single step.
40
CMP Filtering: an improvement
Input
1. Compose
B_1
B0
Filter
response
B1
Point wise
Type-II DCT, B
Type-I DCT of Filter
Multiply
2. Filter
3. Decompose
Type-II DCT, Bf
Bf-1
Bf 0
Bf 1
Steps 1-3: combined step
41
Filtering in the block DCT space
1.
2.
3.
5.
4.
42
Filtering in the block DCT space
In 2-D:
43
Gaussian Filtering…
44
Results: Gaussian filter
Close to 300 dB
Note:- Sparse DCT block
4x4
45
Performance: Gaussian filtering
Variation in
Number of blocks
No variation
in PSNR and cost
Cost varies with L and M
46
Threshold variation
47
Performance of proposed filtering for
noisy images
48
Results:Threshold variation
Original Images
Threshold =10-3
Threshold =10-4
49
Blocking artifacts removal
50
Image Sharpening
Sharpened
Images
51
Observations
 The proposed technique provides equivalent
results as obtained by Linear convolution.
 Complexity can be reduced by varying the threshold
value.
 The technique is comparable with existing
techniques.
 The technique is based on simple linear
operations.
52
Thanks
53