CS489-02 & CS589-02 Multimedia Processing Lecture 2. Intensity Transformation and Spatial Filtering Spring 2009 Spatial Domain vs. Transform Domain Spatial domain Image plane itself, directly process the intensity values of the image plane Transform domain Process the transform coefficients, not directly process the intensity values of the image plane 4/13/2015 2 Spatial Domain Process g ( x, y ) T [ f ( x, y )]) f ( x, y ) : input image g ( x, y ) : output image T : an operator on f defined over a neighborhood of point ( x, y) 4/13/2015 3 Spatial Domain Process 4/13/2015 4 Spatial Domain Process Intensity transformation function s T (r ) 4/13/2015 5 Some Basic Intensity Transformation Functions 4/13/2015 6 Image Negatives Image negatives s L 1 r 4/13/2015 7 Example: Image Negatives Small lesion 4/13/2015 8 Log Transformations Log Transformations s c log(1 r ) 4/13/2015 9 Example: Log Transformations 4/13/2015 10 Power-Law (Gamma) Transformations s cr 4/13/2015 11 Example: Gamma Transformations 4/13/2015 12 Example: Gamma Transformations 4/13/2015 13 Piecewise-Linear Transformations Contrast Stretching — Expands the range of intensity levels in an image ― spans the full intensity range of the recording medium Intensity-level Slicing — Highlights a specific range of intensities in an image 4/13/2015 14 4/13/2015 15 Highlight the major blood vessels and study the shape of the flow of the contrast medium (to detect blockages, etc.) 4/13/2015 Measuring the actual flow of the contrast medium as a function of time in a series of images 16 Bit-plane Slicing 4/13/2015 17 Example 4/13/2015 18 Example 4/13/2015 19 Histogram Processing Histogram Equalization Histogram Matching Local Histogram Processing Using Histogram Statistics for Image Enhancement 4/13/2015 20 Histogram Processing Histogram h( rk ) nk rk is the k th intensity value nk is the number of pixels in the image with intensity rk nk Normalized histogram p (rk ) MN nk : the number of pixels in the image of size M N with intensity rk 4/13/2015 21 4/13/2015 22 Histogram Equalization The intensity levels in an image may be viewed as random variables in the interval [0, L-1]. Let pr (r ) and ps ( s) denote the probability density function (PDF) of random variables r and s. 4/13/2015 23 Histogram Equalization s T (r ) 0 r L 1 a. T(r) is a strictly monotonically increasing function in the interval 0 r L -1; b. 0 T (r ) L -1 for 0 r L -1. 4/13/2015 24 Histogram Equalization s T (r ) 0 r L 1 a. T(r) is a strictly monotonically increasing function in the interval 0 r L -1; b. 0 T (r ) L -1 for 0 r L -1. T (r ) is continuous and differentiable. ps (s)ds pr (r )dr 4/13/2015 25 Histogram Equalization r s T (r ) ( L 1) pr (w)dw 0 ds dT (r ) d r ( L 1) p ( w ) dw r 0 dr dr dr ( L 1) pr (r ) pr (r )dr pr (r ) 1 pr ( r ) ps ( s ) ( L 1) pr (r ) L 1 ds ds dr 4/13/2015 26 Example Suppose that the (continuous) intensity values in an image have the PDF 2r , 2 pr (r ) ( L 1) 0, for 0 r L-1 otherwise Find the transformation function for equalizing the image histogram. 4/13/2015 27 Example r s T (r ) ( L 1) pr (w)dw 0 ( L 1) r 0 2w dw 2 ( L 1) 2 r L 1 4/13/2015 28 Histogram Equalization Continuous case: r s T (r ) ( L 1) pr (w)dw 0 Discrete values: k sk T (rk ) ( L 1) pr (rj ) j 0 L 1 k ( L 1) nj MN j 0 j 0 MN k 4/13/2015 nj k=0,1,..., L-1 29 Example: Histogram Equalization Suppose that a 3-bit image (L=8) of size 64 × 64 pixels (MN = 4096) has the intensity distribution shown in following table. Get the histogram equalization transformation function and give the ps(sk) for each sk. 4/13/2015 30 Example: Histogram Equalization 0 s0 T (r0 ) 7 pr (rj ) 7 0.19 1.33 1 s1 T (r1 ) 7 pr (rj ) 7 (0.19 0.25) 3.08 3 j 0 1 j 0 4/13/2015 s2 4.55 5 s3 5.67 6 s4 6.23 6 s5 6.65 7 s6 6.86 7 s7 7.00 7 31 Example: Histogram Equalization 4/13/2015 32 4/13/2015 33 4/13/2015 34 Question Is histogram equalization always good? No 4/13/2015 35 Histogram Matching Histogram matching (histogram specification) —A processed image has a specified histogram Let pr (r ) and pz ( z ) denote the continous probability density functions of the variables r and z. pz ( z ) is the specified probability density function. Let s be the random variable with the probability r s T (r ) ( L 1) pr ( w)dw 0 Obtain a transformation function G z G ( z ) ( L 1) pz (t )dt s 0 4/13/2015 36 Histogram Matching r s T (r ) ( L 1) pr ( w)dw 0 z G ( z ) ( L 1) pz (t ) dt s 0 1 z G (s) G 4/13/2015 1 T (r) 37 Histogram Matching: Procedure Obtain pr(r) from the input image and then obtain the values of s r s ( L 1) pr ( w)dw 0 Use the specified PDF and obtain the transformation function G(z) z G( z ) ( L 1) pz (t )dt s 0 Mapping from s to z z G 1 (s) 4/13/2015 38 Histogram Matching: Example Assuming continuous intensity values, suppose that an image has the intensity PDF 2r , 2 pr (r ) ( L 1) 0, for 0 r L -1 otherwise Find the transformation function that will produce an image whose intensity PDF is 3z 2 , 3 pz ( z ) ( L 1) 0, 4/13/2015 for 0 z ( L -1) otherwise 39 Histogram Matching: Example Find the histogram equalization transformation for the input image r r 0 0 s T (r ) ( L 1) pr ( w)dw ( L 1) 2 r 2w dw 2 ( L 1) L 1 Find the histogram equalization transformation for the specified histogram z z 0 0 G( z ) ( L 1) pz (t )dt ( L 1) The transformation function 1/3 z ( L 1) s 2 4/13/2015 3t 2 z3 dt s 3 2 ( L 1) ( L 1) 1/3 2 r ( L 1) L 1 2 2 1/3 ( L 1)r 40 Histogram Matching: Discrete Cases Obtain pr(rj) from the input image and then obtain the values of sk, round the value to the integer range [0, L1]. k k ( L 1) sk T (rk ) ( L 1) pr (rj ) nj MN j 0 j 0 Use the specified PDF and obtain the transformation function G(zq), round the value to the integer range [0, L-1]. q G( zq ) ( L 1) pz ( zi ) sk i 0 Mapping from sk to zq zq G1 (sk ) 4/13/2015 41 Example: Histogram Matching Suppose that a 3-bit image (L=8) of size 64 × 64 pixels (MN = 4096) has the intensity distribution shown in the following table (on the left). Get the histogram transformation function and make the output image with the specified histogram, listed in the table on the right. 4/13/2015 42 Example: Histogram Matching Obtain the scaled histogram-equalized values, s0 1, s1 3, s2 5, s3 6, s4 7, s5 7, s6 7, s7 7. Compute all the values of the transformation function G, 0 G( z0 ) 7 pz ( z j ) 0.00 0 j 0 G ( z1 ) 0.00 G ( z3 ) 1.05 0 1 G ( z5 ) 4.55 5 G ( z7 ) 7.00 7 4/13/2015 G( z2 ) 0.00 0 G( z4 ) 2.45 2 G( z6 ) 5.95 6 43 Example: Histogram Matching 4/13/2015 44 Example: Histogram Matching Obtain the scaled histogram-equalized values, s0 1, s1 3, s2 5, s3 6, s4 7, s5 7, s6 7, s7 7. Compute all the values of the transformation function G, 0 G( z0 ) 7 pz ( z j ) 0.00 0 j 0 G ( z1 ) 0.00 G ( z3 ) 1.05 0 1 s0 G ( z5 ) 4.55 5 G ( z7 ) 7.00 7 4/13/2015 s2 G( z2 ) 0.00 0 G( z4 ) 2.45 2 s1 G( z6 ) 5.95 6 s3 s4 s5 s6 s7 45 Example: Histogram Matching s0 1, s1 3, s2 5, s3 6, s4 7, s5 7, s6 7, s7 7. rk 4/13/2015 0 1 2 3 4 5 6 7 46 Example: Histogram Matching rk zq 03 1 4 25 36 47 57 67 77 4/13/2015 47 Example: Histogram Matching 4/13/2015 48 Example: Histogram Matching 4/13/2015 49 4/13/2015 50 Local Histogram Processing Define a neighborhood and move its center from pixel to pixel At each location, the histogram of the points in the neighborhood is computed. Either histogram equalization or histogram specification transformation function is obtained Map the intensity of the pixel centered in the neighborhood Move to the next location and repeat the procedure 4/13/2015 51 Local Histogram Processing: Example 4/13/2015 52 Using Histogram Statistics for Image Enhancement Average Intensity L 1 m ri p(ri ) i 0 1 MN M 1 N 1 f ( x, y) x 0 y 0 L 1 un (r ) (ri m)n p(ri ) i 0 Variance L 1 u2 (r ) (ri 2 i 0 4/13/2015 1 2 m) p(ri ) MN M 1 N 1 f ( x, y) m 2 x 0 y 0 53 Using Histogram Statistics for Image Enhancement Local average intensity L 1 msxy ri psxy (ri ) i 0 sxy denotes a neighborhood Local variance 4/13/2015 L 1 2 sxy (ri msxy ) psxy (ri ) 2 i 0 54 Using Histogram Statistics for Image Enhancement: Example E f ( x, y ), if msxy k0 mG and k1 G sxy k2 G g ( x, y ) otherwise f ( x, y ), mG : global mean; G : global standard deviation k0 0.4; k1 0.02; k2 0.4; E 4 4/13/2015 55 Spatial Filtering A spatial filter consists of (a) a neighborhood, and (b) a predefined operation Linear spatial filtering of an image of size MxN with a filter of size mxn is given by the expression g ( x, y) a b w(s, t ) f ( x s, y t ) s a t b m 2a 1; 4/13/2015 n 2b 1 56 Spatial Filtering 4/13/2015 57 Spatial Correlation The correlation of a filter w( x, y) of size m n with an image f ( x, y), denoted as w( x, y ) f ( x, y) w( x, y) f ( x, y) a b w(s, t ) f ( x s, y t ) s a t b 4/13/2015 58 Spatial Convolution The convolution of a filter w( x, y ) of size m n with an image f ( x, y), denoted as w( x, y ) f ( x, y) w( x, y) f ( x, y) a b w(s, t ) f ( x s, y t ) s a t b 4/13/2015 59 4/13/2015 60 Smoothing Spatial Filters Smoothing filters are used for blurring and for noise reduction Blurring is used in removal of small details and bridging of small gaps in lines or curves Smoothing spatial filters include linear filters and nonlinear filters. 4/13/2015 61 Spatial Smoothing Linear Filters The general implementation for filtering an M N image with a weighted averaging filter of size m n is given a g ( x, y ) b w(s, t ) f ( x s, y t ) s a t b a b w(s, t ) s a t b where m 2a 1, 4/13/2015 n 2b 1. 62 Two Smoothing Averaging Filter Masks 4/13/2015 63 4/13/2015 64 Example: Gross Representation of Objects 4/13/2015 65 Order-statistic (Nonlinear) Filters — Nonlinear — Based on ordering (ranking) the pixels contained in the filter mask — Replacing the value of the center pixel with the value determined by the ranking result E.g., median filter, max filter, min filter 4/13/2015 66 Example: Use of Median Filtering for Noise Reduction 4/13/2015 67 Sharpening Spatial Filters ► Foundation ► Laplacian Operator ► Unsharp Masking and Highboost Filtering ► Using First-Order Derivatives for Nonlinear Image Sharpening 4/13/2015 68 Sharpening Spatial Filters: Foundation ► The first-order derivative of a one-dimensional function f(x) is the difference f f ( x 1) f ( x) x ► The second-order derivative of f(x) as the difference 2 f f ( x 1) f ( x 1) 2 f ( x) 2 x 4/13/2015 69 4/13/2015 70 Sharpening Spatial Filters: Laplace Operator The second-order isotropic derivative operator is the Laplacian for a function (image) f(x,y) 2 2 f f 2 f 2 2 x y 2 f f ( x 1, y ) f ( x 1, y) 2 f ( x, y) 2 x 2 f f ( x, y 1) f ( x, y 1) 2 f ( x, y) 2 y 2 f f ( x 1, y) f ( x 1, y ) f ( x, y 1) f ( x, y 1) - 4 f ( x, y) 4/13/2015 71 Sharpening Spatial Filters: Laplace Operator 4/13/2015 72 Sharpening Spatial Filters: Laplace Operator Image sharpening in the way of using the Laplacian: g ( x, y) f ( x, y) c 2 f ( x, y) where, f ( x, y ) is input image, g ( x, y ) is sharpenend images, c -1 if 2 f ( x, y ) corresponding to Fig. 3.37(a) or (b) and c 1 if either of the other two filters is used. 4/13/2015 73 4/13/2015 74 Unsharp Masking and Highboost Filtering ► Unsharp masking Sharpen images consists of subtracting an unsharp (smoothed) version of an image from the original image e.g., printing and publishing industry ► Steps 1. Blur the original image 2. Subtract the blurred image from the original 3. Add the mask to the original 4/13/2015 75 Unsharp Masking and Highboost Filtering Let f ( x, y ) denote the blurred image, unsharp masking is g mask ( x, y ) f ( x, y ) f ( x, y ) Then add a weighted portion of the mask back to the original g ( x, y ) f ( x, y ) k * g mask ( x, y ) k 0 when k 1, the process is referred to as highboost filtering. 4/13/2015 76 Unsharp Masking: Demo 4/13/2015 77 Unsharp Masking and Highboost Filtering: Example 4/13/2015 78 Image Sharpening based on First-Order Derivatives For function f ( x, y ), the gradient of f at coordinates ( x, y ) is defined as f g x x f grad( f ) g y f y The magnitude of vector f , denoted as M ( x, y) Gradient Image 4/13/2015 M ( x, y) mag(f ) g x 2 g y 2 79 Image Sharpening based on First-Order Derivatives The magnitude of vector f , denoted as M ( x, y) M ( x, y) mag(f ) g x 2 g y 2 M ( x, y) | g x | | g y | z1 z4 z7 4/13/2015 z2 z5 z8 z3 z6 z9 M ( x, y) | z8 z5 | | z6 z5 | 80 Image Sharpening based on First-Order Derivatives Roberts Cross-gradient Operators M ( x, y) | z9 z5 | | z8 z6 | z1 z4 z7 4/13/2015 z2 z5 z8 z3 z6 z9 Sobel Operators M ( x, y) | ( z7 2 z8 z9 ) ( z1 2 z2 z3 ) | | ( z3 2 z6 z9 ) ( z1 2 z4 z7 ) | 81 Image Sharpening based on First-Order Derivatives 4/13/2015 82 Example 4/13/2015 83 Example: Combining Spatial Enhancement Methods Goal: Enhance the image by sharpening it and by bringing out more of the skeletal detail 4/13/2015 84 Example: Combining Spatial Enhancement Methods Goal: Enhance the image by sharpening it and by bringing out more of the skeletal detail 4/13/2015 85