Image Enhancement (Spatial Domain Filters) Spatial Domain • Spatial Filtering – (As opposed to frequency domain filter) – For linear filters • Foundation based on the convolution theorem – g(x,y) = h(x,y)*f(x,y) – G(u,v) = H(u,v)F(u,v) • Goal is to either remove, or isolate frequencies in the image • Types – Linear – Non-linear 1 Response of a linear mask R = w1z1 + w2z2 + w3z3 ... + w9z9 Where wi are mask coefficients and zi are pixel intensities. Spatial Filtering using a Mask • Neighborhood operators – Convolution z1 z2 z3 w1 w2 w3 z4 z5 z6 w4 w5 w6 z7 z8 z9 w7 w8 w9 Response of a linear mask, R, R = ( w1z1 + w2z2 + w3z3 ... + w9z9) = 9 ∑ wizi 1 f(x,y) is centered around “z5” so g(x,y) = R =filter_about*f(x,y) 2 Smoothing Filters • Smoothing Filters – blurring • “pre-processing” – removal of small details before object extraction – noise reduction • removal of noise in an image • Often called “Low-Pass” Filters – Filter lets low-frequencies pass – Stops high-frequencies Low-pass spatial filtering 1 0 Frequency Domain 0 Spatial Domain 3 Low-pass spatial filtering •We can build a gaussian filter w1 w2 w3 w4 w5 w6 w7 w8 w9 •But, only requirement for a low-pass filter is that wi be positive •Note, that the result can be larger than the valid output range(L-1) •Can pre-scale the filter scale_factor = (Σ wi )-1 Low-pass spatial filter 1/9 * 1 1 1 1 1 1 1 1 1 Average Filter 4 Extends to larger filters 1 1 1 1/9 * 1 1 1 1 1 1 1/81 * 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Example original n=15 (nxn mask) n=5 (nxn mask) n=25 (nxn mask) 5 Other arrangements 1 1 1 1 2 1 1 1 1 1 1 1 1 1 Gaussian 1 2 3 2 1 1 3 4 3 1 1 2 3 2 1 1 1 1 1 1 Weighted Remember that the resulting gray-levels may be out of the range of the original image. Isotropic Gaussian Filter G ( x, y ) = 1 2πσ 2 − e x2 + y 2 2σ 2 2D Gaussian distribution with mean (0,0) and σ=1 6 Computing a discrete gaussian mask Center the Mask at (0,0). So, a 5x5 mask would compute values by at: -2 -1 0 1 2 -2 G ( x, y ) = -1 0 1 2πσ 2 − e x2 + y 2 2σ 2 Use G(x,y) to compute values for mask. 1 2 Often scaled by Max-Gray Level; ie, 255*G(x,y) Computing a discrete gaussian mask G ( x, y ) = 1 2πσ 2 − e x2 + y 2 2σ 2 7 Linear vs. Non-linear • “Convolution”/Linear Filters – Linear operation – Have corresponding frequency domain filter • Non-linear Filters – Mask used to determine the proper substitution of a “good” pixel value – Examine neighbors using various orderings • Often use Rank or Order Statistics – Harder to interpret effect in frequency domain Selected Smoothing • (non-linear filter) • kth nearest neighbors 2 3 8 3 4 10 4 2 9 Find the kth nearest neighbors of (x,y) in a neighborhood nxn k=6 nearest neighbors = {2,3,3,4,4,2} (x,y) = (2+3+3+4+4+2} * (1/6) = 3 Note: Straight Averaging would have equaled 5 pixel values about (x,y) window 3x3 8 Max Homogeneity Smoothing • non-linear • Use 5 different windows with f(x,y) included in all of them Max Homogeneity Smoothing • Compute the gradient of each of these “windows” • f(x,y) is the average of the window with the smallest “variation” 9 Max Homogeneity Smoothing • This filter is trying to find the window that (x,y) is the most similar too. ? Median Filter • • • • Very popular non-linear filter Find the median of the window Preserves edges Removes impulse noise, avoids excessive smoothing 2 3 8 3 4 10 4 2 9 neighbor sort = {2,2,3,3,4,4,8,9,10} f(x,y) = median pixel values about (x,y) window 3x3 10 Example Noisy Image Original Median Filter Low-Pass Filter Median Filter • Often referred to as “de-speckle operation” • It converges – That is, if you perform it over and over, eventually the image will not change. 11 Iterative Smoothing • Local Averaging – Converges to an image with constant intensity • Max-homogeneity and K-nearest – converges to a constant intensity or a piecewise constant intensity • Median Filter – Converges to an image invariant to the filter Sharpening • Objective of sharpening – Highlight fine detail in an image – Enhance detail that has been blurred – We can think of this as high-pass filtering • Letting High Frequencies Pass • Removing Low Frequencies 12 High-pass spatial filtering 1 0 0 Frequency Domain Spatial Domain Basic hi-pass spatial filter •Positive coefficients near its center 1/9 * -1 -1 -1 -1 8 -1 -1 -1 -1 •Negative coefficients near the outer periphery •Note, the sum of the coefficients is zero •Thus, when the mask is over an area of constant intensity, the result is zero 13 Hi-pass filter • The results may be negative • You’ll need to scale and/or clip so that the gray levels of the result span [0, L-1] • Don’t take absolute value of the response – Bad idea Example Original High-Pass 14 High Pass filter • Another way to think about the high pass filter: – Highpass = Original – Lowpass • Should be able to prove this using the linear ‘R’ response equality and examples of high-pass and low-pass filters High Boost Filter • High Boost = (A)(Original) – Lowpass = (A-1)(Original) + Original - Lowpass = (A-1)Original + Highpass • Setting ‘A’ controls how much of the original image you would like to have show up in the result • • A=1 gives you the high-pass filter A > 1 allows some of the low-frequencies back in – Result, an image that looks more like the original with accents on the high-frequencies 15 Constructing a High Boost Filter 1/9 * -1 -1 -1 -1 w -1 -1 -1 -1 w = 9A-1 Examples Original A=1.1 A=1.15 A=1.2 16 Derivative Filters • Averaging pixels – blur – analogous to integration • Differentiation – has the opposite effect of “blurring” – sharpens an image First Order Derivatives • Gradient • Function of 2 variables x, y ⎡ ∂f ⎤ ⎢ ∂x ⎥ ⎢ ⎥ ∇f = ⎢ ⎥ ⎢ ∂f ⎥ ⎢ ∂y ⎥ ⎣ ⎦ x y 17 First Order Derivatives • For each (x,y) you are storing two values: ⎡ ∂f ⎤ ⎢ ∂x ⎥ ⎢ ⎥ ∇f = ⎢ ⎥ ⎢ ∂f ⎥ ⎢ ∂y ⎥ ⎣ ⎦ • Often have two images to represent this • X-Gradient and Y-Gradient – Can be computed independently First Order Derivatives • X-Gradient and Y-Gradient Original df/dx df/dy f 18 Gradient • Gradient Magnitude ⎡⎛ ∂f ⎞ 2 ⎛ ∂f ⎞ 2 ⎤ ∇f = mag(∇f) = ⎢⎜ ⎟ + ⎜⎜ ⎟⎟ ⎥ ⎢⎣⎝ ∂x ⎠ ⎝ ∂y ⎠ ⎥⎦ 1/ 2 Gradient • Gradient Angle/Direction ⎛ ∂f/∂y ⎞ Ψ (∇f ) = tan −1 ⎜ ⎟ ⎝ ∂f/∂x ⎠ Note: the angle is with respect to the “x” image axis 19 Basic Derivative • Consider the pixels z1 z2 z3 z4 z5 z6 z7 z8 z9 • f at z5 can be computed ∇f ≈ [( z5 − z6 ) 2 + ( z5 − z8 ) 2 ]1/ 2 Basic Derivative • Consider the pixels z1 z2 z3 z4 z5 z6 z7 z8 z9 • f at z5 can be computed quicker ∇f ≈| z5 − z6 | + | z5 − z8 | 20 Basic Derivative z1 z2 z3 z4 z5 z6 z7 z8 z9 • f sometimes is computed using the “cross” difference ∇f ≈| z6 − z8 | + | z5 − z9 | Robert Gradient • Roberts cross-gradient operators 1 1 0 -1 0 0 -1 0 Even masks are awkward to implement Results in a phase-shift 21 Odd Sized Masks ∇f ≈| ( z7 + z8 + z9 ) − ( z1 + z 2 + z3 ) | + | ( z3 + z6 + z9 ) − ( z1 + z 4 + z7 ) | • Difference between first and third row (df/dy) • Difference between first and third column (df/dx) Odd sized mask • Prewitt operator -1 0 1 -1 -1 -1 -1 0 1 0 0 0 -1 0 1 1 1 1 22 Odd sized mask • Sobel operator -1 0 1 -1 -2 -1 -2 0 2 0 0 0 -1 0 1 1 2 1 • Weights closer neighbor a little more Requirements for Derivative Mask • Measuring Change – df/dx = A*f(x,y) – B* f(x+1,y) – df/dy = A*f(x,y) – B*f(x,y+1) • Derivative – Sum of the coefficients has to be zero • This makes sure the derivative is zero in a region of constant intensity 23 Example Second Order Derivative • Laplacian of a 2-D function f(x,y) • is a second-order derivative defined as: ∂ 2f ∂ 2f ∇f= 2 + 2 ∂ x ∂ y 2 24 Second Order Derivative • For a 3x3 region, the form most encountered is: ∇ 2 f ≈ 4 z 5 − ( z 2 + z 4 + z 6 + z8 ) • Requirements – Center pixel coefficient be positive – Outer coefficients be negative – Its a derivative, so sum of coefficient be zero Example of a Laplacian • Typical Laplacian Operator (not used in practice) 0 -1 0 -1 4 -1 0 -1 0 25 Example This is actually an example of a mask which is a Laplacian of a Gaussian (LoG Filter) See pp 582 of GW Spatial Filters • Spatial Convolution – We are convolving a function about each (x,y) – approximation of filters in the frequency domain (at least for the linear filters . . non-linear is hard to analyze) – types • Blurring, Smoothing, Sharpening, Derivatives • and the non-linear (Max Homogeneity, Kth Nearest, Median) • Input gray-levels may be different than output levels – very common – May need to scale your image for visualization • Filter coefficients do not have to be integers – Results are non-integer – use a float image 26