Image Analysis Image Restoration Image Restoration g ( x, y ) h( x, y) * f ( x, y) n( x, y) Image enhancement tries to improve subjective image quality. Image restoration tries to recover the original image. Bahadir K. Gunturk EE4780 2 Noise Models • Noise may arise during image due to sensors, digitization, transmission, etc. • Most of the time, it is assumed that noise is independent of spatial coordinates, and that there is no correlation between noise component and pixel value. • Noise may be considered as a random variable, its statistical behavior is characterized by a probability density function (PDF). Gaussian noise 1 p( z ) e 2 Bahadir K. Gunturk 2 z 2 2 EE4780 3 Noise Models Uniform noise Bahadir K. Gunturk Impulse (salt-and-pepper) noise EE4780 4 Noise Models Original Noisy images and their histograms Bahadir K. Gunturk EE4780 5 Noise Models How to estimate noise parameters? • If imaging device is available • Take a picture of a flat surface. • See the shape of the histogram; decide on the noise model. • Estimate the parameters. (e.g., find mean and standard deviation.) • When only images already generated are available • Get a small patch of image with constant gray level • Inspect histogram • Estimate the parameters Bahadir K. Gunturk EE4780 6 Restoration When There is Only Noise g ( x , y ) f ( x , y ) n ( x, y ) Low-Pass Filters: • Smoothes local variations in an image. • Noise is reduced as a result of blurring. • For example, Arithmetic Mean Filter is ˆf ( x, y ) 1 g ( s, t ) mn ( s ,t )N xy Bahadir K. Gunturk EE4780 Convolve with a uniform filter of size m-by-n. 7 Restoration When There is Only Noise Adaptive, 2local noise reduction filter • Let n ( x, y ) be the noise variance at (x,y). • Let L2 ( x, y ) be the local variance of pixels around (x,y). mL be the local mean of pixels around (x,y). • Let We want a filter such that • If noise variance is zero, it should return g(x,y). • If local variance is high relative to noise variance, the filter should return a value close to g(x,y). (Therefore, edges are preserved!) • If two variances are equal, the filter should return the average of the pixels within the neighborhood. 2 ˆf ( x, y ) g ( x, y ) n ( x, y) g ( x, y ) m L L2 ( x, y) Bahadir K. Gunturk EE4780 8 Restoration When There is Only Noise Bahadir K. Gunturk EE4780 9 Restoration When There is Only Noise Median Filter • Replaces the value of a pixel by the median of intensities in the neighborhood of that pixel. • Is very effective against the salt-and-pepper noise. Bahadir K. Gunturk EE4780 10 Restoration When There is Only Noise Adaptive Median Filter: The basic idea is to avoid extreme values • Let • z_min: minimum gray level value in a neigborhood of a pixel at (x,y). • z_max: maximum gray level value… • z_med: median… • z(x,y): gray level at (x,y). • Is z_med=z_min or z_med=z_max? (That is, is z_med an extreme value?) • No: • Is z(x,y) an extreme value? (Is z(x,y)=z_min or z(x,y)=z_max?) • No: Output is z(x,y) • Yes: Output is z_med. • Yes: Increase window size (to find a non-extreme z_med) and go to the first step. (When a maximum allowed window size is reached, stop and output z(x,y).) Bahadir K. Gunturk EE4780 11 Restoration When There is Only Noise Bahadir K. Gunturk EE4780 12 Restoration When There is Only Noise Removing Periodic Noise with Band-Reject Filters Spikes are due to noise Periodic Noise Band-reject filter Bahadir K. Gunturk EE4780 13 Restoration When There is Only Noise Finding Periodic Noise from the Spectrum and Using Notch Filters Filter out these spikes Noise due to interference Bahadir K. Gunturk EE4780 14 Image Restoration Spatial domain: g ( x, y ) h( x, y) * f ( x, y) n( x, y) Frequency domain: G (u, v) H (u, v) F (u, v) N (u, v) Bahadir K. Gunturk EE4780 15 Image Restoration Inverse Filtering G (u, v) H (u, v) F (u, v) N (u, v) Fˆ (u, v) Bahadir K. Gunturk 1 N (u, v) G(u, v) F (u, v) H (u, v) H (u, v) EE4780 This could dominate signal. 16 Image Restoration Bahadir K. Gunturk EE4780 17 Image Restoration Cut off the inverse filter for large frequencies. (Signal-to-noise ratio is typically low for large frequencies.) Bahadir K. Gunturk EE4780 18 Image Restoration Minimum Mean Square Error (Wiener) Filtering: • Find Fˆ (u , v) such that the expected value of error is minimized: Error E Fˆ (u, v) F (u, v) 2 • Solution is 2 H ( u , v ) 1 Fˆ (u, v) 2 H (u, v) N ( u , v ) 2 H (u , v) 2 F ( u , v ) G (u, v) Investigate this equation for different signal-noise ratios. Bahadir K. Gunturk EE4780 19 Original image Bahadir K. Gunturk EE4780 20 Image Restoration Bahadir K. Gunturk EE4780 21 Least Squares Filtering G (u, v) H (u, v) F (u, v) N (u, v) Find F(u,v) that minimizes the following cost function: Cost G (u , v) H (u , v) F (u , v) 2 The solution is H * (u, v) Fˆ (u, v) G(u, v) 2 H (u, v) (Unconstrained solution) (See the derivations in the classroom) Bahadir K. Gunturk EE4780 22 Least Squares Filtering G (u, v) H (u, v) F (u, v) N (u, v) Find F(u,v) that minimizes the following cost function: Cost G (u , v) H (u, v) F (u, v) P(u, v) F (u, v) 2 2 Choose a P(u,v) to have a smooth solution. (A high-pass filter would do the trick.) Bahadir K. Gunturk EE4780 23 Least Squares Filtering The frequency domain solution to this optimization problem is 2 1 H ( u , v ) Fˆ (u, v) G(u, v) 2 2 H (u, v) H (u, v) P(u, v) (Constrained solution) where P(u,v) is the Fourier Transform of p(x,y), which is typically chosen as a high-pass filter. Example: Bahadir K. Gunturk 0 1 0 p( x, y ) 1 4 1 0 1 0 EE4780 24 Least Squares Filtering Bahadir K. Gunturk EE4780 25