Gaussian Filters - Computer Graphics Laboratory
advertisement
Accelerating Spatially Varying Gaussian Filters Jongmin Baek and David E. Jacobs Stanford University Motivation Input Gaussian Filter Spatially Varying Gaussian Filter Roadmap 1) Accelerating Spatially Varying Gaussian Filters 2) Accelerating Spatially Varying Gaussian Filters 3) Accelerating Spatially Varying Gaussian Filters 4) Applications Gaussian Filters Position Value Given ππ, ππ pairs as input, ˆ π£π = π −β₯ ππ − ππ β₯2 π£π ⋅ exp 2σ2 Gaussian Filters Each output value … ˆ π£π = π −β₯ ππ − ππ β₯2 π£π ⋅ exp 2σ2 Gaussian Filters … is a weighted sum of input values … ˆ π£π = π −β₯ ππ − ππ β₯2 π£π ⋅ exp 2σ2 Gaussian Filters … whose weight is a Gaussian … ˆ π£π = π −β₯ ππ − ππ β₯2 π£π ⋅ exp 2σ2 Gaussian Filters … in the space of the associated positions. ˆ π£π = π −β₯ ππ − ππ β₯2 π£π ⋅ exp 2σ2 Gaussian Filters: Uses Gaussian Blur π π = ππ, ππ , π π = (ππ, ππ, ππ) Gaussian Filters: Uses Bilateral Filter π π = ππ, ππ, ππ, ππ, ππ , π π = (ππ, ππ, ππ) Gaussian Filters: Uses Non-local Means Filter π π = ππ, ππ, π π = (ππ, ππ, ππ) Gaussian Filters: Summary Applications ο± Denoising images and meshes ο± Data fusion and upsampling ο± Abstraction / Stylization ο± Tone-mapping ο± ... Previous work on fast Gaussian Filters ο± Bilateral Grid (Chen, Paris, Durand; 2007) ο± Gaussian KD-Tree (Adams et al.; 2009) ο± Permutohedral Lattice (Adams, Baek, Davis; 2010) Gaussian Filters: Implementations Summary of Previous Implementations: ο± ο± A separable blur flanked by resampling operations. Exploit the separability of the Gaussian kernel. Spatially Varying Gaussian Filters ˆ π£π ⋅ N ππ − ππ ; π 2 πΌ π£π = π Spatially Invariant ˆ π£π = π£π ⋅ N ππ − ππ ; πΎπ π Spatially varying covariance matrix Spatial Variance in Previous Work Trilateral Filter (Choudhury and Tumblin, 2003) ο± ο± Tilt the kernel of a bilateral filter along the image gradient. “Piecewise linear” instead of “Piecewise constant” model. Spatially Varying Gaussian Filters: Tradeoff Benefits: ο± Can adapt the kernel spatially. ο± Better filtering performance. Cost: ο± No longer separable. ο± No existing acceleration schemes. Input Bilateral-filtered Trilateral-filtered Acceleration Problem: ο± Spatially varying (thus non-separable) Gaussian filter Existing Tool: ο± Fast algorithms for spatially invariant Gaussian filters Solution: ο± Re-formulate the problem to fit the tool. ο± Need to obey the “piecewise-constant” assumption Naïve Approach (Toy Example) I LOST THE GAME Input Signal 1 2 1 3 1 4 Desired Kernel filtered w/ 1 filtered w/ 2 filtered w/ 3 filtered w/ 4 1 1 1 2 3 4 Output Signal Challenge #1 In practice, the # of kernels can be very large. Desired Kernel K(x) Range of Kernels needed Pixel Location x Solution #1 Sample a few kernels and interpolate. Desired Kernel K(x) K1 Sampled kernels K2 Interpolate result! K3 Pixel Location x Assumptions Interpolation needs an extra assumption to work: ο± ο± The covariance matrix Ζ©i is either piecewiseconstant, or smoothly varying. Kernel is spatially varying, but locally spatially invariant. Challenge #2 Runtime scales with the # of sampled kernels. Desired Kernel K(x) K1 Sampled kernels K2 K3 Filter only some regions of the image with each kernel. (“support”) Pixel Location x Defining the Support In this example, x needs to be in the support of K1 & K2. Desired Kernel K(x) K1 K2 K3 Pixel Location x Dilating the Support Desired Kernel K(x) K1 K2 K3 Pixel Location x Algorithm 1) Identify kernels to sample. 2) For each kernel, compute the support needed. 3) Dilate each support. 4) Filter each dilated support with its kernel. 5) Interpolate from the filtered results. Algorithm 1) Identify kernels to sample. 2) For each kernel, compute the support needed. 3) Dilate each support. 4) Filter each dilated support with its kernel. 5) Interpolate from the filtered results. K1 K2 K3 Algorithm 1) Identify kernels to sample. 2) For each kernel, compute the support needed. 3) Dilate each support. 4) Filter each dilated support with its kernel. 5) Interpolate from the filtered results. K1 K2 K3 Algorithm 1) Identify kernels to sample. 2) For each kernel, compute the support needed. 3) Dilate each support. 4) Filter each dilated support with its kernel. 5) Interpolate from the filtered results. K1 K2 K3 Algorithm 1) Identify kernels to sample. 2) For each kernel, compute the support needed. 3) Dilate each support. 4) Filter each dilated support with its kernel. 5) Interpolate from the filtered results. K1 K2 K3 Algorithm 1) Identify kernels to sample. 2) For each kernel, compute the support needed. 3) Dilate each support. 4) Filter each dilated support with its kernel. 5) Interpolate from the filtered results. K1 K2 K3 Applications ο± HDR Tone-mapping ο± Joint Range Data Upsampling Application #1: HDR Tone-mapping Base Input HDR Output Detail Tone-mapping Example Bilateral Filter Kernel Sampling Application #2: Joint Range Data Upsampling Range Finder Data Scene Image ο± Sparse ο± Unstructured ο± Noisy Output Synthetic Example Scene Image Ground Truth Depth Synthetic Example Scene Image Simulated Sensor Data Synthetic Example : Result Bilateral Filter Kernel Sampling Synthetic Example : Relative Error Bilateral Filter Kernel Sampling 2.41% Mean Relative Error 0.95% Mean Relative Error Real-World Example Scene Image Range Finder Data *Dataset courtesy of Jennifer Dolson, Stanford University Real-World Example: Result Input Bilateral Naive Kernel Sampling Performance Kernel Sampling Choudhury and Tumblin (2003) Naïve Tonemap1 5.10 s 41.54 s 312.70 s Tonemap2 6.30 s 88.08 s 528.99 s Kernel Sampling Kernel Sampling (No segmentation) Depth1 3.71 s 57.90 s Depth2 9.18 s 131.68 s Conclusion 1. A generalization of Gaussian filters • Spatially varying kernels • Lose the piecewise-constant assumption. 2. Acceleration via Kernel Sampling • Filter only necessary pixels (and their support) and interpolate. 3. Applications
