siggraph2008 - Microsoft Research
advertisement
Inverse Texture Synthesis Li-Yi Wei1 Jianwei Han2 Kun Zhou1,2 Hujun Bao2 Baining Guo1 Harry Shum1 1Microsoft 2Zhejiang University Example-based texture synthesis For a small input texture produce an arbitrarily large output with similar look Why? may not possible to obtain large input texture synthesis input output Inverse texture synthesis From a large input texture produce a small output that best summarizes input inverse texture synthesis output input Why? Textures are getting large Advances in scanning technology High dimensionality: time-varying, BRDF Expensive to store, transmit, compute Yale University MSR Asia Columbia University Overview input (large) inverse texture synthesis output (small) texturing (fast) texturing (slow) similar quality Related work: image compression pixel-wise identical compress input inverse synth decompress texture synth perceptual similar Related work: epitome Epitome [Jojic et al. 2003] Jigsaw [Kannan et al. 2007] Major source of inspiration for us For general images, not just textures We provide better quality Bidirectional similarity [Simakov et al. 2008] Factoring repeated content [Wang et al. 2008] Related work: manual crop stationary globally varying original manual crop our result Globally-varying textures Markov Random Field (MRF) textures local & stationary Globally-varying textures local, but not necessarily stationary MRF globally varying Globally varying textures Previous work MRF input → globally varying output texture-by-numbers in Image analogies [Hertzmann et al. 2001] progressively variant textures [Zhang et al. 2003] texture design and morphing [Matusik et al. 2005] Globally varying input appearance manifold [Wang et al. 2006] spatially & time varying BRDF [Gu et al. 2006] context-aware texture [Lu et al. 2007] Globally varying textures Definition texture + control maps Examples of control maps user-specified colors [Hertzmann et al. 2001] spatially-varying parameters [Gu et al. 2006] texture (paint crack) weathering degree-map [Wang et al. 2006] context information [Lu et al. 2007] control map (paint thickness) Globally varying textures Including time-varying textures as well Large data size! time-varying BRDF [Gu et al. 2006] 512 x 512 x 33, 288 MB context-aware texture [Lu et al. 2007] 1226 x 978 x 50, 35 MB Inverse texture synthesis Compacting globally varying textures including both texture + control map inverse synthesis texture control map input texture control output compaction Compaction as summary of original Re-synthesis with user control map faster slower forward synthesis + compaction user control re-synthesis re-synthesis from from compaction original Basic formulation Inspired by texture optimization [Kwatra et al. 2005] 1 2 E ( x; z ) † | x p z p | † | xq zq |2 | X | pX † | Z | qZ † inverse term (New!) forward term [Kwatra et al. 2005] xp Zp zq Z (output) xq X (input) energy Energy plot 1 2 E ( x; z ) † | x p z p | † | xq zq |2 | X | pX † | Z | qZ † original compaction size Why both terms? 1 E ( x; z ) |x z | |x |X | |Z | 2 † pX † p p inverse † qZ † 2 z | q q forward inverse term preserves all input features forward term avoids artifacts in compaction f-only both missing feature i-only both garbage i-only both discontinuity Comparing with epitome [Jojic et al. 2003] Similar to our method but only inverse term blur, discontinuity epitome epitome our original our original Comparing with epitome [Jojic et al. 2003] Re-synthesis epitome epitome our our original original Solver 1 2 E ( x; z ) † | x p z p | † | xq zq |2 | X | pX † | Z | qZ † How to solve this? Texture optimization [Kwatra et al. 2005] Discrete solver [Han et al. 2006] Optimization [Kwatra et al. 2005] 1 2 E ( x; z ) † | x p z p | † | xq zq |2 | X | pX † | Z | qZ † NO inverse term E-step fix xq argminz E(x,z) least square M-step fix Z argminxq |xq-zq|2 search forward term [Kwatra et al. 2005] xq Zq zq Z xq X Our solver 1 2 E ( x; z ) † | x p z p | † | xq zq |2 | X | pX † | Z | qZ † inverse term forward term [Kwatra et al. 2005] E-step fix xq xp argminz E(x,z) least square discrete solver [Han et al. 2006] M-step (forward) M-step (inverse) fix Z fix xp argminxq |xq-zq|2 argminzp |xp-zp|2 discrete search solver discrete solver xq Zqzp zq Z xp xq X Results GPU synthesis – small texture better Extension from [Lefebvre & Hoppe 2005] 3 fps, original 6 fps, compact cheese mold 1214 x 1212 1282 3.5 fps, original 7.0 fps, compact dirt 271x481 original 1282 compaction Limitation: Correlation between texture & control texture control original reconstruction compaction Orientation field for anisotropic textures Orientation field w as part of energy function E(x, z) → E(x, z; w) Good orientation field yields better solution comp. no w original comp. with w orientation field Future work Higher dimensional textures e.g. video General images, not just textures Bidirectional similarity [Simakov et al. CVPR 2008] Image compression Acknowledgements Yale graphics group Bennett Wilburn Columbia graphics group Eric Stollnitz Sylvain Lefebvre Dwight Daniels Hughes Hoppe Reviewers Matusik et al. 2005 Dinesh Manocha Mayang.com Ming Lin Jiaping Wang Chas Boyd Xin Tong Brandon Lloyd Jian Sun Avneesh Sud Frank Yu Billy Chen Thank You!
