Texture Optimization for Example-based Synthesis
Vivek Kwatra, Irfan Essa, Aaron Bobick, Nipun Kwatra
Computational Perception Laboratory, GVU Center
College of Computing, Georgia Tech
eorgia
ech
Computational
Perception
Lab
gvu center
http://www.cc.gatech.edu/cpl/projects/textureoptimization/
Goals
Flow-guided Texture Animation
Synthesis Algorithm
Animated texture sequence: Texture appears to follow given
flow field. Sub-goals:
Optimization-based approach for texture synthesis:
Energy Minimization using a simple iterative algorithm
Alternate between X and {Zp} as optimization variables
1. Initialize output texture X randomly
Explicit improvement of texture quality
2. Find input neighborhoods {Zp} nearest to output neighborhoods {Xp}
Controllable synthesis: Flow-guided texture animation
3. Minimize Σ||Xp - Zp|| w.r.t X (linear solve)
Texture Energy
Texture Energy measures quality of the synthesized
texture w.r.t. a given input sample
1. Define Energy for a single neighborhood
Zq
Z
Xq
X0
Xp
Blend
4. Repeat until convergence
Nearest Neighbor
X
X
Single Neighborhood Energy = ||Xp - Zp||
Zp
Multi-resolution synthesis:
Full, Half, Quarter scales
32x32, 16x16, 8x8 nbd sizes
300
Level 1
3-5 iterations per level
7-10 min. for 256x256 textures
Flow Energy and Optimization
Level 4
Level 6
250
200
X0
Level 2
Level 3
Level 7
50
0
X1
Level 5
100
0
5
10
15
20
25
30
35
40
45
Xq
Flow Energy = ||X1 - W0||2
Warp
W0
50
Number of Iterations
Xp
Frame-by-Frame Synthesis
Example flow field
150
2. Total Energy = Sum over single neighborhood energies
Region of overlap:
accumulates error on mismatch
Correct
W0
350
2
Zq
Xn
400
Xp
Z
X1
Warp
Results
pixel p
Z
2. Texture Similarity: Shape, size, orientation of texture
elements should be similar to input texture
Zp
Texture Energy
Zp
2
1. Flow Consistency: Perceived motion should be similar
to flow
Optimize Total Energy = Flow Energy + Texture Energy
Zp
X
INPUT
WEI - LEVOY
IMAGE Q
U ILTING
GRAPH
- CUTS
Z
TEXTURE - OPTIMIZ
ATION
Wp
Zq
Xq
Xp
Blend
W
X
Results
ROTATIONAL FLOW
SOURCE
SINK
VORTEX
FRAME
1
FRAME
40 OUR RESULT
FRAME
40 WARPING
COMPLEX
Each frame synthesized at
single resolution
20-60 seconds per frame