Texture Synthesis on
[Arbitrary Manifold] Surfaces
Presented by:
Sam Z. Glassenberg*
Topics Investigated Thus Far
• 2D Texture Mapping (CS318)
• 3D Textures (Hypertexture)
• 2D Texture Synthesis
Today’s Goal
Synthesize a texture on a surface by coloring mesh vertices
+
Input Texture
Input Mesh
Result
Vertex Painting
• Advantages
– Speed
• No additional
transformation
required
– Simplicity
– Uniformly distributed
across geometry
• Disadvantages
– Texture resolution
dependent on mesh
complexity
2 SIGGRAPH Papers:
1. Texture Synthesis on Surfaces, by
Greg Turk
2. Texture Synthesis over Arbitrary
Manifold Surfaces, by Li-Yi Wei and
Marc Levoy
Desirable Properties
• Share advantages of 2D algorithm
– Quality
– Efficient
– General
– Easy to use
• Minimum distortion
• Minimum discontinuity
Differences between
Images and Meshes
image
Pixels/Vertices
Local Orientation
(Vector field)
Synthesis Order
Neighborhood
mesh
grid
?
normal
v
u
scanline
?
tangent
?
bitangent
?
Wei/Levoy Solution
image
Pixels/Vertices
Local Orientation
(Vector field)
Synthesis Order
Neighborhood
Wei and Levoy
mesh
grid
?
normal
v
u
scanline
?
?
tangent
mesh re-tiling
[Turk’92]
user-specified
relaxation
random
bitangent
?
random
flattening/resampling
Turk Solution
image
Pixels/Vertices
Local Orientation
(Vector field)
Synthesis Order
Neighborhood
Turk
mesh
grid
repulsion
?
normal
v
u
scanline
?
[Turk’91]
User specified/
interpolated
?
tangent
bitangent
?
sweeping
Surface marching
Both Papers Extend
Fast Texture Synthesis using
Tree-structured Vector
Quantization
Specifically by generalizing their definition
of “search neighborhoods” to apply to
meshes
Texture Synthesis by
Neighborhood Search
Copy
noise
Search
noise
Input pyramid
Output pyramid
Surface Texture Synthesis by
Neighborhood Search (Wei/Levoy)
Search
Input pyramid
Output pyramid
Process
1. Build image/mesh
pyramids
2. Assign texture
orientation/Computation
order
3. Generate texture
What aspects of image pyramids
must we maintain in mesh
pyramids?
• Uniform density
• Power-of-two complexity differences
between levels
Image & Mesh Pyramids
Mesh Retiling [Turk’92]
2 Turk-ish Methods for Mesh Retiling
• Turk ’92, used by
Wei/Levoy
– Uniformly distributes
mesh vertices
– Requires “shooting
normals” to move
between levels
• Turk ’91, used by
Turk
– Uniformly distributes
mesh vertices
– Maintains
parent/child
relationship between
levels
Retiling Density
24576 vertices
73728 vertices
Turk ‘91
• Create a mesh hierarchy in which mesh Mk =
(Vk, Tk) is defined by its Vertices and Triangles
– For the lowest mesh in the hierarchy, place n points
on the surface
– Use repulsion to distribute points easily
– Add 3n points to make the next level
– Repeat
• Connect points by projecting nearby points
onto a tangent plane
• Perform Delaunay triangulation to reduce
triangles
Now, we need to determine
orientation…
In Turk’s method, we use this orientation to
determine the computation order (Surface
Sweeping)
In Wei/Levoy’s method, orientation is needed to
“flatten” the neighborhoods.
Texture Orientation
• Generate a coordinate frame
• Three orthogonal axes
– s (texture right)
– t (texture up)
– n (surface normal)
Texture Orientation
• Methods for orienting textures
– user-specified (Turk)
– random (Wei/Levoy)
– smooth or symmetric (Wei/Levoy)
Texture Orientation (User-specified)
Texture Orientation
(Symmetry)
4-way symmetric texture
4-way symmetric vector field
Texture Orientation
random
2-way symmetry 4-way symmetry
Texture Orientation
(Relaxation)
• Minimize an error function
Results (Wei/Levoy):
Random Orientation
Results (Wei/Levoy):
Other Orientations
Random
User-specified
Relaxation
2-way symmetry
4-way symmetry
Texture Synthesis on Surfaces:
Wei/Levoy Style
Synthesis
Synthesis : 2 Lowest Levels
Synthesis : Lowest Level
Random copy
Synthesis Pass1 :
Extrapolation
Copy
Shooting normal
Search
Mesh Neighborhood
Resampled Grid
Compare
3D Patch
Resample
Flatten (Maillot’93)
2D Patch
Neighborhood Flattening
• Project the triangles adjacent to p onto p’s local
texture coordinate system
• Add triangles one-at-a-time until neighborhood
template is covered
Neighborhood Comparison
compare ?
Synthesis Pass 2 :
Full Neighborhood
Copy
Search
Multiresolution Synthesis
Texture Synthesis on Surfaces:
According to Turk
Turk’s Surface Sweeping
• Select an anchor vertex A
• Assign s(v) = sweep distance to vertex v along
the vector field from A (for all v)
• Consensus orientation b/w 2 vertices v and w: Ovw
= (O(v) + O(w))/2
• Calculate a new s(v) as a weighted average that
its neighboring values dictate it should have.
• Visit vertices in order by sweep distance
Turk’s Texture Synthesis
(Pseudocode)
I = Input texture
N(v) = Neighborhood around v
M(a,b) = Neighborhood around (a,b)
D(M,N) = Match Value (sum of squared differences)
Turk’s Mesh Neighborhoods
•
•
•
•
r = average distance between mesh vertices
O(v) = Surface tangent vector
P(v) = O(v) rotated 90o about surface normal
Together, O and P make a coordinate frame !
normal
v
u
P
O
• Now, we can traverse the surface by point
repelling
Point Repelling
• Use color interpolation to determine
color at current surface point
• Move r in the direction of O or P
• When an edge is reached, fold the path
over the next polygon
Results
smooth
4-way symmetry
random
Results
Surface
displacement
smooth
2-way symmetry
2-way symmetry
Results
Summary of Differences
Turk’s approach
Our approach
Vector field
smooth
random, symmetric
Traversal order
sweeping
random
Neighborhood
surface marching
flattening/resampling
Mesh hierarchy
explicit parent/child
shooting normal