Context-based
Surface
Completion
Andrei Sharf, Marc Alexa, Daniel Cohen-Or
Introduction
Holes in models:
• Imperfect range scanned
data:
– Complex objects with non-visible
regions.
– Misalignment of multiple-views
depth image scans.
– Material reflections.
• Surface editing operations.
Motivation
Smooth filling is sufficient:
• Small holes.
• Smooth surfaces.
Smooth filling is insufficient :
• Surfaces containing fine
geometric detail.
• Topology of hole is more
complex than a disk.
Context-based Completion
Complete the missing region with patches
that conform with its context
Smooth
Context-based
Method
Import patches with matching context from
the surface itself :
• Analyze surface characteristics.
• Find best matching patch.
• Fit imported patch to boundary.
Related Work
• Example-based image completion:
[Drori et al. 2003; Jia and tang 2003; Sun et al. 2003;]
Drori et al. 03
• Texture synthesis:
[Efros and Leung 1999; Efros and Freeman 2001;Wei and
Levoy 2000; Ying et al. 2001;]
• Smooth surface completion:
Wei and Levoi 00
[Curless and Levoy 1996; Davis et al. 2002; Ilic and Fua
2003; Verdera et al. 2003; Liepa 2003;]
• Model-based surface reconstruction
Liepa 03
[Savchenko et al. 2002]
• Curve analogies [Hertzmann et al. 2002]
Hertzmann et al. 02
Moving to 3D Surfaces
Images have a regular spatial structure domain.
Problems in 3D:
• Topology and geometry of missing region.
• Fitting a patch to the boundary of the missing region.
• Definition of similarity of shapes.
• Definition of a surface patch.
Algorithm
• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
– Compute a local shape representation.
– Compute a shape signature.
• For each empty cell:
– Find matching nonempty cell ω’.
– Copy patch of ω’ into ω.
• Subdivide cells and repeat
• Until completed region matches
its neighborhood.
Algorithm
• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
– Compute a local shape representation.
– Compute a shape signature.
• For each empty cell:
– Find matching nonempty cell ω’.
– Copy patch of ω’ into ω.
• Subdivide cells and repeat
• Until completed region matches
its neighborhood.
Shape Representation
• Shape sampled point-set pi , ni
– Range scan output
– Easy to merge
• Octree hierarchy on top of the point set
• Implicit surface approximation by fitting
polynomials [Ohtake et al. 2003]:
– General quadric: Q( x)  x Ax  b x  c
– Bivariate quadratic: Q( x)  w  ( Au 2  Buv  Cv2  Du  Ev  F )
– Edge or corner fitting
t
t
Q  x  0
Shape Analysis
A local signature of a cell consists:
• Implicit shape characteristics:
– Signed distance
– Normal variation
• Detail amount:
– Depth in octree
Algorithm
• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
– Compute a local shape representation.
– Compute a shape signature.
• For each empty cell:
– Find matching nonempty cell ω’.
– Copy patch of ω’ into ω.
• Subdivide cells and repeat
• Until completed region matches
its neighborhood.
Missing Region
Automatic identification:
• Intersect shape approximation with cells.
• Empty cell that intersects surface is part
of missing region.
Empty cell definition can alter due to
refinement of missing region.


Matching
Find most similar non-empty cell:
d(,') = widc (,')+(1-wi )da (,')+wldl (,')
Distance metrics:
• dc : Signature distance inside cells.
• da : Signature distance of adjacent cells.
• dl : Difference in amount of detail.
Matching-Candidate Set
• Non-empty cells of the same size
• Symmetry rotations of cells
(/2, , ...)
• Rotation of all point-set with some
angle (/4, /3, ...)
Transferring Points
ICP:
•
•
•
•
Copy points into empty cell.
Find closest point correspondence
Align points rigidly
Align points non-rigidly using
polynomial form.
Algorithm
• Given an incomplete shape
• Create initial spatial subdivision
• For each cell
– Compute a local shape representation.
– Compute a shape signature.
• For each empty cell:
– Find matching nonempty cell ω’.
– Copy patch of ω’ into ω.
• Subdivide cells and repeat
• Until completed region matches
its neighborhood.
David’s Hair
Original
Downsampled
Smooth
completion
Contextbased
completion
Completion Process
Original
Initial
approximation
Final result
Manual Editing of Bunny Model
Manual Editing of Knot Model
Scan of “Youth” Statue (Rear View)
Original
Smooth
Result
Scan of Human Bone
Original
Smooth
Result
Limitations: Semantics
Sample Rate vs. Detail
Frequency
Summary
A fully automatic method to complete a
missing region in a surface from its context.
• Completed patches geometrically conform with
neighborhood.
• Incremental scale-space framework for finer
approximation of the unknown region.
Future Work
• Explore with other spatial hierarchies.
• Enlarge the search space of examples by building a
class based example set.
• Couple method with image completion methods for
texture completion.
• Rotation and translation invariant signature
Acknowledgments:
– David Levin and Olga Sorkine
– Israel science foundation; Israeli ministry of science
– Digital Michelangelo project 3D model repository; Stanford 3D
scanning repository; Darmstadt university of technology;
Imager computer graphics laboratory of the university of
British Columbia
Thank you