Geometry Videos
Symposium on Computer Animation 2003
Hector M. Briceño
Collaborators: Pedro V. Sander, Leonard McMillan, Steven Gortler,
and Hugues Hoppe
2
2
Motivation

Many sources of 3D Animation
data:
Motion Capture
Visual Hulls
Physical Simulations
Sensor Data
Skilled Animators






Wide variety of formats, data,
and reconstruction schemes…
2
2
3
3
Problem: Sharing 3D Animations



Render a Video of the animation
Use the similar software and/or hardware
Use static mesh compression for each
frame
DEMO
3
DEMO
3
4
4
Approach:

By representing manifold 3D objects using a
global 2D parametrization (mapping) it is
possible to use existing video techniques to
represent 3D animations.
4
4
5
5
Assumptions of Geometry Videos




One or more manifold surfaces
Consistent connectivity through the
duration of the animation
No changes in topology
Can undergo arbitrary deformations as
well as rigid-body transformations
5
5
6
6
Outline


Related Work
Geometry Images and Geometry Videos






Cuts
Parametrization
Compression
Exploiting Temporal Coherence
Results
Future Work and Conclusions
6
6
7
7
Related Work: Mesh Compression

Maintaining connectivity:




Topological Surgery [Taubin98]
Progressive Meshes [Hoppe96]
Spectral Compression [Karni00]
Re-parametrizing:


Semi-regular: Progressive
Compression [Khodakovsky00]
Fully regular: Geometry Images:
fully regular [Gu02]
7
7
8
8
Related Work: Animated Meshes





MPEG4, VRML Animated Meshes
“Multi-Resolution Dynamic Meshes with Arbitrary
Deformations” [Shamir00]
“Representing Animations by PCA” [Alexa00]
“Compression of Time-dependent geometry” [Lengyel99]
“Dynapack” [Ibarria03]
8
8
9
9
Related Work: Video

MPEG


Spatial, Temporal, SNR Scalability, Motion
Compensation, High Compression, VBR…
Other…


Layered Coding L-DCT [Amir96]
Multi-resolution Video [Finkelstein96]


NAIVE [Briceno99]

9
LOD both time and space.
Graceful degradation, error resilience
9
10
10
Geometry Images


Represents a manifold surface in 3D
space as an 2D array of 3D points.
Works in 3 steps:


Cutting: maps 3D surfaces to manifold
Parametrization


10
Maps 3D space -> 2D parameter space
Rasterization and Compression
10
11
11
Parametrization

Maps 3D manifold surface onto 2D square

11
Different criteria or metrics: Conformal, Areapreserving, Geometric-Stretch
11
12
12
Rasterization/Compression


Sample points of parametrization obtain a
2D grid of triplets (x,y,z)
Compress resulting “image”
DEMO
12
12
13
13
Cutting: Geometry Image

Iteratively Cut and Reparametrize
Final
13
13
14
14
Animated Meshes: Approach

How do we cut, parametrize and compress
considering a time-sequence of meshes?
14
14
15
15
Cutting: Animations

Animation frames should have the same
cut and parametrization
No Correspondence
Different Cuts and Parametrization
15
15
c
16
16
Cuts, how to pick?


Looking at single frame might miss something?
Approach: find a global cut considering all
frames.
16
16
17
17
Cut from frame 2
misses spike on
frame 1 and
spikes on frame 3
Frame 1
Frame 2
Frame 3
Global Cut

Cut 2
Global Cut
17
17
18
18
Global Cut: how it works

Frame 2
Frame 1

Run the iterative algorithm on all frames at the
same time.
Pick worst avg. face on all parametrizations…
18
18
19
19
Parametrization: Animation


Cut and parametrization has to
be fixed for all frames in order to
use one texture for whole
animation
We currently apply the global cut
to the first frame and compute
parametrization on that frame.
19
19
20
20
Compression

Spatial Compression:


Wavelets: Can support multiple levels of
detail…
Temporal Compression


20
Predictive Coding similar to MPEG
Use affine transformations for predictor
20
21
21
Encoder Architecture
Input
Frame
Cut &
Parametrize
Rasterize/
Encode
Diff
Transform
Reference
Frame


Decode
Basic Delta Encoder
Uses affine transformations
21
21
22
22
Transformations: Global

Global Trans. form a good approximation
Frame 1
22
Frame 2
Transformed Frame 1
22
23
23
Transformations: Global con’t

Global cannot capture well
deformations within the object
Frame 1
Frame 2
Predictor of Frame 2 from Frame 1
23
23
24
24
Transformations: Local

Apply transformation on charts
Frame 1
Frame 2
Predictor
24
24
25
25
Transformations: Local w/ Spread & Blend


Spread. Include neighbors in the
computation of the transformation
Blend between patches.
Target
Predictor
No blend
No spread
Predictor
w/blend
w/spread
25
25
d
c
26
26
Results



Comparing Geometry Images
Comparison to PCA
Predictive Coding: Transformations




Global
Local
Timing/Performance
Level of Detail
26
26
27
27
Comparing Geometry Images: Snake
27
27
28
28
Comparison to PCA
28
28
29
29
Transformations: Global vs. Local
29
29
30
30
Transformation Performance
2bpv P
8bpv I Baseline
4bpv P
8bpv B
8bpv P
30
DEMO
30
s
d
31
31
Performance Timings







Finding Cut (one frame): 2-7 mins
Finding Cut (100 frames): 3-5 hrs
Parametrization: 2-6 mins
Encoding: 2-3 fps @ 256x256
Encoding: 6-16 fps @ 64x64
Decoding: 10 fps @ 256x256
Decoding: 30-60 fps @ 64x64
31
31
32
32
Level of Detail
32
32
33
33
Future Work





Video Compression
Transformations
Chartification
Parametrization
Non-manifold objects
33
33
34
34
Conclusions


Geometry Video as way to encode and
represent 3D animations
Can use many of the 2D Video
Techniques/Features



Spatial/Temporal scalability
Error resiliency
Many other features to be exploited, i.e.
fast clipping and hardware
implementation…
34
34
35
35
Acknowledgements


Collaborators: Pedro Sander, Leonard
McMillan, Steven Gortler, Hughes Hoppe,
and Gu Xianfen.
Animations: Matthias Mueller and Daniel
Vlasic
Questions?
35
35