Image Synthesis
Point-Based Computer Graphics
computer graphics & visualization
Why Points?
• huge geometry complexity of current CG
models
• overhead introduced by connectivity of
polygonal meshes
• acquisition devices generate point samples
“digital 3D photography”
• points complement triangles
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Polynomials
• Rigorous mathematical concept
• Robust evaluation of geometric entities
• Shape control for smooth shapes
• Require proper parameterization
• Discontinuity modeling
• Topological flexibility
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Polynomilas  Triangles
• Piecewise linear approximations
• Irregular sampling of the surface
• No parameterization needed (geometry only)
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Triangles
•
•
•
•
Simple and efficient representation
Hardware pipelines support triangles
Advanced geometric processing
The widely accepted queen of graphics primitives
•
•
•
•
Sophisticated modeling is difficult
(Local) parameterizations still needed
Complex LOD management
Compression and streaming is highly non-trivial
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Triangles  Points
•
•
•
•
Piecewise linear functions to Delta distributions
Discrete samples of geometry
No connectivity or topology – most simple
Store all attributes per surface sample
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Points
•
•
•
•
geometry complexity of current CG models
connectivity overhead of polygonal meshes
acquisition devices generate point samples
points complement triangles
• holes
• compression
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Taxonomy
Acquisition
Rendering
Point Based Graphics
Representation
Processing & Editing
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
How can we capture reality?
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Acquisition
• Contact digitizers
– intensive manual labor
• Passive methods
– require texture, Lambertian BRDF
• Active light imaging systems
– restrict types of materials
• in general fuzzy, transparent,
and refractive objects are difficult
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
First Method, Laser Range Scanner
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Basic Idea
Detector
Detector
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Computing the Distance
Detector
a’
d’
H
a
Laser
L
d
O
a a'
d'
  d  a
d d'
a'
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Scattering Issues
How optically cooperative is marble?
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Image based Acquisition
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Image based Acquisition
Acquisition Stage 2
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Image based Acquisition
Acquisition Stage 3
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
IBA - Process
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Visual Hull
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Visual Hull
• the quality of the visual
hull geometry is a
function of the number of
viewpoints / silhouettes
• the method is unable to
capture all concavities
 image based lighting
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Point Based Rendering
Surfels (surface element)
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Extended Surfels
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Rendering Pipeline
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Uniform Reconstruction
• For uniform samples, use signal processing theory
• Reconstruction by convolution with low-pass
(reconstruction) filter
• Exact reconstruction of band-limited signals
using ideal low-pass filters
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Non-Uniform Reconstruction
• Signal processing theory not applicable for
nonuniform samples
• Local weighted average filtering
– Normalized sum of local reconstruction kernels
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Reconstruction 1D in 2D
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Reconstruction 2D in 3D
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Algorithm
for each sample point {
shade surface sample;
splat = projected reconstruction kernel;
rasterize and accumulate splat;
}
for each output pixel {
normalize;
}
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Results
without Normalization
with Normalization
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Visibility
ε-z-buffering
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Implementation
Use a three pass algorithm:
1. Draw depth image with a small depth offset ε
away from the viewpoint
Perform regular z-buffering (depth tests and
updates), no color updates
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Second Pass
• Draw colored splats with additive blending
enabled
• Perform depth tests, but no updates
• Accumulate
– Weighted colors of visible splats in the color
channels
– Weights of visible footprint functions in the alpha
channel
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Third Pass
• Normalization of the color channels by dividing
through the alpha channel
• Implemented by
– render to texture
– drawing a screen filling quad with this texture
– performing the normalization in the pixel shader
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Efficient Data Structures
DuoDecim
A Structure for Point Scan
Compression and Rendering
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
a.d. 1500
•
created beautiful statues
… but missed to make them portable
•
•
•
•
David: 434 cm
Atlas: 208 cm
Barbuto: 248 cm
…
Florence, Galleria dell'Accademia
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
a.d. 1999
• Marc Levoy et al. (Stanford University) did a
great job scanning the statues
… but still “missed” to make them “portable”
• David:
• Atlas:
• …
1.1 GB
10 GB
All in all 32 gigabytes raw data !!!
– WS
07/08 and Visualization Group, TUJensImage
KrügerSynthesis
- Computer
Graphics
Dr. Jens Krüger – Computer Graphics and Visualization Group
München
Image courtesy of Marc Levoy
computer graphics & visualization
Today…
… we present novel algorithms to make the statues of
Michelangelo „portable“ and „viewable“ on PCs.
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
The Numbers
The Atlas Statue Scan
Size of raw scans:
Reconstruction:
Size of reconstruction:
Filesize (without normals):
approx 600 mio. points
at 1/4 mm
approx 250 mio. vertices,
approx 500 mio. triangles
9.94 GB
„Your Home PC“
Size of Main Memory:
1 – 2 GB
Size of Graphics Card Memory: 0.125 – 0.5 GB
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
230 MB vs. 10 GB
Rendering and decoding time in full resolution 4 sec.
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
The Key Idea
CPU
• Sample the point scan into a regular grid
• Divide the grid up into 2D slices
• Look for connected runs within the slices
• Store the starting position/normal per run
• Store position/normal delta for the rest
GPU
• Store the compressed runs into textures
• Upload the compressed runs onto the GPU
• Decode the points with normals on the fly
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
1. Sample the point scan into a regular grid
Hexagonal close sphere packing (HCP) grid
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Cell Search (HCP)
2D Simplification
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
2. Divide the grid up into 2D slices
1
2
sqrt(3)
sqrt(3)
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
3. Look for connected runs within the slices
h
e
a
b
c
f
g
d
i
k
j
l
m
p
n
o
q
abc defghgijigfedklmnopoq
dklmn opoq
op
oq
def ghgij
gh
gij
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
3. Connected runs (cont.)
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
4. Store the starting position/normal per run
• Start Position  two (16bit) indices per point
– one (16bit) index per slice
• Start Normal  one (16bit) index
– one (16bit) Codebook per dataset
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
5. Store position/normal delta for the rest
• Delta Position  6 cases  2.25 bit
• Delta Normal  5bit index
– one (5bit) Codebook per dataset
• contains sin/cos of delta angles
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
Rendering
GPU Decoding and Rendering
• Store the compressed runs into textures
• Upload the compressed runs onto the GPU
• Decode the points with normals on the fly
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
1. Store the compressed runs into textures
Start Position/Normal
16bit (RG)(B)
Delta Position/Normal
8bit (3+5)
Per Slice:
•16bit height
•8 Floats to the GPU
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
3. Decode the points with normals on the fly
Start Position/Normal
Position
Normal
n
n
m
m
Render as Points via:
PBO/VBO Copy
Cast to Vertex Array
Vertex Texture Fetch
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
3. Decode the points with normals on the fly
Start Position/Normal
Position
Normal
n
n
m
m
Delta Position/Normal 1
Position
Pixelshader
n
m
Render as Points
Normal
n
m
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization
One Image about quality…
Image Synthesis – WS 07/08
Dr. Jens Krüger – Computer Graphics and Visualization Group
computer graphics & visualization