Rendering Synthetic
Objects into Real Scenes:
Bridging Traditional and Image-based
Graphics with Global Illumination and
High Dynamic Range Photography
Paul Debevec, University of California at Berkley
Motivation

Visual Effects
– Adding synthetic actors/props

Architectural simulation
– Adding buildings to landscapes
Related Work

Previous methods used to insert
synthetic objects into real scenes
– Drawing
– Environment mapping
– Approximate geometric model and
lighting


Global illumination algorithms
Recovery of HDR radiance maps from
photographs
Related Work


High Dynamic Range refers to the range of
brightness values present in a scene
Eyes, photos, displays, all capture a limited,
non-linear range
1000:1
100:1
Everyday brightness range
Source: Microsoft Developers Network
(about 1012:1)
Related Work
 Why is this a problem?
 Limited brightness range
results in lost scene
detail from saturation
and/or underexposure
 OK for traditional image
use
Photos from OpenEXR by LucasFilm, LTD. http://www.openexr.com/samples.html
Related Work

What about image-based lighting?
– Photo pixel values are not representative
of relative Radiance in the scene
Images by Paul Debevec, 98
Related Work

Solution: Take multiple photos with varying
shutter speeds and ‘combine’ somehow to
create accurate relative radiance maps
Images by Paul Debevec, 98
Related Work

Is HDR really important?
Method

Divide the scene into three parts
Method

Distant Scene
– Light-based model
– Required to provide correct incident
illumination to local/synthetic objects
from desired viewpoint
– Assumed that no light from the model will
affect the distant scene
– Any level of geometric detail wanted
Method

Local Scene
– Material-based model (has BRDF)
– Must approximate the real scene local to
the position of the synthetic objects
– ‘Local’ means photometrically local
– ‘Approximate’ means geometrically AND
photometrically representative of the real
scene
Method

Synthetic Objects
– Complete materials-based models of
elements that are to be inserted into the
scene
– Geometric and lighting information
– Any shape and material supported by the
global illumination method that you plan
to use
Method

Compositing using
a light probe
– One method to
create the distant
scene from a real
location
– Uses a metal sphere
and a camera to
gather light from a
point
Method

Compositing using
a light probe
– This light is mapped
onto the distant
scene geometry and
used to calculate
the lighting
– Lighting is
calculated with
global illumination
Method

Results
Method

Problem with the current definition of
the local scene
– We said that it must approximate the
material of the real scene
– It can be really hard to get the
BRDF/material of a real object!
– Can we modify the local scene so it
doesn’t require this?
Method

Local Scene 2 (Differential Rendering)
– Approximate the material of the scene to
within reason (not exact)
– Render with and without your synthetic
objects and calculate the difference
– Add that difference back into the original
image
Method

Difference between local scene render with and
without synthetic objects
=
Results
Conclusion

This paper presented a framework for
rendering synthetic objects into real
world environments with real lighting
– High Dynamic Range radiance maps
– Global Illumination
– Distant and local scenes, synthetic
objects
– Differential rendering
Future Work


Method to generate boundary between
distant and local scene
More efficient global illumination
Comments

Pros
– Open framework
– Good results

Cons
– Hard to decide what local scene should
be!
– Local scene can be very complicated