Light Propagation Volumes in
CryEngine® 3
Anton Kaplanyan
Advances in Real-Time Rendering in 3D Graphics and Games
AntonK@Crytek.de
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Agenda
 Introduction
 CryEngine® 3 lighting pipeline overview
 Core idea
 Applications (with video)
 Improvements
 Combination with other technologies (with video)
 Optimizations for consoles
 Conclusion and future work
 Live demo
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Introduction into real-time graphics
 Strictly fixed budget per frame
 Many techniques are not physically-based
 Consistent performance
 Game production is complicated
 This talk is mostly about massive and indirect lighting
 This is a high level talk
– More implementation details in the paper
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
CryEngine® 3 renderer overview (1 / 5)
 Xbox 360 / PlayStation 3 / DirectX 9.0c / 10 / (11 soon…)
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
CryEngine® 3 renderer overview (2 / 5)
 Unified shadow maps solution [Mittring07]
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
CryEngine® 3 renderer overview (3 / 5)
 SSAO [Kajalin09], [Mittring09]
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
CryEngine® 3 renderer overview (4 / 5)
 Deferred lighting [Mittring09]
– Minimal G-Buffer
– Sun / Omni / Projectors / Caustics / Deferred light probes
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
CryEngine® 3 renderer overview (5 / 5)
 Lighting accumulation pipeline:
– Apply global / local hemispherical ambient
– Optionally: Replace it with Deferred Light Probes locally
– [Global illumination solution should take place here]
– Multiply indirect term by SSAO to apply ambient occlusion
– Apply Direct Lighting on top of Indirect Lighting
Indirect lighting term
Ambient
Deferred
light probes
Global
Illumination
SSAO
Optional
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Direct
lighting
Real-time rendering development trends
 Rendering is a multi-dimensional query [Mittring09]
– R = R(View, Geometry, Material, Lighting)
 Divide-and-conquer strategy, some examples:
– Shadow maps (decouple visibility queries)
– Deferred techniques (decouple lighting / shading)
– Screen-space techniques (SSAO, SSGI, etc.)
– Reprojection techniques (partially decouples view)
 Why?
– Less interdependencies => more consistent performance
– Future trends: parallel and distributed computations friendly
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Paper reference icon
 This icon means that details are in the paper
TM
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Light Propagation Volumes
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Light Propagation Volumes: Goals
 Decouples lighting complexity from screen coverage
(resolution×overdraw)
– Radiance caching and storing technique
 Massive lighting with point light sources
 Global illumination
 Participating media rendering (still work in progress…)
 Consoles friendly (Xbox 360, PlayStation 3)
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Related work
 Irradiance Volumes [GSHG97], [Tatarchuk04], [Oat05]
 + Signed Distance Fields [Evans06]
 Lightcuts: A Scalable Approach to Illumination
[WFABDG05]
 Multiresolution Splatting for Indirect Illumination [NW09]
 Hierarchical Image-Space Radiosity for Interactive Global
Illumination [NSW09]
 Non-interleaved Deferred Shading of Interleaved Sample
Patterns [SIMP06]

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
SH Irradiance volumes
 A grid of irradiance
samples is taken
throughout the scene
 Each irradiance sample
stored in SH form
 At render time, the
volume is queried and
near-by irradiance
From [GSHG97], [Tatarchuk04]
samples are interpolated to estimate the global
illumination at a point in the scene
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Low-frequency radiance volumes
 Similar to SH Irradiance Volumes [Tatarchuk04]
 Stores radiance distribution instead
 Low resolution 3D texture on GPU (up to 323 texels)
 SH approximation is low order (up to linear band)
 Radiance is not smooth [GSHG97]
– But what is the error introduced by approximating it?
From [GSHG97]
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Radiance approximation
 Error of the spatial approximation depends on
– density and size / radii of light sources
 Error of the angular approximation depends on
– Shape of light source
– Frequency of angular radiance distribution of light source
– Distance to the light source
• Compensated by the energy fall-off
 Preserves mean energy and
major radiance flow direction
 Enough if we want to eventually get irradiance
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Light propagation in radiance volume
 Start with given initial radiance distribution from emitters
 Iterative process of radiance propagation
 6-points axial stencil for adjacent cells
– Gathering, more efficient for GPUs
– Energy conserving
 Each iteration adds to result, then propagates further

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Light propagation in radiance volume
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Rendering with Light Propagation Volume
 Regular shading, similar to SH Irradiance Volumes
– Simple 3D texture look-up using world-space position
– Integrate with normal’s cosine lobe to get irradiance
• Simple computation in the shader for 2nd order SH
– Lighting for transparent objects and participating media
 Deferred shading / lighting
– Draw volume’s shape into accumulation buffer
– Supports almost all deferred optimizations
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Massive Lighting with point light sources
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Massive lighting
 Option 1: Inject initial energy, then propagate radiance
– A bit faster for crazy amount of lights
 Option 2: Add pre-propagated radiance into each cell
– Simple analytical equation in the shader for point lights
– Higher quality, no propagation error
 Error depends on the ratio (light source radius / cell size)
– Radius threshold for lighting with radiance volume

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Glossy reflections with Light Propagation Volumes
 Accumulative traversal (ignores reflection occlusion)
 Several look-ups along reflected ray from camera
 Collect incoming radiance from this direction
– Integrate over the cone of incoming direction
– Cone angle depends on:
• Glossiness of surface
• Distance from look-up to point p
 Approximates the integration with Phong BRDF

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Glossy reflections example
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Massive lighting: Results
 NVIDIA GeForce GTX 280 GPU, Intel Core 2 Quad CPU @ 2.66 GHz,
DirectX 9.0c API, HDR rendering @ 1280x720, no MSAA, Volume size: 323
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Massive lighting video
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Global Illumination with Light
Propagation Volumes
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Global Illumination with Light Propagation Volumes
 Instant Radiosity [Keller97]
– The main idea is to represent light bouncing as a set of
secondary light sources: Virtual Point Lights (VPL)
 Splatting Indirect Illumination [DS07]
– Based on Instant Radiosity
– Reflective Shadow Maps (RSM) are used to generate
initial set of VPLs on GPU
– Importance sampling of VPLs from RSM
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Reflective Shadow Maps
 Reflective Shadow Map – efficient VPL generator
 Shadow map with MRT layout: depth, normal and color
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Global Illumination with Light Propagation Volumes
 Inject the initial radiance from VPLs into radiance volume
– Point rendering
– Place each point into appropriate cell
• Using vertex texture fetch / R2VB
– Approximate initial radiance of each VPL with SH
• Simple analytical expression in shader
 Propagate the radiance
 Render scene with propagated radiance

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Implementation details
 Light Propagation Volume moves with camera
 3D cell-size snapping for volume movement
 2D texel-size snapping for RSM movement
 RSM is higher in resolution than radiance volume
 Smart down-sampling of RSM

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Global Illumination with Light Propagation Volumes
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Issue: Cell-alignment of VPLs
 Injection of VPLs involves
position shifting
– Position of injected VLP
becomes grid-aligned
– Consequence of spatial
radiance approximation
 Unwanted radiance
bleeding
– Lighting of double-sided
and thin geometry
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Cell-alignment of VPLs: Bleeding example
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Cell-alignment of VPLs: Solution
 VPL half-cell shifting
– towards normal
– towards light direction
 Coupled by anisotropic bilateral filtering
– During final rendering pass
– Sample radiance with offset by surface normal
– Compute radiance gradient
– Compare radiance with radiance gradient

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Cascaded Light Propagation Volumes for GI
 One grid is limited in dimensions and low resolution
 Multiresolution approach for radiance volumes
– Similar to Cascaded Shadow Maps technique [SD02]
– Preserves surrounding radiance outside of the view
 Each cascade is independent
– With separate RSM for each cascade
– Transmit radiance across adjacent edges
– Filter objects by size for particular RSM
 Efficient hierarchical representation of radiance emitters
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Global Illumination Video
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Global Illumination: Combination with SSAO
 No secondary occlusion for light propagation volumes
 Can be approximated by Ambient Occlusion term
SSAO on, GI off
SSAO off, GI on
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
GI + SSAO
Global Illumination: Combination with SSGI
 Screen-Space Global Illumination [RGS09]
 Limitations of SSGI
– Only screen-space information
– Huge kernel radius for close objects
 Limitations of Light Propagation Volumes
– Local solution
– Low resolution spatial approximation
 Supplementing each other
– Custom blending

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Global Illumination: Combination with SSGI
SSGI on
SSGI off
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Optimizations for consoles: Xbox 360 / PS3
 3D texture look-up with trilinear filtering
 Radiance volume is 32 bpp for all three SH textures
 Xbox 360, ~3,5 ms per frame
– Vertex texture fetching for RSM injection
– Work-around to resolve into particular slice of 3D texture
 PlayStation 3, ~3,4 ms per frame
– Emulate signed blending in the shader
– R2VB for RSM injection (using memory remapping)
– Render to unwrapped 2D RT then remap as 3D texture

Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Future work
 Better radiance approximation…
 Participating media rendering
 Occlusion for indirect lighting
 Multiple bounces
 Improve quality
– Improved propagation scheme
– Better angular approximation
– Adaptive grids
 Support for arbitrary types of light sources
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
References









[DS07] Dachsbacher, C., Stamminger, M. 2007. Splatting Indirect Illumination
[Evans06] Evans, A. 2006. Fast Approximations for Global Illumination on Dynamic Scenes
[GSHG97] Greger, G., Shirley, P., Hubbard, P., Greenberg, D. 1997. The Irradiance Volume
[Isidoro05] Isidoro J. 2005. Filtering Cubemaps: Angular Extent Filtering and Edge Seam Fixup Methods
[Kajalin09] Kajalin, V. 2009. Screen-space ambient occlusion, Shader X7
[Keller97] Keller, A. 1997. Instant radiosity
[Mittring07] Mittring, M. 2007. Finding Next Gen – CryEngine 2
[Mittring09] Mittring, M. 2009. A bit more Deferred – CryEngine3.
[NSW09] Nichols, G., Shopf, J., Wyman, C. 2009. Hierarchical Image-Space Radiosity for Interactive Global
Illumination

[NW09] Nichols, G., Wyman, C. 2009. Multiresolution Splatting for Indirect Illumination

[Oat05] Oat, C., 2006 Irradiance Volumes for Real-Time Rendering, ShaderX 5

[RGS09] Ritschel, T., Grosch, T., Seidel, H.-P. 2009. Approximating Dynamic Global Illumination in Image Space

[SD02] Stamminger, M., Drettakis, G. 2008. Perspective shadow maps

[SIMP06] Segovia, B., Iehl, J. C., Mitanchey, R., Peroche, B. 2006. Non-interleaved Deferred Shading of
Interleaved Sample Patterns

[Tatarchuk04] Tatarchuk, N. 2004. Irradiance Volumes for Games

[WFABDG05] Walter, B., Fernandez, S., Arbree, A., Balda, K., Donkikian, M., Greenberg, D. 2005. Lightcuts: A
Scalable Approach to Illumination
 More details in the paper at
http://www.crytek.com/technology/presentations/
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Acknowledgment
 Michael Endres, Felix Dodd, Marco Siegel, Frank Meinl,
Alexandra Cicorschi, Helder Pinto, Efgeni Bischoff and
other artists and designers at Crytek for created scenes
 Martin Mittring, Vladimir Kajalin, Tiago Sousa, Ury
Zhilinsky, Mark Atkinson, Evgeny Adamenkov and the
whole Crytek R&D team
 Special thanks to Carsten Dachsbacher and Natalia
Tatarchuk
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)
Thank you for your attention!
Questions?
AntonK@Crytek.de
Advances in Real-Time Rendering in 3D Graphics and Games
New Orleans, LA (August 2009)