Real Time Ray Tracing:
Can ray tracing be a viable option for rendering real time interactive
applications such as video games?
Callum Brighting
Bsc (Hons) Games Design and Development
2011
Student ID: 14823, Course Code: GD1110, Submitted 24th October 2011, Word count: XXXX
i
Declaration
I hereby declare that I wrote this written assignment / essay / dissertation on my own
and without the use of any other than the cited sources and tools and all
explanations that I copied directly or in their sense are marked as such, as well as
that the dissertation has not yet been handed in neither in this nor in equal form at
any other official commission.
……………………………………………….
24th October 2011, West Sussex
© 2010 SAE Education Ltd. | Subject to change without notice
ii
Abstract
Currently there are very few fundamentally different algorithms that are used for rendering 3D
environments. The one currently most used in games is a projection based algorithm: 3D data is
projected onto the screens plane via a vertex shader kernel, and then a fragment shader kernel is
run for every pixel that lies in-between the transformed points.
The other type of algorithm that has survived the test of time (SUFFERN, Kevin, 2007)is an imagespace algorithm. Here, each pixel is modelled separately and the final colour is based on finding out
where the light came from for this particular pixel. This method relies heavily on 3D geometric maths
to work out the path of a “ray of light” through a 3D scene. Working backwards from the final image
involves path tracing along the ray, the name given to this method is: Ray tracing.
Moore’s Law predicts that there will be an approximate doubling in transistor counts every eighteen
to twenty-four months still holds true (GREGORY, Jason, 2009). However, in 2004 this assumed
relationship between transistor count and CPU speed was shown to be no longer valid due to
thermal constraints in chip design. Chip manufacturers have since shifted their focus to multicore
CPU’s.
The advent of multicore processors and therefore the parallelization of software has had a huge
impact on the games industry; “By 2008, most game studios had completed the transformation for
the most part (to multiprocessor architectures (GREGORY, Jason, 2009). Even mobile devices are
going multicore: Sony Playstations latest handheld the “Playstation Vita” has four cores for its CPU
and four cores for its GPU (SONY, 2011).
All these extra cores mean that a massively parallel technique like ray tracing is becoming an
increasingly attractive option to games developers. John Carmack, technical director at ID Software
announced via a social networking site that he himself had been looking into Ray tracing, his
response when I asked about performance was as follows:
There has been an entire industry built up around the current methods of rending 3D scenes via
vertex and fragment shaders. People have trained hard to become technical artists and adapt
algorithms to work in a parallel way. Changing over to a much less flexible method such as ray
tracing would indeed have its many production values, not to mention skill and knowledge problems
too.
My results show that ray tracing scales well with an increasing number of cores, allowing for a taskbased architecture maximises the amount of work per thread and minimises the amount of
performance overhead per thread, but ray tracing for now is still too slow to be considered as a real
replacement for current generation consoles and hardware.
iii
Foreword



Thank Jason as project supervisor
Thank everyone at Qantm
Thank MyCreateFuture for 1st year scholarship, and SAE for 2nd year scholarship
iv
Table of Contents
Declaration .............................................................................................................................................. ii
Abstract .................................................................................................................................................. iii
Foreword................................................................................................................................................ iv
Introduction ............................................................................................................................................ 1
Objectives ........................................................................................................................................... 5
Research Question .............................................................................................................................. 5
Overview ............................................................................................................................................. 5
Ray Tracing History ......................................................................................................................... 5
The Algorithm ................................................................................................................................. 6
Use in other industries .................................................................................................................... 6
Previous Work ..................................................................................................................................... 6
Literature Summary ................................................................................................................................ 6
Normal Sources ................................................................................................................................... 6
Other Sources ..................................................................................................................................... 7
Methodology........................................................................................................................................... 7
Water fall development ...................................................................................................................... 7
Dynamic Sprint Development ............................................................................................................. 7
Sprints ................................................................................................................................................. 8
Initial prototype .............................................................................................................................. 8
Core classes ..................................................................................................................................... 8
Test / Benchmark builds ................................................................................................................. 8
Spatial Data Structures ................................................................................................................... 8
Scenes ............................................................................................................................................. 8
Shadows .......................................................................................................................................... 8
Reflections....................................................................................................................................... 8
Transparency................................................................................................................................... 8
Meshes ............................................................................................................................................ 8
Threading setup .............................................................................................................................. 8
8 Thread version ............................................................................................................................. 8
Task based Threading...................................................................................................................... 8
Observations ................................................................................................................................... 8
Results ..................................................................................................................................................... 9
Findings ................................................................................................................................................... 9
v
Recommendations .................................................................................................................................. 9
Discussion.............................................................................................................................................. 10
Works Cited ........................................................................................................................................... 10
Bibliography .......................................................................................................................................... 11
Appendices............................................................................................................................................ 11
vi
Notes from Jen:
Intro: what is ray tracing
Why is it slow
Aim: Build a multithreaded ray traced game
Methodology:



Do this,
Do that
Then, do this
Report:
Sprint breakdowns, 3-4 major sprints
‘I’ remove !!
Except introduction which is first tense
PAST TENSE
Methodology is very bullet pointed.
Introduce current issues: in the intro. State why it is so slow,
Aim is to increase speed of ray tracing
Literature review: how is it done by others, people doing something similar?
3rd chapter: methodology recipe: I’m going to try and use agile scrum,
How did I asses the quality of a sprint:
Asses as often as possible
4th: Report: Detail the sprints, 3 – 4 main sprint sections. Sprint cycle visuals
5th: conclusion:
Initial aims: revisit intro – this is how they were or were not met
Reflection: what worked well, look back at proposal
Conclusion:
1
Binding / Printing:
Introduction
Graphics are of extreme importance to games. Many debates as to why graphics get precedence
over arguably more important areas of a game such as AI and Audio take place online in forums.
Regardless of the reason, graphic development is a huge focus on any game project; John Carmack
technical director of ID Software said at quakecon 2011: “20% of ID Tech 5 is related entirely to the
Mega Texture graphics method.” (CARMACK, John, 2011). The ID Tech5 engine that he speaks about
is reportedly several million lines of code. The fact that 1 fifth of the engine is entirely devoted to
one small part of the rendering method is proof that graphics are important in games.
However much effort is placed into graphics today, there are still some fundamental issues with the
chosen methods: aliasing, unreliable frame rates and colour washing are all major issues that have
plagued graphics programmers for generations. All of these issues stem from the projection based
algorithms that have become so standard in games today.
Films on the other hand are renowned for their high quality visuals; life like video effects and
photorealistic animations have allowed things like dead actors coming to life in Tron Evolution.
Animation studios such as Pixar have spent millions of dollars and years of investment in developing
their technology. Pixar’s in-house software is about as industry standard as it gets; with dozens of
blockbuster hits using its technology, and numerous awards and constant SIGGRAPH papers being
written, Renderman is a big name in movies (PIXAR ENTERTAINMENT, 2011). A big component of
Renderman is its ray tracing capabilities; it allows for advanced global lighting features such as
ambient occlusion, colour bleeding, reflective and refractive surfaces along with a whole host of
other advanced graphical techniques. It also allows a programmable pipeline allowing engineers to
customise the ray tracing engine, even distributing it among multiple servers. Recently in 2009, one
of Pixar’s employees was awarded ‘The Coons Award’ for “contributions to physically-based
reflectance models and distribution ray tracing” (PIXAR ENTERTAINMENT, 2011), showing that
Renderman is being kept at the cutting edge of graphical advancements.
Ray Tracing works as follows;
2
For each pixel on the screen or image, work out the direction that the photon of light would be
coming ‘in’ from. Reverse this direction, and trace backwards along the path in a straight line or ray.
Check for collisions with geometry, and reverse engineer the path of the light until you either hit a
light source or run out of another resource (processing time, recursive counters etc). The image
below summarises this process well:
(Image source: (ATWOOD, Jeff, 2008) )
Ray Tracing was at the forefront of computer science in 1980 (SUFFERN, Kevin, 2007) because of its
naturally high quality and its ability to incorporate realistic effects (shadows, reflection and
refraction) easily. Some effects that are difficult to reproduce with projective algorithms come as a
bye product of using ray tracing as the rendering method (SUFFERN, Kevin, 2007). Ray tracings
realism and quality are why films adopted the technique, but the performance cost has kept the
games industry at a distance. Rendering times for films are a big area of research; many engineers
spend hours trying to optimise the ray tracing process or distribute it among multiple computers to
reduce the render times. Toy Story 3 (done using Renderman) was given a technical breakdown for
Wired magazine and the developers discussed frame render times; “a complex frame like this one
required eleven hours.” (LEHRER, Jonah, 2010).
Video games operate under different conditions to films; everything is calculated in real time,
usually at a fixed frame rate of 30 or 60 a second (GREGORY, Jason, 2009)(this means each frame is
rendered in 0.0333 seconds or 0.0167 seconds respectively). Thus, ray tracing is not the tool of
choice when it comes to rendering the 3D data used by modern day games. Currently, a projective
algorithm is used that projects the geometry of the game onto a 2D plane (called the view plane)
and then the transformed data is used to work out which pixels correspond to the current piece of
geometry (rasterisation).

Insert image for rasterisation *
The current method of rasterising triangles used in games does not take advantage of the increasing
focus on parallel computation that is sweeping the games industry. Dedicated pieces of hardware
called GPU’s (graphics processing units) had to be created in order to keep up with the demand for
increasing numbers of polygons, and an arms race ensued between the vendors of these devices to
force more and more cores into smaller and smaller dies.
This move to parallel architectures has captured the imagination of the games industry already;
NVidea a major player in the GPU world has released its own real-time ray tracing engine called
‘OptiX’. According to their site it “increases interactive realism by greatly increasing ray tracing
3
speeds on NVIDIA® GPUs using the NVIDIA® CUDA™ GPU computing architecture” (NVIDEA, 2011).
The results are stunning and rendered at an interactive rate (approx 10 fps on my machine).
(Image source: (NVIDEA, 2011) ‘Julia Set’ )
Currently, we are starting to see that the electronics industry is circling around on itself; Intel have
just released a new chip called ‘Sandy Bridge’ featuring several leaps in thermal engineering along
with an important games related feature: “All cores plus graphics are integrated on a single chip,
enabling new levels of visual experiences on a PC.” (INTEL, 2011). As CPU’s appear to be catching up
on the requirements previously met by GPU’s, we are seeing that eventually one chip will rule them
all.
This means that a parallel technique such as ray tracing is back on trend and finding itself completely
relevant. Without dedicated hardware to battle with, software engineers are going to find
themselves with great amounts of control over the rendering method. Previously, a programmer
would simply invoke some calls to the operating system which would intern call some drivers to
render some polygons, and each individual card would handle the function call in its own way.
*Insert image explainging the graphics pipline: GPU gems has a good one *
Now, programmers are going to be able to implement that function anyway they like in their own
engine, calculated entirely on the CPU and using main system memory.
*Insert image explaining ray trace pipeline *
4
Objectives
My objectives for this project are three fold:
Firstly, I am to build a basic ray tracing application in c++ using the Simple Direct Media Layer (SDL)
library of code for window management and OpenGL (OGL) graphics API for drawing my scene to the
screen.
Secondly, I will take the basic application and make it multithreaded to take advantage of multicore
processors. This will take the largest portion of time and the most sprints.
Thirdly, I will take the multithreaded application and parameterise it, allowing the user to
dynamically change things like the number of threads, the amount of recursion etc.
Research Question
“Can ray tracing be a viable option for rendering real time interactive applications such as
video games? “
Overview





I will first explain ray tracing; where it came from and how it is done. The theory behind light
and materials and their interaction.
I will then talk about how it has been used to date: In films, medicine and other industries.
I will then examine my sources
I will then explain how I chose a development methodology, and then go into detail as to
how I implemented that methodology and developed my application.
I will explain my results and findings, and discuss them, before summarising and concluding
Ray Tracing History
 Mention the 1980’s Amiga ray tracing demo
5
The Algorithm
 Show diagram of how I plan to cast one ray per pixel from the eye position
 Talk about collision detection between planes, show old ray trace screen shot
 Talk about spheres and triangles, barycentric co-ordinate spaces
Ray Tracing Uses
Use in Films
 Transformers, Cars 1 & 2, Topy Story (find first ray traced film? )
 Talk about Mental Ray plug in for Maya
Use in Medicine
 Talk about ray tracing through 3D data structures / voxel grids generated by MRI scanners
Use in other industries
 Talk about Quake Ray Traced project, 2,3 and now 4.
 Talk about NVidea Optix engine
 Talk about OpenRT development platform
Previous Work

Mention HLSL project : Show screen shots
Literature Summary
Normal Sources








Ray Tracing from the group up: mention his code lay out, mention his focus on accuracy
rather than speed, and readability rather than performance
Game Code complete: mention his focus on good scalable design and modular code
55 ways to improve your programming mention his focus on writing code that is easy to use
correctly, but hard to use incorrectly, mention defensive programming
Mention Intels article online about building a ray tracing framework for real time
applications, their focus on performance is key
Mention the article online about ray tracing on the PS3 using the CELL SPU’s. He talks about
the cross compatabilioty issues and the threading overhead problem
Mention the Faster Isosurface Ray Tracing using Implicit KD-Trees article which talks a lot
about efficiently populating a data structure and traversing it with WOOs algrothm
Mention the Saarland university paper@: Realtime Ray Tracing for Current and Future
Games article about how to implement ray tracing into a gameing application
Mention uni of Edinburgh school of informatics article about difference camera modes and
perspective calculations
6









Mention http://www.codermind.com ‘s 5 part article on how to build a ray tracer: with
game like effects aswell like depth of field and refraction
Mention the Gamasutra sponsored article Cloud based ray tracing, talking about how to
build a ray tracer that scales with N number of threads working on it: allowing for many PCs
to help ray trace one frame
Mention devmaster.nets article on phong lighting and how to add shadows to a ray tracer
Mention Andrew Woo’s article on fast voxel traversal: AMAZING algorithm, really quick and
optmisable.
Mention Vlastimil Havran’s dissertation thesis for the Faculty of Electrical Engineering,
Czech Technical University, Prague, in partial fulfillment of the requirements for the degree
of Doctor: talks about LOADS of different ways of ray tracing: step through, recursive etc
Mention Realistic Ray Tracing book, for its focus on accuracy
Mention Visibility Using Ray Tracing Algorithms and Data Structures: Investigating a few Ray
Tracing Algorithms and Data Structures in the Context of Visibility: GREAT book talking about
taking advantage of the data striucture your scene is in: Build the scene first and then design
the ray tracer around it
http://www2.gcc.edu/dept/comp/faculty/gribblecp/research/papers/gribble08ray.pdf
mention that for its focus on SIMPLE code layout and extendable engine. Explains stuff really
well with diagrams too
Physics based rendering from theory to implementation: Good book about lighting
Other Sources




3D math primer for graphics and games : Maths!
Physics: Light and materials, photons and energy levels
Real time collision detection: AABB’s and ray – triangle ray – shere tests.
The C++ programming language
Methodology
Water fall development





Talk about pros and cons of waterfall
Show diagrams
Talk about inexperience in planning
Talk about iterative development: Having SOMETHING to hand in at the end
Talk about email from CODEMASTERS dude regards to their technique (hybrid of both)
Dynamic Sprint Development




Talk about pros and cons
Talk about sprints allowing for many milestones
Talk about contingency plans
Talk about documentation
7


Talk about evidence of sprints being left in the code
Talk about Code being hacked in for a milestone only to be removed
Sprints
For Each Sprint title Below:
Describe the demo at the end of the sprint: Show a screen shot
Table of risks and mitigations
Table of goals
Table of issues found tracked or solved
Describe any problems encountered, and solutions
Describe the contingency plan for this sprint
Describe the final outcome of this sprint
Initial prototype
Core classes
Test / Benchmark builds
Spatial Data Structures
Scenes
Shadows
Reflections
Transparency
Meshes
Threading setup
8 Thread version
Task based Threading
Observations
Talk about:






Good code layout
Good bug tracking
Good documentation
Poor time planning
Annoying code left in engine
Late implementation of useful debugging method
8

Cross platform compatibility
Results
Talk about:







Performance in frames per second of lots of different scenes
Lots of different meshes
Lots of different light environments
For each of the above, run it on one, 2, 4 and 8 threads
Graphs graphics and more graphics
Compare against the ideal solution
Talk about where the performance bottle neck is in each case : Visual studio ‘hot path’
Findings
Talk about:




No linear scale up with cores
Good performance of the Grid
Not usable for games yet
Describe problems doing a post process
Recommendations
The following are things to add in:
*I have a tonne of links talking about SIMD optimisations including an INTEL article on how to
raytrace 4 rays at the same time using SIMD extension library
A Visibility Field for Ray Tracing : http://cs.swan.ac.uk/~csravi/p/rowTracingUsingHOMs.pdf
-
Optimises the ray tracing algrothm by reducing and quickening the time spent in the data
structure of the scene
Toward A Multicore Architecture for Real-time Ray-tracin
ghttp://pl887.pairlitesite.com/papers/micro08-copernicus.pdf
-
Talks about cache coherency and other optmisiations
Interactive Global Illumination using Fast Ray Tracing http://www.uni-kl.de/AG-Heinrich/IGI.pdf
Talks about adding global illumination
http://graphics.stanford.edu/papers/egSTAR03/star03.pdf does the same as the above]
9
http://madebyevan.com/webgl-path-tracing/ WEBGL ray tracing!
Discussion
Not sure.
Works Cited
ATWOOD, Jeff. 2008. real-time-raytracing. [online]. [Accessed 15 August 2011]. Available from
World Wide Web: <http://www.codinghorror.com/blog/2008/03/real-time-raytracing.html>
CARMACK, John. 2011. QuakeCon 2011 - John Carmack Keynote. [online].
GREGORY, Jason. 2009. Game Engine Architecture. Massachusetts: A K Peters, LTD.
INTEL. 2011. Sandy Bridge Specifications. [online]. [Accessed 10 Aug 2011]. Available from World
Wide Web: <http://www.intel.com/content/www/us/en/architecture-andtechnology/microarchitecture/intel-microarchitecture-codename-sandy-bridge.html>
LEHRER, Jonah. 2010. Animating a Blockbuster: How Pixar Built Toy Story 3. [online]. [Accessed 14
August 2011]. Available from World Wide Web:
<http://www.wired.com/magazine/2010/05/process_pixar/all/1>
NVIDEA. 2011. Optix Engine Examples. [online]. [Accessed 14 August 2011]. Available from World
Wide Web: <http://developer.nvidia.com/optix-interactive-examples>
NVIDEA. 2011. Optix Engine. [online]. [Accessed 14 August 2011]. Available from World Wide Web:
<http://www.nvidia.com/object/optix.html>
PIXAR ENTERTAINMENT. 2011. What is Render Man? [online]. [Accessed 14 August 2011]. Available
from World Wide Web: <https://renderman.pixar.com/products/whats_renderman/movies.html>
SONY. 2011. specifications. [online]. [Accessed 10 August 2011]. Available from World Wide Web:
<http://uk.playstation.com/psvita/#select-tab-specifications>
SUFFERN, Kevin. 2007. Ray Tracing from the Group Up. Massachusetts: A H Peters, LTD.
10
Bibliography
Appendices
11