cs123
INTRODUCTION TO COMPUTER GRAPHICS
Polygon Rendering Techniques
Swapping the Loops
Andries van Dam©
11/12/2015
1 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Review: Ray Tracing Pipeline



Ray tracer produces visible samples of scene
Samples are convolved with a filter to form pixel image
Ray tracing pipeline:
-Traverse scene graph
Acceleration Data Structure (ADS)
Preprocessing
Ray Tracing
Post processing
Andries van Dam©
Scene Graph
Generate ray
for each sample
All samples
11/12/2015
-Accumulate CTM
-Spatially organize objects
-traverse ADS
-for nearest cell to farthest cell
-intersect objects in cell with ray
-return nearest intersection in cell
-otherwise continue to next cell
Convolve with filter
suitable for ray tracing
Evaluate lighting
equation at sample
Generate reflection rays
Final image pixels
2 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Rendering Polygons



Often need to render triangle meshes
Ray tracing works well for implicitly defined surfaces
Many existing models and modeling apps are based on polygon meshes – can
we render them by ray tracing the polygons?



Easy to do: ray-polygon intersection is a simple calculation (use barycentric coords)
Very inefficient: common for an object to have thousands of triangles, for a scene to
have hundreds of thousands or even millions of triangles – each needs to be
considered in intersection tests
Traditional hardware pipeline is more efficient for many triangles:



Process the scene polygon by polygon, using “z-buffer” to determine visibility
Local illumination model
Use crude interpolation shading approximation to compute the color and/or uv of
most pixels, fine for small triangles
Andries van Dam©
11/12/2015
3 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Traditional Fixed Function Pipeline (disappearing)


Polygons (usually triangles) approximate the desired geometry
One polygon at a time:


Each vertex is transformed into screen space, illumination model evaluated at each
vertex. No global illumination (except through ambient term hack)
One pixel of the polygon at a time (pipeline is highly simplified):

depth is compared against z-buffer; if the pixel is visible, color value is computed using either
linear interpolation between vertex color values (Gouraud shading) or a local lighting model
for each pixel (e.g., Phong shading using interpolated normals and various types of hardware
assisted maps)
Geometry Processing
Rendering/Pixel Processing
World Coordinates
Selective traversal
of Acc Data Str (or
traverse scene
graph) to get CTM)
Screen Coordinates
Transform
vertices to
canonical
view
volume
Andries van Dam©
Trivial
accept/reject
and backface culling
11/12/2015
Lighting:
calculate light
intensity at
vertices (lighting
model of choice)
View
volume
clipping
Imageprecision VSD:
compare pixel
depth (z-buffer)
Shading:
interpolate vertex
color values
(Gouraud) or
normals (Phong)
4 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Visible Surface Determination - review

Back face culling:



View volume clipping:



Often don’t need to render triangles “facing away” from camera
Skip triangle if screen space vertex positions aren’t in counter clockwise order
If triangle lies entirely outside of view volume, no need to render it
Some special cases requiring polygon clipping: large triangles, triangles that
intersect faces of the view volume
Occlusion culling:

Triangles that are behind other opaque triangles don’t need to be rendered
Andries van Dam©
11/12/2015
5 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Programmable Shader-Based Pipeline

Allows programmer to fine tune and
optimize various stages of pipeline

This is what we have used in labs and
projects this semester

Shaders are fast! (They use multiple ultrafast ALU’s, or arithmetic logic units).
shaders can do all the techniques
mentioned in this lecture in real time

For example, physical phenomena such as
shadows and light refraction can be
emulated using shaders
Andries van Dam©
11/12/2015
The image on the right is rendered with a custom shader.
Note that only the right image has realistic shadows. A
normal map is also used to add detail to the model.
6 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Classical Shading Models Review (1/6)

Constant Shading:


No interpolation, pick a single representative intensity and propagate it over
entire object
Loses almost all depth cues
Pixar “Shutterbug” images from:
www.siggraph.org/education/materials/HyperGraph
/scanline/shade_models/shading.htm
Andries van Dam©
11/12/2015
7 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Classical Shading Models Review (2/6)

Faceted Shading:



Single illumination value per polygon
With many polygons approximating a
curved surface, this creates an undesirable
faceted look.
Facets exaggerated by “Mach banding”
effect
Andries van Dam©
11/12/2015
8 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Classical Shading Models Review (3/6)

Gouraud Shading:





Linear Interpolation of intensity across triangles
to eliminate edge discontinuity
Eliminates intensity discontinuities at polygon
edges; still have gradient discontinuities
Mach banding is largely ameliorated, not
eliminated
Must differentiate desired creases from
tessellation artifacts (edges of cube vs. edges on
tessellated sphere)
Can miss specular highlights between vertices
because it interpolates vertex colors instead of
calculating intensity directly at each point
Andries van Dam©
11/12/2015
9 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Classical Shading Models Review (4/6)

Gouraud shading can miss specular highlights
because it interpolates vertex colors instead of
calculating intensity directly at each point, or even
interpolating vertex normals (Phong shading)
𝑁𝑎 and 𝑁𝑏 would cause no appreciable specular
component, whereas 𝑁𝑐 would, with view ray
aligned with reflection ray. Interpolating between
Ia and Ib misses the highlight that evaluating I at c
using 𝑁𝑐 would catch

Phong shading:



Interpolated normal comes close to actual normal of
true curved surface at a given point
Reduces temporal “jumping” effect of highlight, e.g.,
when rotating sphere during animation (example on
next slide)
Andries van Dam©
11/12/2015
10 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Classical Shading Models Review (5/6)

Phong Shading:





Linear interpolation of normal vectors instead
of color values
Always captures specular highlight
More computationally expensive
At each pixel, interpolated normal is renormalized and used to compute lighting model
Gouraud
Phong
Gouraud
Phong
Looks much better than Gouraud, but still no
global effects
http://www.cgchannel.com/2010/11/cg-science-for-artistspart-2-the-real-time-rendering-pipeline/
Andries van Dam©
11/12/2015
11 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Classical Shading Models Review (6/6)

Global illumination models:


Objects enhanced using texture, shadow, bump, reflection, etc., mapping
These are still hacks, but more realistic ones
Andries van Dam©
11/12/2015
12 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Maps Overview



Originally designed for offline rendering to improve quality
Now used in shader programming to provide real time quality improvements
Maps can contain many different kinds of information




Simplest kind contains color information, i.e., texture map
Can also contain depth, normal vectors, specularity, etc. (examples coming…)
Can use multiple maps in combination (e.g., use texture map to determine object
color, then normal map to perform lighting calculation, etc.)
Indexing into maps – what happens between
pixels in the original image?



Depends! OpenGL allows you to specify
filtering method
GL_NEAREST: returns nearest pixel
GL_LINEAR: triangle filtering
Andries van Dam©
11/12/2015
13 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Mipmapping

Choosing a texture map resolution



Want high resolution textures for nearby
objects
But high resolution textures are inefficient
for distant objects
Simple idea: Mipmapping




MIP: multum in parvo, “much in little”
Maintain multiple texture maps at different
resolutions
Use lower resolution textures for objects
further away
Example of “level of detail” (LOD)
management common in CG
Andries van Dam©
11/12/2015
Mipmap for a brick texture
http://flylib.com/books/en/1.541.1.66/1/
14 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Shadows (1/5) – Simplest Hack

Render each object twice




First pass: render normally
Second pass: use transformations to project object
onto ground plane, render completely black
Pros: Easy, can be convincing. Eye more sensitive
to presence of shadow than shadow’s exact shape
Cons: Becomes complex computational geometry
problem in anything but simplest case


Easy: projecting onto flat, infinite ground plane
How to implement for projection on stairs? Rolling
hills?
Andries van Dam©
11/12/2015
http://web.cs.wpi.edu/~matt/courses/cs563/t
alks/shadow/shadow.html
15 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Shadows (2/5) – More Advanced


For each light 𝐿
For each point 𝑃 in scene
If 𝑃 is in shadow cast by 𝐿 //how to compute?
Only use indirect lighting
(e.g., ambient term for Phong lighting)
Else
Evaluate full lighting model
(e.g., ambient, diffuse, specular for Phong)
Next: different methods for computing
whether 𝑃 is in shadow cast by 𝐿
Andries van Dam©
11/12/2015
Stencil shadow volumes implemented by
former cs123 TA and former UTA, now
Ph.D. Kevin Egan and former PhD student
and book co-author Prof. Morgan McGuire,
on nVidia chip
16 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Shadows (3/5) – Stencil Shadow Volumes

For each light + object pair, compute mesh
enclosing area where the object occludes the light

Find silhouette from light’s perspective





Every edge shared by two triangles, such that one triangle
faces light source and other faces away
On torus, where angle between normal vector and vector
to light becomes >90°
Project silhouette along light rays
Generate triangles bridging silhouette and its
projection to obtain the shadow volume
A point P is in shadow from light L if any shadow
volume V computed for L contains P


Can determine this quickly using multiple passes and a
“stencil buffer”
More here on Stencil Buffers, Stencil Shadow Volumes
Andries van Dam©
11/12/2015
Silhouette
Projected
Silhouette
Example shadow volume
(yellow mesh)
http://www.ozone3d.net/tutorials/images/stencil_
shadow_volumes/shadow_volume.jpg
17 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Shadows (4/5) – Another Multi-Pass Technique: Shadow Maps

Render scene using each light as center of
projection, saving only its z-buffer


Resultant 2D images are “shadow maps”,
one per light
Next, render scene from camera’s POV

To determine if point P on object is in
shadow:




compute distance dP from P to light source
convert P from world coordinates to shadow
map coordinates using the viewing and
projection matrices used to create shadow map
look up min distance dmin in shadow map
P is in shadow if dP > dmin , i.e., it lies behind a
closer object
Shadow map (on right) obtained by rendering
from light’s point of view (darker is closer)
Light
Camera
dmin
dP
P
Andries van Dam©
11/12/2015
18 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Shadows (5/5) – Shadow Map Tradeoffs

Pro: Can extend to support soft shadows



Con: Naïve implementation has impressively bad
aliasing problems


Soft shadows: shadows with “soft” edges (e.g. via blurring)
Stencil shadow volumes only useful for hard-edged
shadows
Hard shadows
Soft shadows
When the camera is closer to the scene than the light,
many screen pixels may be covered by only one shadow
map pixel (e.g., sun shining on Eiffel tower)
Many workarounds for aliasing issues



Percentage-Closer Filtering: for each screen pixel, sample
shadow map in multiple places and average
Cascaded Shadow Maps: multiple shadow maps, higher
resolution closer to viewer
Variance Shadow Maps: use statistical modeling
instead of simple depth comparison
Andries van Dam©
11/12/2015
Aliasing produced by naïve
shadow mapping
http://http.developer.nvidia.com/GPUGems3/gpugems3_ch08.html
http://www-sop.inria.fr/ reves/Marc.Stamminger/psm/
19 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Environment Mapping (for specular reflections) (1/2)

Approximate reflections by creating a skybox a.k.a.
environment map a.k.a. reflection map




Object
To create environment map, render entire scene from
center of an object one face at a time




A hack to approximate recursive raytracing
Often represented as six faces of a cube surrounding
scene
Can also be a large sphere surrounding scene, etc.
Skybox
Can use a single environment map for a cluster of objects
Can do this offline for static geometry, but must generate
at runtime for moving objects
Rendering environment map at runtime is expensive
(compared to using a pre-computed texture)
Can also use photographic panoramas (see Lab 8!)
Andries van Dam©
11/12/2015
20 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Environment Mapping (2/2)

Environment map
To sample environment map
reflection at point P:



Computer vector E from P to eye
Reflect E about normal N to obtain R
Use the direction of R to compute
the intersection point with the
environment map


Treat P as being center of map, i.e.
move environment map so P is in
center
See more here:
http://http.developer.nvidia.com/CgTutorial/cg
_tutorial_chapter07.html
Andries van Dam©
11/12/2015
R
P
N
E
eye
21 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Overview: Surface Detail

Observation




What if we replaced the 3D sphere on the right with a 2D circle?
The circle would have fewer triangles (thus renders faster)
If we kept the sphere’s normals, the circle would still look like a sphere!
Works because human visual system infers shape from patterns of light and
dark regions (“shape from shading”). Brightness at any point is determined by
normal vector, not by actual geometry of model
Andries van Dam©
11/12/2015
Image credit: Dave Kilian, ‘13
22 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Idea: Surface Detail






Two ideas: use texture map either to vary normals or
vary mesh geometry itself. First let’s vary normals
Start with a hi-poly (high polygon count) model
Decimate the mesh (remove triangles)
Encode hi-poly normal information into texture
Map texture (i.e., normals) onto lo-poly mesh
In fragment shader:


Look up normal for each pixel
Use Phong shading (normal interpolation0 to calculate
pixel color normal obtained from texture
Andries van Dam©
11/12/2015
Original hi-poly model
Lo-poly model with hi-poly
model’s normals preserved
23 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Normal Mapping

Idea: Fill a texture map with normals



Easiest to render, hardest to produce


Fragment shader samples 2D color texture
Interprets R as Nx ; G as Ny ; B as Nz
Usually need specialized modeling tools like
Pixologic’s Zbrush; algorithm on slide 27
Original mesh
Object-space normal map
Variants


Object-space: store raw, object-space normal
vectors (absolute specification of normal)
Tangent-space: store normals in tangent space




Mesh with normal map
applied (colors still there
for visualization purposes)
Tangent space: UVW system, where W is aligned
with the original low-res normal
RGB corresponds to perturbations of low-res normal
Relative specification of normal
Have to convert normals to world space to do
lighting calculation
Andries van Dam©
11/12/2015
Tangent space
Tangent-space normal map
http://www.3dvf.com/forum/3dvf/Blender-2/modelisation-organique-tutoriel-sujet_16_1.htm
24 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Normal Mapping Example

Normal mapping can completely alter the perceived geometry of a model
Andries van Dam©
11/12/2015
Image courtesy of www.anticz.com
25 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Normal Mapping Video
https://youtu.be/m-6Yu-nTbUU?t=2m21s
Andries van Dam©
11/12/2015
26 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Creating Normal Maps




PM
Algorithm (“baking the texture map”)
Original mesh M simplified to mesh S
For each pixel in normal map for S:

Find corresponding point on S: PS

Find closest point on original mesh: PM

Average nearby normals in M and save value in
normal map
Manual


M
PS
Normal map
Artist starts with S
Uses specialized tools to draw directly onto
normal map (‘sculpting’)
Andries van Dam©
S
11/12/2015
Each normal vector’s
X,Y,Z components stored
as R,G,B values in texture
27 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Bump Mapping: Example
+
Original object
(plain sphere)
=
Bump map
(height map to
perturb normals
- see next slide )
Sphere with
bump-mapped
normals
http://cse.csusb.edu/tong/courses/cs520/notes/texture.php
Andries van Dam©
11/12/2015
28 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Bump Mapping, Another Way to Perturb Normals

Idea: instead of encoding normals themselves in the map,
encode relative heights (or “bumps”)




Black: minimum height delta
White: maximum height delta
Much easier to create than normal maps
How to compute a normal from a height map?




Collect several height samples from texture
Convert height samples to 3D coordinates to calculate the average
normal vector at the given point
Transform computed normal from tangent space to object space (and
from there into world space)
You computed normals like this for a terrain mesh in Lab 4 (terrain)!
Normal vectors for triangles
neighboring a point. Each dot
corresponds to a pixel in the bump map.
Andries van Dam©
11/12/2015
Original tangent-space normal = (0, 0, 1)
Nearby values in (1D) bump map
Bump map visualized as tangentspace height deltas
Transformed tangent-space normal
29 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Other Techniques: Displacement Mapping

Actually move the vertices along their normals
by looking up height deltas in a height map



Displacing the vertices of the mesh will deform
the mesh, producing different vertex normals
because the face normals will be different
Unlike bump/normal mapping, this will produce
correct silhouettes and self-shadowing
By default, does not provide detail between
vertices like normal/bump mapping


To increase detail level we can subdivide the
original mesh
Can become very costly since it creates
additional vertices
Displacement map on a plane at different levels of subdivision
Andries van Dam©
11/12/2015
http://en.wikipedia.org/wiki/Displacement_mapping
http://www.nvidia.com/object/tessellation.html
https://support.solidangle.com/display/AFMUG/Displacement
30 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Other Techniques: Parallax Mapping (1/2)


Extension to normal/bump mapping
Distorts texture coordinates right before sampling, as a function of normal and eye
vector (next slide)



Example below: looking at stone sidewalk at an angle
Texture coordinates stretched along the near edges of the stones, which are “facing” the viewer
Similarly, texture coordinates compressed along the far edges of the stones, where you
shouldn’t be able to see the “backside” of the stones
Andries van Dam©
11/12/2015
http://amdstaging.wpengine.com/wordpress/media/2012/10/I3D2006Tatarchuk-POM.pdf
31 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Other Techniques: Parallax Mapping (2/2)

Modify original texture coordinates (u,v) to better approximate where the intersection
point of eye ray with perturbed surface would be on the bumpy surface

Option 1







For illustration purposes, we use a smooth
curve to show the varying heights in the
neighborhood of point (u,v)
Option 2


Sample the height map at point (u,v) to get height h
Approximate region around (u,v) with a surface of constant
height h
Intersect eye ray with approximate surface to get new texture
coordinates (u’, v’)
Sample the height map in region around (u,v) to get the
normal vector N’ (use the same normal averaging technique
that we used in bump mapping)
Approximate region around (u,v) with the tangent plane
Intersect eye ray with approximate surface to get (u’, v’)
Both produce artifacts when viewed from a steep angle
Other options discussed here: http://www.cescg.org/CESCG2006/papers/TUBudapest-Premecz-Matyas.pdf
Andries van Dam©
11/12/2015
32 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Other Techniques: Steep Parallax Mapping

Traditional parallax mapping only works for low-frequency bumps, does
not handle very steep, high-frequency bumps

Using option 1 from previous slide:



the black eye ray correctly
approximates the bump surface
the gray eye ray misses the first
intersection point on the highfrequency bump
Steep Parallax Mapping

Instead of approximating the bump surface, iteratively step along eye ray




at each step, check height map value to see if we have intersected the surface
more costly than naïve parallax mapping
Invented by Brown PhD ‘06 Morgan McGuire and Max McGuire
Adds support for self-shadowing
Andries van Dam©
11/12/2015
http://graphics.cs.brown.edu/games/SteepParallax/mcguire-steepparallax.pdf
33 of 34
cs123
INTRODUCTION TO COMPUTER GRAPHICS
Comparison
Creases between bricks look flat
Andries van Dam©
Creases look progressively deeper
11/12/2015
Objects have self-shadowing
34 of 34