CS 352:
Computer Graphics
Chapter 8:
The
Rendering
Pipeline
Chapter 8 - 2
Interactive Computer Graphics
Overview



Geometric processing: normalization,
clipping, hidden surface removal, lighting,
projection (front end)
Rasterization or scan conversion, including
texture mapping, etc. (back end)
Fragment processing and display
Chapter 8 - 3
Interactive Computer Graphics
Geometric Processing

Front-end processing steps (3D floating point)










Evaluators (converting curved surfaces to polygons)
Normalization (modeling transform, convert to world
coordinates)
Projection (convert to screen coordinates)
Hidden-surface removal (object space)
Computing texture coordinates
Computing vertex normals
Lighting (assign vertex colors)
Clipping
Perspective division
Backface culling
Chapter 8 - 4
Interactive Computer Graphics
Rasterization


Back-end processing works on 2D objects in
screen coordinates
Processing includes








Scan conversion of primitives including shading
Texture mapping
Fog
Scissors test
Alpha test
Stencil test
Depth-buffer test
Other fragment operations: blending, dithering,
logical operations
Chapter 8 - 5
Interactive Computer Graphics
Display


Convert frame buffer to video signal
Other considerations:


Color correction
Antialiasing
Chapter 8 - 6
Interactive Computer Graphics
Geometric Transformations

Six coordinate systems of interest:






Object coordinates
World coordinates
Eye coordinates [after modelview]
Clip coordinates [after projection]
Normalized device coordinates [after ÷w]
Window (screen) coordinates [scale to screensize]
Chapter 8 - 7
Interactive Computer Graphics
Line-Segment Clipping


Clipping may happen in multiple places in the
pipeline (e.g. early trivial accept/reject)
After projection, have lines in plane, with
rectangle to clip against
Chapter 8 - 8
Interactive Computer Graphics
Clipping a Line Against xmin






Given a line segment from (x1,y1) to (x2,y2),
Compute m=(y2–y1)/(x2–x1)
Line equation: y = mx + h
h = y1 – m x1 (y intercept)
Plug in xmin to get y
Check if y is between y1 and y2.
This take a lot of floating-point math. How to
minimize?
Chapter 8 - 9
Interactive Computer Graphics
Cohen-Sutherland Clipping


For both endpoints compute a 4-bit outcode
(o1, o2) depending on whether coordinate is
outside cliprect side
Some situations can be handled easily
Chapter 8 - 10
Interactive Computer Graphics
Cohen-Sutherland Conditions

Cases.




1. If o1=o2=0, accept
2. If one is zero, one nonzero, compute an
intercept. If necessary compute another intercept.
Accept.
3. If o1 & o2  0. If both outcodes are nonzero
and the bitwise AND is nonzero, two endpoints lie
on same outside side. Reject.
3. If o1 & o2  0. If both outcodes are nonzero
and the bitwise AND is zero, may or may not have
to draw the line. Intersect with one of the window
sides and check the result.
Chapter 8 - 11
Interactive Computer Graphics
Cohen-Sutherland Results



In many cases, a few integer comparisons
and Boolean operations suffice.
This algorithm works best when there are
many line segments, and most are clipped
But note that the y=mx+h form of equation
for a line doesn’t work for vertical lines
(need a special case in your code)
Chapter 8 - 12
Interactive Computer Graphics
Polygon Clipping




Clipping a polygon can result in
lots of pieces
Replacing one polygon with
many may be a problem in the
rendering pipeline
Could treat result as one
polygon: but this kind of
polygon can cause other
difficulties
Some systems allow only
convex polygons, which don’t
have such problems (OpenGL
has tessellate function in glu
library)
Chapter 8 - 13
Interactive Computer Graphics
Sutherland-Hodgeman
Polygon Clipping



Could clip each edge of polygon individually
Pipelined approach: clip polygon against each
side of rectangle in turn
Treat clipper as “black box” pipeline stage
Chapter 8 - 14
Clipping Pipeline

Clip each bound in turn
Interactive Computer Graphics
Chapter 8 - 15
Interactive Computer Graphics
Clipping in Hardware


Build the pipeline stages in hardware so you can
perform four clipping stages at once
[A partial answer to the question of what to do with
all that chip area, 1 billion + transistors…]
Chapter 8 - 16
Interactive Computer Graphics
Clipping complicated objects



Suppose you have many complicated objects,
such as models of parts of a person with
thousands of polygons each
When and how to clip for maximum
efficiency?
How to clip text? Curves?
Chapter 8 - 17
Interactive Computer Graphics
Clipping Other Primitives




It may help to clip more complex shape early
in the pipeline
This may be simpler and
less accurate
One approach: bounding
boxes (sometimes called
trivial accept-reject)
This is so useful that
modeling systems often
store bounding box
Chapter 8 - 18
Interactive Computer Graphics
Clipping Curves, Text





Some shapes are so
complex that they
are difficult to clip
analytically
Can approximate with line segments
Can allow the clipping to occur in the frame
buffer (pixels outside the screen rectangle
aren’t drawn)
Called “scissoring”
How does performance compare?
Chapter 8 - 19
Interactive Computer Graphics
Hidden surface removal

Object space vs. Image space

The main image-space algorithm: depth buffer

Drawbacks




Aliasing
Rendering invisible objects
Doesn’t handle transparency correctly
How would object-space hidden surface
removal work?
Chapter 8 - 20
Interactive Computer Graphics
Depth sorting


The painter’s algorithm:
draw from back to front
Depth-sort hidden surface removal:



sort display list by z-coordinate from back to front
render
Drawbacks


it takes some time (especially with bubble sort!)
it doesn’t work
Chapter 8 - 21
Interactive Computer Graphics
Depth-sort difficulties




Polygons with overlapping
projections
Cyclic overlap
Interpenetrating polygons
What to do?
Chapter 8 - 22
Interactive Computer Graphics
Scan Conversion



At this point in the pipeline, we have only
polygons and line segments. Render!
To render, convert to pixels (“fragments”)
with integer screen coordinates (ix, iy),
depth, and color
Send fragments into fragment-processing
pipeline
Chapter 8 - 23
Interactive Computer Graphics
Rendering Line Segments



How to render a line segment from (x1, y1) to
(x2, y2)?
Use the equation
y = mx + h
What about horizontal
vs. vertical lines?
Chapter 8 - 24
Interactive Computer Graphics
DDA Algorithm

DDA: Digital Differential Analyzer
for (x=x1; x<=x2; x++)
y += m;
draw_pixel(x, y, color)


Handle slopes 0 <= m <= 1; handle others
symmetrically
Does this
need floating
point math?
Chapter 8 - 25
Interactive Computer Graphics
Bresenham’s Algorithm


The DDA algorithm requires a floating point
add and round for each pixel: eliminate?
Note that at each step we will go E or NE.
How to decide which?
−
Chapter 8 - 26
Interactive Computer Graphics
Bresenham Decision Variable


Note that at each step we will go E or NE. How to
decide which?
Hint: consider d=a-b, where a and b are distances to
NE and E pixels
−
Chapter 8 - 27
Interactive Computer Graphics
Bresenham Decision Variable




Bresenham algorithm uses decision variable d=a-b,
where a and b are distances to NE and E pixels
If d<=0, go NE;
if d>0, go E
Let dx = x2-x1, dy=y2-y1
Use decision variable
−
d = dx (a-b)
[only sign matters]
Chapter 8 - 28
Interactive Computer Graphics
Bresenham Decision Variable



d =(a-b) dx
Let dk be the value of d at x = k + ½
Move E:



dk = dx(a-b) = dx((j+3/2–yk) – (yk–(j+1/2)))
dk+1 = dx(a-b) = dx((j+3/2–yk–m) – (yk+m–(j+1/2)))
dx+1 – dk = dx (-2m) = - 2 dY
−

Algorithm:


dk+1 = dk – 2 dy (if dk > 0) (last move was E)
dk+1 = dk – 2 (dy-dx) (if dk <= 0) (last move was NE)
Chapter 8 - 29
Interactive Computer Graphics
Bresenham’s Algorithm

Set up loop computing d at x1, y1
for (x=x1; x<=x2; )
x++;
d += 2dy;
if (d >= 0) {
y++;
d –= 2dx; }
drawpoint(x,y);


Pure integer math, and not much of it
So easy that it’s usually implemented in one
instruction for several points in parallel
Chapter 8 - 30
Interactive Computer Graphics
Rasterizing Polygons


Polygons may be or may not be simple,
convex, flat. How to render?
Amounts to inside-outside testing: how to tell
if a point is in a polygon?
Chapter 8 - 31
Interactive Computer Graphics
Winding Test

Most common way to tell if a point is in a
polygon: the winding test.


Define “winding number” w for a point: signed
number of revolutions around the point when
traversing boundary of polygon once
When is a point “inside” the polygon?
Chapter 8 - 32
Interactive Computer Graphics
OpenGL and Complex Polygons



OpenGL guarantees correct rendering only for
simple, convex, planar polygons
OpenGL tessellates concave polygons
Tessellation depends on winding rule you tell
OpenGL to use: Odd, Nonzero, Pos, Neg,
ABS_GEQ_TWO
Chapter 8 - 33
Winding Rules
Interactive Computer Graphics
Chapter 8 - 34
Interactive Computer Graphics
Scan-Converting a Polygon


General approach: ideas?
One idea: flood fill


Draw polygon edges
Pick a point (x,y) inside and flood fill with DFS
flood_fill(x,y) {
if (read_pixel(x,y)==white) {
write_pixel(x,y,black);
flood_fill(x-1,y);
flood_fill(x+1,y);
flood_fill(x,y-1);
flood_fill(x,y+1);
} }
Chapter 8 - 35
Interactive Computer Graphics
Scan-Line Approach





More efficient: use a scan-line rasterization
algorithm
For each y value, compute x
intersections. Fill according
to winding rule
How to compute intersection
points?
How to handle shading?
Some hardware can handle
multiple scanlines in parallel
Chapter 8 - 36
Interactive Computer Graphics
Singularities




If a vertex lies on a scanline,
does that count as 0, 1, or 2
crossings?
How to handle singularities?
One approach: don’t allow.
Perturb vertex coordinates
OpenGL’s approach: place pixel
centers half way between
integers (e.g. 3.5, 7.5), so
scanlines never hit vertices
Chapter 8 - 37
Interactive Computer Graphics
Aliasing


How to render the line
with reduced aliasing?
What to do when polygons
share a pixel?
Chapter 8 - 38
Interactive Computer Graphics
Anti-Aliasing




Simplest approach: area-based weighting
Fastest approach: averaging nearby pixels
Most common approach: supersampling
(compute four values per pixel and avg, e.g.)
Best approach: weighting based on distance
of pixel from center of line; Gaussian fall-off
Chapter 8 - 39
Interactive Computer Graphics
Temporal Aliasing


Need motion blur for motion that doesn’t
flicker
Common approach: temporal supersampling


render images at several times within frame time
interval
average results
Chapter 8 - 40
Interactive Computer Graphics
Display Considerations




Color systems
Color quantization
Gamma correction
Dithering and Halftoning
Chapter 8 - 41
Interactive Computer Graphics
Additive and Subtractive Color
Chapter 8 - 42
Interactive Computer Graphics
Common Color Models
Chapter 8 - 43
Color Systems

RGB
YIQ
CMYK
HSV, HLS
Chromaticity

Color gamut




Interactive Computer Graphics
Chapter 8 - 44
HLS




Hue: “direction” of color:
red, green, purple, etc.
Saturation: intensity.
E.g. red vs. pink
Lightness: how
bright
To the right: original,
H, S, L
Interactive Computer Graphics
Chapter 8 - 45
Interactive Computer Graphics
YIQ







Used by NTSC TV
Y = luma, same as
black and white
I = in-phase
Q = quadrature
The eye is more sensitive to
blue-orange than purple-green,
so more bandwidth is allotted
Y = 4 MHz, I = 1.3 MHz, Q = 0.4 MHz,
overall bandwidth 4.2 MHz
Linear transformation from RBG:



Y = 0.299 R + 0.587 G + 0.114 B
I = 0.596 R – 0.274 G – 0.321 B
Q = 0.211 R – 0.523 G + 0.311 B
Chapter 8 - 46
Interactive Computer Graphics
Chromaticity

Color researchers often
prefer chromaticity
coordinates:






t1 = T1 / (T1 + T2 + T3)
t2 = T2 / (T1 + T2 + T3)
t3 = T3 / (T1 + T2 + T3)
Thus, t1+t2+t3 = 1.0.
Use t1 and t2; t3 can be
computed as 1-t1-t2
Chromaticity diagram uses
XYZ color system based on
human perception
experiments



Y, luminance
X, redness (roughly)
Z, blueness (roughly)
Chapter 8 - 47
Interactive Computer Graphics
Color temperature



Compute color temperature
by comparing chromaticity
with that of an ideal blackbody radiator
Color temperature is that
were the headed blackbody radiator matches
color of light source
Higher temperatures are
“cooler” colors – more
green/blue; warmer colors
(yellow-red) have lower
temperatures
CIE 1931 chromaticity diagram
(wikipedia)
Chapter 8 - 48
Interactive Computer Graphics
Halftoning



How do you render a colored image when colors can
only be on or off (e.g. inks, for print)?
Halftoning: dots
of varying sizes
[But what if only
fixed-sized pixels
are available?]
Chapter 8 - 49
Interactive Computer Graphics
Dithering



Dithering (patterns of b/w or colored dots)
used for computer screens
OpenGL can dither
But, patterns can be visible and bothersome.
A better approach?
Chapter 8 - 50
Interactive Computer Graphics
Floyd-Steinberg Error
Diffusion Dither

Spread out “error term”





7/16
3/16
5/16
1/16
right
below left
below
below right
Note that you can
also do this for color
images (dither a color
image onto a fixed
256-color palette)
Chapter 8 - 51
Interactive Computer Graphics
Color Quantization



Color quantization: modifying a full-color
image to render with a 256-color palette
For a fixed palette (e.g. web-safe colors),
can use closest available color, possibly with
error-diffusion dither
Algorithm for selecting an adaptive palette?

E.g. Heckbert Median-Cut algorithm



Make a 3-D color histogram
Recursively cut the color cube in half at a median
Use average color from each resulting box
Chapter 8 - 52
Interactive Computer Graphics
Hardware Implementations






Pipeline architecture for speed
(but what about latency?)
Originally, whole pipeline on CPU
Later, back-end on graphics card
Then, whole pipeline in hardware on graphics
card
Then, parts of pipeline done on GPUs
Now, hardware pipeline is gone—done in
software on a large number of GPUs
Chapter 8 - 53
Interactive Computer Graphics
Future Architectures?

20 years ago, performance of 1 M polygons per
second cost millions





Performance limited by memory bandwidth
Main component of price was lots of memory chips
Now an iPhone 5 does 115 million triangles (1.8 billion
texels) per second
Fastest performance today achieved with many
cores on each graphics chip (and several chips in
parallel)
How to use parallel graphics cores/chips?



All vertices go through vertex shaders – can do in parallel
All fragments go through fragment shaders – can do in parallel
Resulting fragments combined into a single image – easy with sufficient
memory bandwidth
Chapter 8 - 54
Interactive Computer Graphics
NVidia GeForce GTX 590

Facts










“Most powerful graphics card” (of 2011)
Dual 512-core GTX 500 GPUs (1024 CUDA unified shaders)
2 x 16 Streaming Multiprocessor (SM) processor groups,
each with 32 Streaming Processors (SPs)
PhysX physics engine on the card (used for flowing clothing,
smoke, debris, turbulence, etc)
Memory bandwidth: 327.7 GB/sec
Texture fill rate: 77.7 billion/sec
4X antialiasing
3 billion transistors x 2
Real-time ray tracing
[GTX 980: 224 GB/sec]