ARTISTIC RENDERING OF NATURAL ENVIRONMENTS
by
Margarita Dobreva Bratkova
A dissertation submitted to the faculty of
The University of Utah
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
in
Computer Science
School of Computing
The University of Utah
December 2009
c Margarita Dobreva Bratkova 2009
Copyright °
All Rights Reserved
THE UNIVERSITY OF UTAH GRADUATE SCHOOL
SUPERVISORY COMMITTEE APPROVAL
of a dissertation submitted by
Margarita Dobreva Bratkova
This dissertation has been read by each member of the following supervisory committee and by
majority vote has been found to be satisfactory.
Co-chair:
William B. Thomspon
Co-chair:
Peter Shirley
Adam Bargteil
Amy Gooch
Harvey Miller
THE UNIVERSITY OF UTAH GRADUATE SCHOOL
FINAL READING APPROVAL
To the Graduate Council of the University of Utah:
in its final form and
I have read the dissertation of
Margarita Dobreva Bratkova
have found that (1) its format, citations, and bibliographic style are consistent and acceptable;
(2) its illustrative materials including figures, tables, and charts are in place; and (3) the final
manuscript is satisfactory to the Supervisory Committee and is ready for submission to The
Graduate School.
Date
William B. Thomspon
Co-chair, Supervisory Committee
Date
Peter Shirley
Co-chair, Supervisory Committee
Approved for the Major Department
Martin Berzins
Chair
Approved for the Graduate Council
David S. Chapman
Dean of The Graduate School
ABSTRACT
Mountain panorama maps are aerial view paintings that depict complex, three-dimensional landscapes in a pleasing and understandable way. Painters and cartographers have
developed techniques to create such artistic landscapes for centuries, but the process
remains difficult and time-consuming.
In this dissertation, we derive principles and heuristics for panorama map creation
of mountainous terrain from a perceptual and artistic analysis of a panorama map of
Yellowstone National Park painted by two different artists, Heinrich Berann and James
Niehues. We then present methods to automatically produce landscape renderings in
the visual style of the panorama map. Our algorithms rely on USGS terrain and its
corresponding classification data. Our surface textures are generated using perceptual
metrics and artistic considerations, and use the structural information present in the
terrain to guide the automatic placement of image space strokes for natural surfaces such
as forests, cliffs, snow, and water.
An integral part of automatic rendering is choosing a viewpoint that is meaningful and
representative of the landscape. In this dissertation we examine the automatic generation
of well-composed and purposeful images in the context of mountainous terrain. We
explore a set of criteria based on utility, perception, and aesthetics applicable to natural
outdoor scenes. We also propose a method that uses the criteria to produce renderings of
terrain scenes automatically.
Finally, arising from difficulties we encountered using color transfer to improve the
final rendered colors of our panorama map renderings, we propose a novel color model,
oRGB, that is based on opponent color theory. Like HSV, it is designed specifically
for computer graphics. However, it is also designed to work well for computational
applications such as color transfer, where HSV falters. Despite being geared towards
computation, oRGB’s natural axes facilitate HSV-style color selection and manipulation.
oRGB also allows for new applications such as a quantitative cool-to-warm metric,
intuitive color manipulations and variations, and simple gamut mapping. This new
color model strikes a balance between simplicity and the computational qualities of color
spaces such as CIE L*a*b*.
v
CONTENTS
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iv
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ix
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
x
ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
CHAPTERS
1.
2.
3.
INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1.1 Map Making . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.1 Cartographic Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 The Mountain Panorama . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2.1 Arguments for Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Dissertation Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.2 Other Publications of This Work . . . . . . . . . . . . . . . . . . . . . . . . . .
1
1
2
3
4
5
6
BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.1 The Mountain Panorama . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1.1 The Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Digital Terrain Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Nonphotorealistic Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
11
12
13
ANALYSIS OF PANORAMA MAP DESIGN . . . . . . . . . . . . . . . . . . . . .
15
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Visual Perception and Pictorial Cues . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Perceptual Elements in Panorama Map Paintings . . . . . . . . . . . . . . . . . .
3.3.1 Texture Gradients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1.1 Texel Shape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1.2 Texel Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1.3 Texel Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1.4 Texel Occlusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.2 Surface Contours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.3 Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.4 Silhouettes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Stylistic Elements in Panorama Map Paintings . . . . . . . . . . . . . . . . . . . .
3.4.1 Base Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
16
18
19
20
20
21
21
21
23
24
24
25
4.
5.
3.4.2 Brushstroke Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.3 Brushstroke Shapes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Summary of Principles and Heuristics for Panorama
Map Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
AUTOMATED PANORAMA MAP RENDERING . . . . . . . . . . . . . . . . .
29
4.1 Goals and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Preliminary Setup Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.1 Terrain Geometry and Classification . . . . . . . . . . . . . . . . . . . . . . .
4.2.2 Vertical Exaggeration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.3 First-Order Terrain Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.4 Geometry Curving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.5 Preliminary Terrain Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.6 Classification Boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 Base Shading Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 Fall Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Surface Textures Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5.1 Forest Textures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5.2 Cliff and Snow Textures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5.3 Water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29
30
30
32
32
33
34
36
36
38
39
40
41
42
43
49
AUTOMATIC VIEW SELECTION FOR OUTDOOR SCENES . . . . . . .
51
5.1 Good Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1.1 Viewpoint Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Viewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2.1 Image Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 Criteria for Automatic View Selection and Image
Composition of Outdoor Scenes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.1 Utility Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.2 Perceptual Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3.3 Aesthetic Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4 Automatic Viewpoint Selection and Image Composition . . . . . . . . . . . .
5.4.1 Finding a Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.2 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.3 The Objective Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.3.1 Utility Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.3.2 Perceptual Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.3.3 Aesthetic Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4.3.4 Weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.6 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
52
54
55
vii
25
26
55
55
56
57
59
59
60
60
61
61
62
63
63
71
71
6.
OPPONENT COLOR SPACE: ORGB . . . . . . . . . . . . . . . . . . . . . . . . . .
76
6.1 Color Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
6.1.1 Hue Spacing Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.1.2 Hering’s Opponent Process Color Theory . . . . . . . . . . . . . . . . . . . 79
6.1.3 Opponent Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
6.1.4 Limitations of Color Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.1.5 Terminology and Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2 The oRGB Color Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.2.1 oRGB Step 1: Linear Transform of R′ G′ B′ to L′C′C′ . . . . . . . . . . . 85
6.2.2 oRGB Step 2: Non-uniform Rotation Around Luma Axis L′
from L′C′C′ to oRGB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.2.3 Properties of oRGB Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.2.3.1 Locations of Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.2.3.2 The oRGB Gamut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.3 Sample Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.3.1 Color Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.3.2 Cool to Warm Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.3.3 Color Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.3.4 Gamut Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
7.
CONCLUSION AND FUTURE WORK . . . . . . . . . . . . . . . . . . . . . . . . . 104
7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2.1 Geometric Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2.2 Enhanced Spatial Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2.3 Landform Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
104
105
105
107
107
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
viii
LIST OF TABLES
4.1 Rendering statistics for our example panoramas. . . . . . . . . . . . . . . . . . . . .
44
5.1 Rectangle ratios, sorted in order of preference by Fechner. The golden
section is preferred in both studies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
58
5.2 Criteria weight ranges that proved useful in practice, as well as weights
used for all the results in this paper. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63
5.3 Rendering statistics for our solution. Total time includes processing of the
dataset, building of the bvh, and rendering as well as evaluation of each of
the 210 images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
64
6.1 Different color spaces tend to produce different out-of-gamut results. The
first column lists the color space tested. The second column shows the
average percentage of out-of-gamut pixels. The remaining three columns
demonstrate the out-of-gamut percentage range per RGB channel. . . . . . . 102
LIST OF FIGURES
1.1 Heinrich Berann’s Panorama Map of Yosemite National Park. . . . . . . . . .
3
2.1 Panorama maps in summer palettes using watercolor. . . . . . . . . . . . . . . . .
8
2.2 Panorama maps in spring palettes using guache and oil. . . . . . . . . . . . . . .
9
2.3 Winter panoramas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
3.1 Panorama Map Paintings of Yellowstone National Park by Heinrich Bec
rann and James Niehues. °Trail
Tracks, used with permission. . . . . . . . .
17
3.2 Pictorial cues for depth and surface orientation arising from perspective
scaling (P), foreshortening (F), interposition (I), intrinsic surface markings (S), and light transport (L). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Trees in the distance (top) and foreground (bottom). If size of trees varied
with true perspective, pixel size of trees in the background would be a
small fraction of their foreground size. Clearly this is untrue, especially in
c
Berrann’s painting (left). °Trail
Tracks, used with permission. . . . . . . . .
3.4 Tree strokes in forest textures are laid down so that they partially overlap
each other. The visual system interprets that to mean that the stroke that
c
overlaps is physically nearer. °Trail
Tracks, used with permission. . . . . .
3.5 This rendering is a visualization of the flow along fall lines using line
integral convolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Tree strokes are placed along the imaginary fall lines of the geometry (top
images). Cliffs and snow are painted with strokes that follow the direction
c
of fall lines (bottom). °Trail
Tracks, used with permission. . . . . . . . . . . .
3.7 Light direction is diagonal to the dominant direction of valleys and mounc
tain ranges. °Trail
Tracks, used with permission. . . . . . . . . . . . . . . . . . .
18
20
22
22
23
24
3.8 Silhouetted curves are used to separate an object from its background and
c
are used in panorama maps to imply depth. °Trail
Tracks, used with
permission. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
3.9 Sample of light (L), medium (M) and dark (D) colors used by Niehues and
Berann in their Yellowstone painting for areas in the foreground and the
background. The colors were sampled from a digitized version of their
paintings using Photoshop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
3.10 Sample of the shapes for tree strokes used by (a) Niehues and (b) Berann.
For each artist, the two samples to the left correspond to strokes used for
evergreen trees, while the two to the right - for deciduous trees. . . . . . . . .
27
4.1 A panorama map rendering produced by our system for Yellowstone National Park, using Berann-style brushstrokes and colors. Refer to Figure 3.1 for the original Berann painting. . . . . . . . . . . . . . . . . . . . . . . . . . .
30
4.2 An overview of the steps taken by our algorithm. It can be grouped in
three stages: preliminary setup, base shading, and surface textures. . . . . . .
31
4.3 When the base of the terrain is curved, the image will include both nonobscured features in the foreground, as well as horizon and sky in the distance. 34
4.4 Comparison of the same terrain rendered from the same viewpoint and
camera parameters, but with different geometry modifications: leaving
the geometry as is (top), our geometry curving method (middle), rotating
the geometry (instead of rotating the view) to show corresponding view
from above (bottom). Geometry curving preserves the 3D properties of
the geometry, while allowing us to see both the horizon and the terrain
features of interest, that would have otherwise been lost due to occlusion. .
35
4.5 In this rendering of Yellowstone National Park, textures are using only the
base Berann colors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
4.6 Our method of tree distribution uses placement of trees along fall lines
(center). As a comparison, you can see the same region with trees placed
using jittered sampling (left) and random sampling (right). . . . . . . . . . . . .
38
4.7 Paths before (left) and after (right) pruning. . . . . . . . . . . . . . . . . . . . . . . .
39
4.8 Yellowstone National Park, rendered with rock, snow, water, and two
kinds of tree textures with Niehues-style strokes and colors. . . . . . . . . . . .
43
4.9 Cliff strokes used by our system for (a) Yellowstone (b) Rocky Mountain
National Park. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44
4.10 White Sands National Monument, rendered with brush tree, sand, and
grass texture. Colors were selected from real images of the terrain. . . . . . .
45
4.11 White Sands National Monument as displayed by Google Earth. . . . . . . . .
45
4.12 Mount Rainier and Mount St. Helens, rendered with tree, snow, grass,
and water texture. Colors were selected from a combination of Berann
and Niehues colors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
46
4.13 Rocky Mountain National Park terrain, rendered with rock, snow, water,
and two kinds of tree textures. Rendered with colors from images of the
terrain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
47
4.14 Color transfer result for Figure 4.13 applied in l αβ . . . . . . . . . . . . . . . . . .
48
4.15 Color transfer result for Figure 4.13 applied in RGB space. . . . . . . . . . . . .
49
4.16 Color transfer result for Figure 4.13 applied in oRGB space. . . . . . . . . . . .
50
5.1 Good and bad view of an outdoor scene. . . . . . . . . . . . . . . . . . . . . . . . . .
52
5.2 Rendering of Yellowstone NP produced with initial camera parameters
and after optimization with energy 0.71. . . . . . . . . . . . . . . . . . . . . . . . . .
xi
65
5.3 Rendering of Rocky Mountain NP produced with initial camera parameters and with optimized parameters with energy 0.63. . . . . . . . . . . . . . . .
66
5.4 Rendering of Death Valley NP produced with initial camera parameters
and after optimization with energy 0.66. . . . . . . . . . . . . . . . . . . . . . . . . .
67
5.5 Value of the objective function and its individual components during the
optimization of Yellowstone NP, as well as corresponding values of the
camera parameters. The horizontal axis represents number of iterations. .
68
5.6 Value of the objective function and its individual components during the
optimization of Rocky Mountain NP, as well as corresponding values of
the camera parameters. The horizontal axis represents number of iterations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
69
5.7 Value of the objective function and its individual components during the
optimization of Death Valley NP, as well as corresponding values of the
camera parameters. The horizontal axis represents number of iterations. .
70
5.8 Rendering of Yellowstone NP produced after optimization with energy
0.66. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
72
5.9 Rendering of Rocky Mountain NP produced after optimization with energy 0.68. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
73
5.10 Rendering of Death Valley NP produced after optimization with energy
0.59. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
74
6.1 Visualization of the oRGB decomposition of the image in the top left into
luma, yellow-blue, and red-green channels. . . . . . . . . . . . . . . . . . . . . . . . .
77
6.2 Visualization of four common types of color circles. The ordering of hues
is preserved in each, while the position of the hues on the circle varies. The
subtractive circle is that used by painters while a cyan-magenta-yellow
circle (not shown) is used by most ink printing processes. . . . . . . . . . . . . .
79
6.3 The color circle divided into its opponent sides. The top part of the circle
is composed of colors that have a red tone in them, the bottom - green, the
right - yellow, and the left - blue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
80
6.4 Visualization of the chroma channels for YCC, l αβ , and CIE L*a*b color
spaces. Compared to our new oRGB space (Figure 6.1), these spaces do
not have a true red-green channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
81
6.5 The pipelines of R′ G′ B′ to CIE L*a*b* (top) and R′ G′ B′ to l αβ (bottom).
85
′ ,C′ ). . . . . . . . . . . . . .
6.6 The pipeline of R′ G′ B′ (R′ , G′ , B′ ) to oRGB (L′ ,Cyb
rg
86
6.7 Orthographic view of RGB cube looking along white-black axis (left). No
affine transform will put red-white-green in line (middle). The y-axis will
have a red-magenta and a green-cyan as its extreme values (right). . . . . . .
86
6.8 To take the red and green directions to the vertical axis, we compress
angles on the blue side, and expand them on the yellow one. . . . . . . . . . . .
87
xii
6.9 Visualization of the gamut of our color space seen along the luma axis
before (left) and after (right) nonuniform rotation. After rotation, the yaxis is a true red-green axis while keeping the yellow-blue the same. . . . .
88
6.10 Colors to the right of the magenta-chartreuse axis are warm, to the left are
cool, and in the center are temperate. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
89
6.11 The original image is modified by doubling the standard deviation in the
red-green channel resulting in out-of-gamut red values but improved green
variation. Clamping in R′ G′ B′ leaves overly saturated reds. Both clamping
and scaling in L′C′C′ preserve luma, while scaling also produces a more
even saturation distribution. Both methods retain the improved green
variation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
92
6.12 Modifying color contrast through deviation adjustment. One step down/left
scales deviation by 1/α while a step up/right scales deviation in that
channel by α . The horizontal axis is a yellow-blue standard deviation
change (left lowers and right increases) and the vertical axis is a red-green
change (down lowers and up increases). . . . . . . . . . . . . . . . . . . . . . . . . . .
93
6.13 Another example of color adjustment using deviation scaling, similar to
Figure 6.12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
6.14 Changing the overall tone of an image through mean shifting. The natural
axes of oRGB provide intuitive shifts along primaries (yellow, red, blue,
and green) and secondaries (orange, magenta, cyan, and chartreuse). The
horizontal axis is yellow (shifting right) and blue (shifting left). The
vertical axis is red (shifting up) and green (shifting down). . . . . . . . . . . . .
95
6.15 Instead of applying mean shifting to the final image, as in Figure 6.14,
we used the intuitive oRGB axes as a color “variations” selector for the
diffuse term in a car rendering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
96
6.16 Retaining an NPR style while maintaining the diffuse lighting look. The
luma channel in oRGB is modified to partially resemble traditional Gouraud
style shading while maintaining a cool-to-warm transition (top). Shading
from Gooch et al. [39] (bottom). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
98
6.17 The statistics of the target image are transferred to the original image in a
variety of color spaces. Transferring red towards blue produces purple in
oRGB instead of yellows, reds, or browns produced by the other spaces. . .
99
6.18 The statistics of the target image are transferred to the original image in a
variety of color spaces. A magenta shifted towards pure red becomes pure
red in oRGB instead of remaining magenta. . . . . . . . . . . . . . . . . . . . . . . . 100
6.19 Linear version of our model for color transfer (original and target images
are the same as the ones used in 6.17 and 6.18). . . . . . . . . . . . . . . . . . . . . 101
7.1 At the top, we have a rendering of the original dataset. At the bottom, we
have a rendering that exaggerates one of the peaks in Little Cottonwood
Canyon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
xiii
ACKNOWLEDGEMENTS
I would like to thank my advisors, William Thompson and Peter Shirley for their
invaluable support and thoughtful guidance. Sincere gratitude is also due to the members
of my supervisory committee, Harvey Miller, Amy Gooch, and Adam Bargteil.
Thank you friends and family for all your support! You know who you are. I could
not have made it without you.
I would like to thank Solomon Boulos for constructive comments and help with
mathematical derivations. Special thanks to Simon Premoze for useful discussions on
terrain rendering and panorama maps. Thank you, Bill Martin, for sharing your LIC
code. I acknowledge the Trail Tracks company (www.trailtracks.com) for allowing me
to use images of the Yellowstone panorama map by James Niehues.
CHAPTER 1
INTRODUCTION
Visualization is the graphical representation of data or concepts. There are many
different types of visualization: information visualization of symbolic data and relations,
scientific visualization of spatial relationships, artistic abstraction of the world around
us. A type of terrain visualization, the geographic map, is fundamental to cartography.
While specific maps vary, they all share the distinction of being a spatial description and
abstraction of geographic reality.
1.1 Map Making
Map making requires the symbiosis of science and aesthetics, and produces an image
that communicates information in perceptually meaningful ways [11, 27]. It is a process
involving three basic steps: data collection, map design, and map production [65]. In
the data collection stage, cartographers assemble the raw data from which information
will be extracted, manipulated, or abstracted. During map design, cartographers apply
cartographic principles and techniques to manipulate the data in perceptually meaningful
ways. And finally, in the course of map production, they make choices about the map
reproduction, including its digital or physical presentation, and its final production.
1.1.1
Cartographic Automation
Because the map making process has been historically complicated and time consuming, there have been major strides towards its automation. The advent of computer graphics has been especially useful in the automation of 3D displays of map data, specifically
using digital terrain models with superimposed topographic features [59]. Availability of
computer software has also brought about improved, customizable maps and has allowed
cartographers to experiment more freely with map design options [65].
2
There has been a significant development in GIS (geographic information systems).
Such computer systems assist a user in capturing, managing, integrating, manipulating,
analyzing, and displaying data which are spatially referenced to the Earth. For 3D terrain
and visualization, GIS systems allow the user to visualize terrain datasets and to drape
geo-referenced textures over the terrain. Such interactive automated systems have proven
very useful and are becoming more and more popular.
Though there has been success in the automation of data collection and map production, very little has been done in the automation of the map design process [59, 102].
The elicitation and subsequent formalization of cartographic knowledge has proved elusive [35]. Mackaness states that the “study of traditional cartography has revealed methods by which maps are generalized, but the overall/collective approach is not applicable
in the automated environment” [77].
1.2 The Mountain Panorama
One of the most artistic geographic maps, the mountain panorama map, visualizes
complex three-dimensional landscapes and conveys a sense of space and appropriate
geographical information. It provides easy visual reference as well as understanding of
a region’s features for those unfamiliar with topographic maps.
Whereas maps provide orientation from every point in the area they cover,
panoramas do that only for a single location. But while map reading is a
skill that must be learned, anyone with a few years of elementary school can
understand a panorama (Imhof, 1963) [54].
Panorama maps have many uses. They are incorporated into ski-resort maps, golf
courses maps, hiking and biking trails, climbing routes, general recreation maps, as
well as regional landscape views. Panoramic images are used in brochures, web sites,
ads, maps, posters, prints, post-cards. Throughout the years, panorama map views have
confirmed their credibility and effectiveness as promotional and informational tools.
Painting panoramas requires cartographic skills, artistic talent, and ability to translate
2D topographic data into graphical 3D representation that people can understand [86].
A magnificent example of a panorama map is Heinrich Berann’s painting of Yosemite
National Park (Figure 1.1).
3
1.2.1
Arguments for Automation
The process of panorama map generation is very time-consuming. It took Heinrich
Berann, a master cartographer and panoramist, an average of six months to complete a
large panorama, while concurrently working on smaller projects [119]. An average ski
area map, a much smaller project than a large panorama, takes James Niehues [12] about
a month in quickly turned around jobs, if he is working on more than one job at a time.
Since panorama maps take artistic, as well as cartographic skill, there is a general
belief that panorama map paintings cannot be automated. For example, artists like
Niehues argue against computer generated maps because “when painted by hand the
paint brush can create different colors, texture and tone in one stroke and do it differently
each time, giving endless variety to the multitude of trees that go into each view. This
is the same with the shadows as well as the slopes themselves and the cliffs. This is just
Figure 1.1. Heinrich Berann’s Panorama Map of Yosemite National Park.
4
not possible with the computer” [12].
Furthermore, with current advancements in technology, one might wonder why bother
with the creation of panorama maps if one could just snap a picture of the place of
interest, from the viewpoint she cares about? Wouldn’t that be most realistic, most true
representation of the terrain environment? The short answer is no, not really. Aerial
photographs often have too much detail. Also, photographs are highly ambiguous. They
represent a polysemic sign system, where we have one-to-many and many-to-many relationships between the map (the sign vehicle) and the object (the referent). Graphic signs,
on the other hand, are unambiguous, monosemic sign systems, and they always have a
one-to-one relationship between those two categories [9]. A third reason is that, often,
panorama maps depict large areas, sometimes more than 800 kilometers in length. But
the maximum line of sight ever observed on the Earth’s surface is 370 kilometers [86].
No renderings resembling panorama maps can currently be generated via GIS-based
approaches. Panorama maps are significantly different from such renderings, because
the artist has manipulated the geometry, the surfaces, and the features of the landscape
in perceptually meaningful ways. The artist has generated detail and textures that are
not present in any of the original data. Such maps enhance the legibility and clarity of
the spatial information in very artistic ways, currently only produced by the hand of a
cartographer.
The automation of panorama maps, however, is highly desirable. For one thing, the
process is very slow. For another, such an automation opens up the panorama format
to many potential users. This would allow for maps to be produced and customized
to specific user needs, rather than the needs of major organizations like the U. S. Park
Service. Thirdly, such a system could propose practical approaches to solutions as well
as make reasonable advances towards addressing the complicated issues of map design.
1.3 Dissertation Overview
This dissertation argues that while the automation of the panorama map is very
difficult, it is feasible. In Chapter 3 we analyze the panorama map in terms of its
perceptual function and artistic form. Our analysis is performed on two distinct panorama
5
map paintings of Yellowstone National Park by stylistically different panorama artists,
Heinrich Berann and James Niehues. From that analysis, we derive a set of principles
applicable to the map design of mountain panoramas.
In Chapter 4 we use these map design principles to guide a practical implementation
of a proof-of-concept computer graphics system that generates images in the visual style
of the panorama map. The algorithms rely on shuttle radar topography mission (SRTM)
elevation data and national land cover database (NLCD) classification data. The system
provides the user with a full control over a set of practical parameters.
An important part of the automation of panorama map renderings is choosing a
meaningful viewpoint. In Chapter 5 we examine existing viewpoint selection methods,
and develop a set of criteria that allow us to rank the quality of the rendering of an
outdoor terrain scene from a specific viewpoint. By varying the variables that control
a selected viewpoint, we show how one can iteratively explore the viewpoint space and
automatically choose a rendering that is meaningful and representative of the landscape.
In our panorama map system, we encounter difficulties with choosing effective colors for the rendering of the panorama map. In Chapter 6 we address these issues by
introducing a novel color space, oRGB, based on opponent color theory. Since oRGB is
designed to work well with computational applications such as color transfer, it allows
us to more effectively adjust the color balance of our panorama renderings.
1.3.1
Contributions
Aside from contributing work towards analysis of the representation of the 3D aerial
panorama map, this dissertation increases the understanding of artistic and illustration
principles and their automation. It also contributes knowledge to the automation of
color choice and composition for rendering, as well as to nonphotorealistic rendering
and related computer graphics issues.
The specific contributions of this work are as follows:
• Analysis of panorama maps in terms of perceptual function and artistic form.
• Algorithms for image space nonphotorealistic rendering (NPR) textures specific to
terrain.
6
• Automatic viewpoint selection based on terrain features and user input.
• A novel color space useful for practical computational applications such as color
transfer for final color adjustment of a panorama map rendering.
• As a system, the contribution of methods, heuristics, and algorithms for panorama
map automation
1.3.2
Other Publications of This Work
Chapters 3 and 4 appear in part in the article ”Artistic Rendering of Mountainous Terrain” published by ACM Transactions on Graphics [14]. Sections of Chapter 5 have been
published as “Automatic Views of Natural Scenes” in the proceedings of Computational
Aesthetics in Graphics, Visualization, and Imaging [15]. Sections of Chaper 6 have been
published as oRGB: A Practical Opponent Color Space for Computer Graphics in IEEE
Computer Graphics and Applications [13].
CHAPTER 2
BACKGROUND
We now discuss the origin and development of the Mountain Panorama map, as well
as previous work in digital terrain and nonphotorealistic rendering.
2.1 The Mountain Panorama
While landscape paintings have existed since antiquity, they typically presented an
idealistic representation of the topography of the terrain. It was not until the late 18th
century that the advent of mass tourism in the Alps inspired the first serious bout of
mountain panoramic painting. There was such an expansion of the art form that by the
beginning of the 20th century hundreds of mountain panoramas were created.
Mountain panoramas fall into the category of 3D aerial view maps, also called bird’s
eye view maps. They are artistic cartographic renderings of extensive geographic areas. Panorama maps are significantly different from images produced by GIS, by photographs, or by direct rendering of the terrain. These maps are painted by artist cartographers who manipulate the geometry, enhance the features of interest, and add texture
details that are not present in the original data [86]. The cartographic and artistic techniques merge to produce an immersive landscape visualization that instills a strong sense
of natural realism, while helping us understand the terrain better. To our knowledge,
there are no existing algorithms that allow us to produce such paintings automatically.
As is the case with any artform, it is almost impossible to describe all the styles used
by panorama artists (see Figures 2.1, 2.2, and 2.3). Most commonly, panoramas painted
in water color do not use brushstrokes, and the colors are softer and more transparent.
On the other hand, the use of guache and tempera produces denser textures, and visible
texture strokes. In this work we will concentrate on examples of the latter kind, as they
contain more defined structural and perceptual properties.
8
(a) Gerald Eddy: Tahoe, California, USA [28].
(b) Pargeter: Northwest Passages, Washington, USA [85].
Figure 2.1. Panorama maps in summer palettes using watercolor.
9
(a) L. Koller: Ober Egadin, Switzerland [64].
(b) Unknown: Grachen, Switzerland [115].
Figure 2.2. Panorama maps in spring palettes using guache and oil.
10
(a) Unknown: Alpbach, Austria [116].
(b) Dieup Art: Flaine, France [5].
Figure 2.3. Winter panoramas.
11
2.1.1
The Process
The appeal of panorama maps is achieved through the use of various artistic and
cartographic techniques ranging from choice of color and detail, to choosing a viewpoint
that emphasizes as many of the recognizable features as possible [12, 83, 86].
The mountain panorama map design process can be generalized by the following
sequence of steps: feature identification, view estimation, geometry manipulation, projection techniques, and appearance rendering.
Feature identification deals with establishing features of importance on the future
panorama map. Recognizable peaks, ridges, visitor centers and lodges, major roads,
important rivers and lakes, and so on, are identified, marked, and taken into account.
During view estimation, a panoramist figures out the best viewpoint from which the
panorama should be rendered. The chosen view should visualize the scene in the most
meaningful and legible way. The goal is to represent each element (feature) in its relative
importance.
Terrain manipulation is needed terrain geometries have multiple faces, some facing
directly away from the viewing point. In such cases, elements may need to be rotated,
moved around, and even scaled, so that the recognizable landmarks are well represented
in the final rendering, though an effort is made to preserve the credibility of the spatial
relationships. In addition, certain recognizable landmarks, for example Denali, North
America’s highest peak, can be accentuated by selective vertical exaggeration.
Perspective projection is used to enhance realism, though typically stronger perspective is used on small-scale landscapes than on large-scale ones. The projection
plane is usually curved, so that visibility is enhanced. The nonlinear projection causes
the foreground and the middle ground to appear more map-like, while the background
appears more realistic, especially since the horizon and the sky play important part of the
background rendering.
Finally, panorama maps are commonly rendered in saturated, complementary primary hues. Artists often develop distinct summer and winter palettes. They generalize
the surfaces and manipulate the complexity of the underlying topography as well as the
surface textures representing vegetation, rocks, and so on. Water surfaces are represented
12
as shimmering water bodies; atmosphere and clouds are realistically depicted. Artists
paint illumination that emphasizes the legibility of the terrain. They establish sharp
contrast between the illuminated and the shadowed slopes by positioning the light source
perpendicular to major trends in the topography. Commonly, panorama map painters
also use cast shadows to enhance the natural realism, but they do vary their placement,
length, orientation, and intensity.
2.2 Digital Terrain Rendering
Algorithmic landscape sketching of terrain was first introduced by Visvalingam and
Dowson [120]. Their goal was to create sketches resembling artistic landscape drawings
by computing terrain profile curves - curves that are perpendicular and orthogonal to the
line of sight. A filtered set of points from these curves is then used to build a sketch of
profile strokes (p-strokes). Whelan and Visvalingam improved on the p-stroke sketch by
adding formulated silhouettes [125]. These are silhouettes that would be brought into
view if a higher viewpoint is selected. Lesage and Visvalingam [73] compute terrain
luminance maps and then extracts sketch lines using four common edge detectors. Most
recently, Buchin et al. [17] advanced a line sketching technique incorporating slope lines
and tonal variations.
There has been some research in the automation of map-making tools. A technique
for semiautomatic digital cliff drawings in topographic mountain maps [52] relies on
extensive interactive control by the cartographer and is based on ridgeline information.
Later work [51] no longer relies on edges and ridgelines, but instead covers continuous
cliff faces of equal slope and aspect with regularly spaced hachures in the horizontal or
vertical direction.
Saito and Takahashi [96] developed methods for the visualization of topographic
maps and provided an algorithm for producing an aerial view map that combined a
contour image, a profile image, and a shaded relief image. While the produced image is
an improved visualization of terrain over some GIS renderings, it is distinctly different
from panorama map paintings, as it does not contain perceptually meaningful surface
textures, nor does it enhance the features or generalizes the terrain geometry itself.
13
Premoze [90] proposed an interactive tool for panorama map creation. The editor
provides a 3D perspective view of the terrain drawn from a selected viewpoint, as well
as a 2D contour view. The user can perform selective vertical exaggeration and vertical
axis rotation. Orthoimage texture mapping can be draped over the terrain, and the user
can selectively paint over the terrain skin. Sky color and atmosphere effects can also
be specified. While the editor provides certain improvements over traditional terrain
visualization packages, it does not allow the resulting maps to have a stylized quality,
nor does it modify the geographical information in ways that improve its perceptual
legibility. Premoze’s tool also requires manual user input at each step.
2.3 Nonphotorealistic Rendering
Nonphotorealistic rendering (NPR) algorithms commonly produce stylized images,
rather than realistic renderings. Synthesized strokes play an important role in the final
rendering result, e.g., pen-and-ink drawing, sketch, or oil painting. Strokes follow visually meaningful directions and can be applied in image space or object space.
Image space approaches often use an existing image as input. By processing it, certain meaningful quantities are produced and later used to guide the rendering. Hertzmann
et al. [46] compute the normals of the image gradients from the image and then use them
as input. Some researchers manually specify the necessary vector fields [97], others rely
on information provided by the eye movement of human subjects [25]. Image space
algorithms tend to be simpler and usually more general. However, necessary information
may not always be correctly computed or even available from the image and can cause
problems for the rendering.
When a model of the scene is present, the stroke placement and direction can be
guided by geometric information. Examples include lines of principle directions of minimal and maximal curvature [30, 47, 56], lines preserving the angle between the surface
normal and the view direction (isophotes) [30], or plane-surface intersections [26]. While
such algorithms are more computationally intensive, they provide more flexibility since
one can extract specific and visually relevant geometric information.
In addition to strokes, nonphotorealistic shading models are also relevant to this work.
14
Specifically, the model of Gooch et al. [39] is built on practices of traditional technical illustration, and produces images that have better visual structure. More recently,
Rusinkiewicz et al. [95] proposed a nonphotorealistic shading model that is inspired by
cartographic relief techniques. Their solution is designed to convey fine-scale detail, as
well as overall geometry shape, regardless of surface orientation.
CHAPTER 3
ANALYSIS OF PANORAMA MAP DESIGN
Panorama maps are aerial view paintings of landscapes, commonly used to present
terrain information to nonexpert viewers [11, 27, 54]. Panoramas are painted by artists
with a strong affinity for cartography. The work involved is difficult, manual, and often
takes months at a time (depending on the size and detail of the panorama). Nowadays,
digital tools and topographic maps are often incorporated in the process, and have given
artists faster turnaround times. However, the process is still complex and time consuming, and it is commonly believed that only artists can create such intricate paintings
successfully.
We are interested in developing algorithms that would allow us to automatically
render images of terrain that have a similar visual quality to panorama map paintings.
While traditional cartography has developed many principles to guide the cartographer
in the making of traditional maps, there are no such conventions when it comes to 3D
cartography [43]. Regardless of professional background, people who currently create
aerial maps are by necessity self-taught [87].
3.1 Overview
Since there is very limited cartographic and artistic theory we can directly apply
to the automation of panorama, we examine two different panorama maps that depict
an identical terrain: Yellowstone National Park. The panoramas are painted by two
well-known and respected, yet stylistically different, panorama artists: Heinrich Berann
c 2009 Association for Computing Machinery, Inc. Reprinted by permission from ”Artistic Ren°
dering of Mountainous Terrain.” in ACM Transactions on Graphics 28, 4 (2009) with authors Margarita
Bratkova, Peter Shirley, and William B. Thompson. http://doi.acm.org/10.1145/1559755.1559759
16
and James Niehues. The reader can compare and contrast their panorama map painting
of Yellowstone side-to-side in Figure 3.1.
In order to understand the process of panorama map design, we analyze their paintings in terms of perceptual function and style, and derive heuristics and principles appropriate for landscape visualization of mountainous terrain. While we base our analysis on
only two examples, we feel they are different enough to extend to a range of styles.
Throughout this chapter, all supporting Yellowstone Panorama Map example closeups are from Berann to the left, and Niehues to the right. We limit our exploration to five
categories of terrain textures, the ones most commonly used in mountainous maps: trees
(evergreens and deciduous), cliffs, snow, lakes, and grass.
First, we provide an overview of pictorial cues and how they can be relevant to the
perception of depth in panorama maps. Then, we examine how the visual elements used
by the two artists act as pictorial cues that can be detected by the visual system of the
observer and help her perceive accurate surface shape and orientation. We also explore
how the visual elements serve a stylistic role and allow the artists to make their paintings
subjectively expressive and pleasing. Finally, we summarize a list of principles and
heuristics that can be used in the automation of panorama map rendering.
3.2 Visual Perception and Pictorial Cues
Maps visually convey a sense of the spatial structure of the terrain, as well as the
spatial context of natural and cultural features located on the terrain. In perceptual terms,
this means conveying information about distances and surface orientations [38, 79].
When considering visual perception, distance information is often divided into depth
(or egocentric distance) and exocentric distance. Depth is the distance between the
viewer and a visible location in the environment, while exocentric distance is the interval
between two arbitrary environmental locations, expressed in an environmental frame of
reference. Conventional plan view (overhead view) maps explicitly represent exocentric
distance. Some plan view maps augment this with an implicit indication of surface
orientation using contour lines or shading.
17
(a) Heinrich Berann
(b) James Niehues
Figure 3.1. Panorama Map Paintings of Yellowstone National Park by Heinrich Berann
c
and James Niehues. °Trail
Tracks, used with permission.
18
Unlike standard top-view cartographic maps, landscape paintings such as panorama
maps create a sense of directly viewing three-dimensional terrain. They are dominated by
perceptual information for depth and orientation. The depth and orientation information
in panorama maps is conveyed through pictorial cues, which do not involve binocular
stereo, ocular information, or motion. Pictorial information arises due to perspective
scaling, foreshortening, interposition (occlusion), intrinsic surface markings [62, 109],
and light transport [62, 74, 84]. Figure 3.2 lists the pictorial cues most relevant to the
perception of depth and surface orientation.
3.3 Perceptual Elements in Panorama Map Paintings
For the purposes of panorama map paintings, the potentially most important pictorial
cues are texture element distributions affected by foreshortening, the partial occlusion
of the texture elements in forested areas (though not the variation in the amount of
occlusion, as used by Leung and Malik [74]), surface contours [108], shading, and
silhouetted terrain features. Panorama maps covering more extensive areas could use
all of these cues, together with position relative to the horizon, relative and familiar size,
texture gradients, and aerial perspective.
Pictorial Cue
Convergency of Parallel Lines
Position Relative to Horizon
Relative Size
Absolute Size
Texture Gradients
Projection of 3D Texture Elements
Edge Interpretation
Surface Contours
Shading
Aerial Perspective
P
F
I
S
L
Figure 3.2. Pictorial cues for depth and surface orientation arising from perspective
scaling (P), foreshortening (F), interposition (I), intrinsic surface markings (S), and light
transport (L).
19
3.3.1
Texture Gradients
Panorama maps of mountainous terrain typically represent outdoor areas in which
forest is common. The three-dimensional nature of forests results in an appearance of
forest textures in projected images that is quite distinct from textures generated by surface
markings [108]. Forests are unusual in that the elements that form them (the trees) are not
tangential to the surface. The image plane projection of forests forms 3D image textures.
Such textures have the potential to provide the human visual system with a number of
distinct perceptual cues, because the projection of the texture elements (texels) to the
image plane varies based on distance and surface orientation.
Shape-from-texture theory suggests one can use the texture distortions present in
the image due to the perspective projection from 3D to recover surface shape and orientation [36, 38, 78, 93, 126]. Such projective distortions can affect the texel shape
(compression or foreshortening, e.g., a circle projects into an ellipse), the texel size
(perspective scaling), and the texel density of texel elements.
Cutting and Millard [23] examine the perception of slant using these three texture
gradients, for flat as well as curved surfaces. They find that for flat surfaces, the perspective gradient accounts for 65% of the variability, while for curved surfaces, it is the
compression gradient that accounts for 96% of the variability, as perceived by humans.
Since most mountainous terrain is not flat, it seems that, of all texture gradients, it is the
compression gradient of texels that should be of utmost importance for this work.
In addition to these three gradients, 3D textures also introduce a fourth cue - texel
occlusion. We examine these four cues and develop guidelines we believe are specifically
applicable to the visualization of forested textures. Our goal is to generate forest textures
that provide as much functional 3D information in terms of perceptual cues as possible.
Before we proceed with the analysis, it is important to understand assumptions that
exist about the 3D surface distribution of texels in the shape-from-texture algorithms.
There are two main assumptions - homogeneity and isotropy. The assumption of homogeneity requires that the texture element surface density of the texels be nearly constant [1, 38, 113]. Isotropy, a weaker assumption, expects that the texture does not have
a dominant direction or an orientation bias [36, 126]. Perceptual studies suggest the
20
visual system uses both assumptions [93]. However, since isotropy cannot deal with
directional textures [78] (such as 3D forest textures), we use the stronger assumption of
homogeneity. Homogeneous texel density is a fairly reasonable assumption, especially
when one deals with natural textures. We now proceed with the analysis of panorama
map paintings.
3.3.1.1
Texel Shape
Texel shape foreshortening in 3D textures has been mostly unexplored in the shapefrom-texture literature. Since the shape of tree texels does not change much in the
projection to the image plane, we believe that in this context foreshortening probably
does not serve a useful purpose, as the small shape variations would be hard to detect.
3.3.1.2
Texel Size
True perspective projection dictates that texel size of elements vary with distance.
However, since panorama maps often represent large distances, we believe that in doing
so, texture elements in the distance will become too small to serve a visual purpose or
to fit the resolution of the painting. Instead, we propose that texel size variations should
be based on the extent of the visualized terrain. The chosen perspective scaling should
guarantee that texture elements present even in the far distance are discernible in the final
rendering. Both panorama artists seem to follow this rule (see Figure 3.3).
3.3.1.3
Texel Density
The projection distortion of texel density is a function of both perspective scaling and
foreshortening [109]. The effects of scaling and foreshortening can be partially separated
under a first order homogeneous assumption (i.e., the texture pattern has a constant area
or density). Under this assumption, Aloimonos [1] showed that density foreshortening
can be used in the recovering of surface orientation. Malik and Rosenholtz provide
evidence that even under the stronger homogeneity assumption (i.e., second and higher
order statistics are translation-invariant), density foreshortening can be used to recover
surface orientation [78].
If we are to benefit from visual information present due to the projection distortion,
21
Figure 3.3. Trees in the distance (top) and foreground (bottom). If size of trees varied
with true perspective, pixel size of trees in the background would be a small fraction
of their foreground size. Clearly this is untrue, especially in Berrann’s painting (left).
c
°Trail
Tracks, used with permission.
we must ensure our texels are homogeneously distributed on the 3D surface. This argues
either for an object-based rendering approach or for an image space approach that takes
into account certain 3D information so that, as far as it is practical, homogeneity is
reasonably approximated. In addition, we believe that texel distributions should be
further constrained, so as to imply additional structural information, as we believe is
done in landscape paintings via the application of tree strokes in the direction of fall
lines. We address this issue when we explore surface contours.
3.3.1.4
Texel Occlusion
When projecting 3D elements to the image plane for certain viewpoints, texel occlusions [74, 98] will occur. The occluded element is visually perceived as positioned
behind the occluding element. The visual overlap is a very strong relative depth cue.
This occlusion never happens for 2D textures. However, it often occurs in forest textures
incorporated in landscape visualizations. Therefore, it is important that rendered texture
elements align such that they partially occlude each other, especially for steeper slopes,
where such occlusions are more likely to naturally occur. In agreement with perceptual
theory, panorama artists use occlusion in their depiction of forests (see Figure 3.4).
22
Figure 3.4. Tree strokes in forest textures are laid down so that they partially overlap
each other. The visual system interprets that to mean that the stroke that overlaps is
c
physically nearer. °Trail
Tracks, used with permission.
3.3.2
Surface Contours
Surface contours were first introduced by Stevens [109] and arise from the projection
of extended surface markings to the image plane. Such surface markings are formed by
a variety of physical processes, such as erosion, etc. Stevens suggests surface contours
help us perceive shape, because the visual system attributes a strict geometric relationship to the surface contours and the curvature of its underlying surface. Knill believes
surface contours help us perceive shape, because people have prior assumptions about
the constraints present in such surface markings [62].
In this work, we are interested in the ways surface contours aid landscape visualization. Artistic line drawings of terrain shape often incorporate fall-lines (sometimes called
slope lines), the lines drawn “downhill” the path of steepest decent [17, 55]. They provide
much information about terrain shape, as seen in the line integral convolution [18] image
in Figure 3.5, and are believed to describe the essential structure of relief [63].
Figure 3.5. This rendering is a visualization of the flow along fall lines using line integral
convolution.
23
Considering how vital fall lines are to relief, it is not surprising that panorama artists
consistently use them as a directional guide for stroke placement in the painting of cliffs
and other mountainous forms (see Figure 3.6, bottom). In addition, the directions of fall
lines is commonly used as imaginary lines along which to place vertical tree strokes,
spacing them according to slope (see Figure 3.6, top). We refer to such strokes as
meta-strokes. We believe that when artists incorporate fall lines in their paintings, they
combine surface contours with texture cues for surface orientation.
3.3.3
Shading
Shading is vital in depicting terrain surface form [48] and is an effective perceptual
depth cue. Imhof emphasizes the importance of using midtone to help the light and dark
areas express the form of the terrain [55]. It is commonly recommended that the tone
of shading gradation covers the spectrum of value - light, medium, and dark, because by
doing so, a painting helps the viewer perceive the form better [24, 112]. We note the
difference between standard rendering practices where continuous shading is the norm,
and painting, where artists explicitly blend palettes of discrete colors.
While it is known that shape-from-shading is an ill-posed problem, in that the same
image can be produced by a family of surfaces [16], it is an effective and very useful
depth cue for people [84]. And indeed, for centuries artists have used light and shade to
create impressive pictorial representations of three-dimensional shapes.
Figure 3.6. Tree strokes are placed along the imaginary fall lines of the geometry (top
images). Cliffs and snow are painted with strokes that follow the direction of fall lines
c
(bottom). °Trail
Tracks, used with permission.
24
Light direction plays an important role in successful landscape visualization. Imhof
[55] recommends that light comes diagonally and at an acute angle to the more dominant direction of valleys and mountain ranges. It is also important to use light coming
angularly from the front, so as to achieve a good balance between the light and shade
of the terrain in the image [43]. Panorama artists also seem to follow these rules (see
Figure 3.7).
3.3.4
Silhouettes
Silhouettes are the contours formed at the boundary of an object and its background.
They are perceptually useful as they indicate edges where depth changes. Such edges
result from an image-based spatial discontinuity and signal real-world distance between
the surfaces separated by the silhouette curve. Silhouette edges are especially appropriate for expansive mountainous terrain, where the landscape commonly overlaps in the
distance (e.g., rolling hills). It is therefore not surprising that artists commonly use that
cue to increase the perception of depth in their paintings (see Figure 3.8 for an example
of the use of silhouettes).
3.4 Stylistic Elements in Panorama Map Paintings
We now analyze the basic stylistic variations present in the two panorama maps of
Yellowstone National Park we have been examining so far. We observe that the physical
act of painting landscapes is achieved by layering color shapes and brush elements that
vary in color and size depending on the natural elements portrayed [86].
We believe our stylistic analysis should examine variations in three basic parameters
- base color, brushstroke color, and brushstroke shape. The base color provides the basic
Figure 3.7. Light direction is diagonal to the dominant direction of valleys and mountain
c
ranges. °Trail
Tracks, used with permission.
25
Figure 3.8. Silhouetted curves are used to separate an object from its background and
c
are used in panorama maps to imply depth. °Trail
Tracks, used with permission.
shading of the terrain, and varies with classification type, while the brushstroke color and
brushstroke shape create the visual appearance of the texture elements.
3.4.1
Base Colors
The base color for all texture regions is chosen such that its interaction with the tree
strokes provides enough contrast and makes the brushstrokes stand out. That usually
implies that the colors tend to be darker and often less saturated. The base color also
varies with surface orientation, as is to be expected. Colors in the light tend to be brighter
and warmer in tone, while colors in the shade, darker and colder in tone. Figure 3.9
provides samples for light, medium, and dark tone in lighted, as well as shaded areas,
for both panoramas. We observe that background colors get desaturated and often have
a blue shift, as would be the case when the effects of Aerial Perspective are taken into
effect.
3.4.2
Brushstroke Colors
Brushstroke colors play an important role, as they add vibrancy to the rendered
image. As expected, they change with respect to the orientation of the surface towards
the light (see Figure 3.9). Niehues tends to vary mostly the value, while Berann also
adds a variation in hue. A good distribution of strokes in light, medium, and dark tones
achieves a stronger articulation of the form. Both artists add brushstrokes for areas that
represent forest, snow, cliffs, or water. They do not seem to use strokes for grassy areas.
Niehues uses brighter and less natural colors. The colors he chooses for areas in shade
have a strong blue shift, both in the foreground and in the background. Berann uses more
realistic tones. The brushstroke colors are more natural in hue and more diverse in value.
26
F O R E G R O U N D
Area in light Area in shade
L M D
L M D
(a) Berann Evergreen Trees
Base
Stroke
(b) Niehues Evergreen Trees
Base
Stroke
(c) Berann Deciduous Trees
Base
Stroke
(d) Niehues Deciduous Trees
Base
Stroke
(e) Berann Cliffs
Stroke
(f) Neihues Cliffs
Stroke
(g) Berann Snow
Stroke
(h) Niehues Snow
Stroke
(i) Berann Grass
Base
(j) Niehues Grass
Base
B A C K G R O U N D
Area in light Area in shade
L M D L M D
Figure 3.9. Sample of light (L), medium (M) and dark (D) colors used by Niehues and
Berann in their Yellowstone painting for areas in the foreground and the background.
The colors were sampled from a digitized version of their paintings using Photoshop.
Snow and cliffs is mostly painted by the use of strokes, and the base color does not come
through much.
Water textures use base colors that vary with viewing angle. Berann uses the brushstroke color to emphasize the viewing angle even further. In Niehues’ work, the base
color varies little with viewing angle, and the stroke colors mimic the effect of physical
water reflection.
3.4.3
Brushstroke Shapes
Niehues varies the way he paints brushstroke shapes for trees, depending on the type
painted - evergreen vs deciduous. Berann does not seem to make such a distinction.
The brushstroke shapes for trees are very simple, elongated, perhaps a little bit fatter at
the bottom, and orientated in the vertical direction. Figure 3.10 provides a sample of
evergreen and deciduous brush strokes from both artists.
Berann’s cliff and snow strokes are applied in various scales - small, long and distinct, or applied sweepingly and semitransparently over large areas so as to unify them.
Niehues uses much simpler, uniform brushstroke lines. They seem to alternate between
27
(a)
(b)
Figure 3.10. Sample of the shapes for tree strokes used by (a) Niehues and (b) Berann.
For each artist, the two samples to the left correspond to strokes used for evergreen trees,
while the two to the right - for deciduous trees.
chosen colors, following the direction of fall lines.
Water strokes seem to be very simple and applied horizontally to the image plane.
3.5 Summary of Principles and Heuristics for Panorama
Map Design
In the hope of reverse-engineering conventions used by panorama artists, we analyzed
the visual elements of two panorama map landscape paintings in terms of perceptual
function and style. Here is a summary of our proposed heuristics and principles:
• The image texel size of surface textures that represent 3D elements (e.g., forest)
should vary with distance, but should not match true perspective (texel size in
Section 3.3.1.2, Texture Gradients).
• The image space distribution of texel elements of 3D textures (e.g., forest) should
mimic one that would result from the projection of homogeneously distributed
surface elements (texel density in Section 3.3.1.3, Texture Gradients).
• Image space texel spacing of 3D textures should ensure that texels overlap, especially in steep areas (texel occlusion in Section 3.3.1.4, Texture Gradients).
• Fall lines follow essential structures of terrain. They act as surface contours,
and are used by panorama artists to paint cliff and snow textures (fall lines in
Section 3.3.2, Surface Contours).
• Fall lines are used as imaginary lines along which tree strokes are placed, acting as
texture meta-strokes (meta-strokes in Section 3.3.2, Surface Contours).
28
• Shading tone should have a good distribution of light, medium, and dark values
(shading in Section 3.3.3, Shading).
• Light position should be placed so that the rendering of the terrain exhibits a good
balance of light and shade, as seen from the selected viewpoint (light direction in
Section 3.3.3, Shading).
• For extended terrain areas, indicating silhouettes, especially between occluding
hills, is useful (silhouettes in Section 3.3.4, Silhouettes).
• Water surfaces should reflect the environment (water textures in Section 3.4.2,
Brushstroke Colors).
Another common heuristic from cartography is vertical exaggeration. It is often used
when subtle topographic features need to be made more prominent, or when the terrain
covers large areas and features get lost due to scale - as would be the case for panorama
maps. Exaggeration scales often vary with the type of depicted terrain, but scales used
by cartographers typically range from 1:1.5 to 1:5.
CHAPTER 4
AUTOMATED PANORAMA MAP RENDERING
In this chapter we present an automatic computer graphics rendering system for
Panorama Map Paintings that incorporates the principles and heuristics we developed
in Chapter 3 and applies them to a given terrain heightfield and its corresponding classification image. Our panorama map system creates renderings that incorporate surface
textures of natural surfaces. Specifically, we produce images of mountainous terrain that
have cliffs, forests, snow, and lakes. Our textures are applied in image space and our algorithms are influenced by structural information present in the terrain, perceptual metrics,
and certain artistic considerations. Refer to Figure 4.1 for a rendering of Yellowstone
National Park produced by our system, in the style of Heinrich Berann.
4.1 Goals and Overview
The goal of this work is to develop methods for meaningful, automatic landscape
visualization, with an emphasis on natural landscape panoramas. It is important to
note that map design and cartographic representation require that complex cognitive
issues be addressed. Full automation does not allow user input, and lacks flexibility
and the ability to correct unclear renderings. If the algorithms used are not successful in
assigning clarity, legibility, appropriate colors, hierarchical structure, patterns, balance,
or appropriate typography, the produced map could exhibit poor design. We advocate
the view that an automated system is only useful when its default settings produce
meaningful results. However, we also provide a full set of basis parameters that the
c 2009 Association for Computing Machinery, Inc. Reprinted by permission from ”Artistic Ren°
dering of Mountainous Terrain.” in ACM Transactions on Graphics 28, 4 (2009) with authors Margarita
Bratkova, Peter Shirley, and William B. Thompson. http://doi.acm.org/10.1145/1559755.1559759
30
Figure 4.1. A panorama map rendering produced by our system for Yellowstone
National Park, using Berann-style brushstrokes and colors. Refer to Figure 3.1 for the
original Berann painting.
user can control, so that she can achieve a reasonable level of customization of the final
product if so desired.
In this chapter we present a set of algorithms that produce images in the style of the
panorama map. We concentrate on elements present in mountainous terrain. We do not
address geometric feature enhancements, though such techniques are very valuable, and
are used by panorama artists [86]. Our algorithm proceeds in three stages: preliminary
setup, base shading, and surface textures. Figure 4.2 provides an overview of the steps
taken by our algorithm.
4.2 Preliminary Setup Stage
4.2.1
Terrain Geometry and Classification
The geometry of the terrain is specified by Shuttle Radar Topography Mission (SRTM)
elevation data. The data represent a heightfield and are stored in a floating point format.
31
HF
Classification Image
Curve Geometry
Build Geometry
Triangulated Terrain Surface
Raytrace
Terrain Image
Smooth Classification
Boundaries
PRELIMINARY SETUP
Stores per-vertex:
3D Normal
3D Gradient
Stores per-triangle:
Classification type
Stores per-pixel:
3D Normal
2D Gradient
Classification type
Depth
Luminance
Light/Shade mask
BASE SHADING
Cliffs
Snow
Evergreen
Deciduous
Grass
Luminance Histogram Equalization for Light & Shade
Apply desired light-middle-dark colors for light and
shade based on LHE luminance values
Base Cliff
Base Snow
Base
Evergreen
Base
Deciduous
Base Water
Base Grass
SURFACE TEXTURES
Fall Line Contour Generator
Line Generator
Seed Generator
Brush
Images
Brush
Images
Tree Strokes
Generator
Stroke Line
Generator
Reflection Strokes
Generator
Composite Final Image
Figure 4.2. An overview of the steps taken by our algorithm. It can be grouped in three
stages: preliminary setup, base shading, and surface textures.
32
For the purposes or rendering, the heightfield is converted into a regular triangulated
mesh.
We also read-in a classification image that corresponds to the terrain data. It is based
on the National Land Cover Database (NLCD) land cover data compiled by the MultiResolution Land Characteristics (MRLC) Consortium, freely available from the U.S.
Geological Survey (USGS).
4.2.2
Vertical Exaggeration
Since panoramas depict terrain that covers large areas, vertical features are often
lost in the vast horizontal extent of the geometry. Cartographers address this problem
by increasing the vertical scale of the geometry in relation to the horizontal scale, thus
improving the recognition of such features. This is a common visualization technique,
and is referred to as “vertical exaggeration”.
We add exaggeration in a similar manner by allowing users to specify two exaggeration parameters: one for lowest elevation terrain and another for the highest elevation
(peaks). For a given elevation value, we determine the exaggeration factor by linearly
blending in between the two exaggeration factors given. This allows for control over
both exaggeration in small scale details such as small hills (that would otherwise be
unnoticeable) and peaks independently of each other. A more crude constant multiplier
for the full terrain tends to exaggerate the geometry unacceptably.
4.2.3
First-Order Terrain Derivatives
A heightfield is a parametric surface of two variables, S(u, v). Let f (u, v) be the
elevation of the heightfield at a given parametric location P = hx(u, v), y(u, v), z(u, v)i.
In the simplest mapping, x(u, v) = u, y(u, v) = v, and z(u, v) = f (u, v). However, more
complex mappings are allowable: the surface can be transformed in any manner, yet
will remain a parametric surface of two variables. In parametric space, ∇ f points in the
direction of maximum elevation change. −∇ f is thus the fall line vector that we need to
guide our texture placement. However, we desire the fall line in cartesian space instead
of parametric space so that we may eventually project that value into image space.
To compute the fall line in cartesian space, we first need to determine the ∇ f vector
33
in cartesian space, and then negate it. Starting at (u, v) in parametric space and moving
along the surface in the direction ∇ f brings us to (u + ∂∂ uf , v + ∂∂ vf ). It is this new parametric location that corresponds to an increase in the height function. The corresponding
point on the surface S in cartesian space is reached by starting at S(u, v) and moving by
∂f
∂u
∂f
∂v
∂ f ∂P
∂u ∂u
along the first tangent ( ∂∂ Pu ) and
parametric space corresponds to
along the second tangent ( ∂∂ Pv ). Therefore, ∇ f in
+ ∂∂ vf ∂∂ Pv in cartesian space. Conveniently, we also
need these tangents to compute the normal to the parametric surface: N =
∂P
∂u
× ∂∂ Pv .
To compute surface derivatives on the terrain in either the u or v parametric directions,
we use central differences when possible. On the boundaries, we use either forward or
backward differences as appropriate.
4.2.4
Geometry Curving
Viewpoint selection affects the final rendered image. Selecting a low elevation view
point produces an image that includes the horizon. However, features in the foreground
occlude features in the background. Selecting a high elevation point generates an image
where we see larger parts of the terrain. However, the view, in the extreme case, resembles the traditional two-dimensional cartographic map and the image loses its 3D spatial
quality.
Since visibility of both the horizon and the terrain features is desirable, some artists
like Heinrich Berann emulate the view from a high-elevation viewpoint and curve the
base of the terrain [86]. This makes it possible to see both nonobscured features in the
foreground as well as the horizon and sky in the distance.
To provide a similar effect, we transform the original terrain surface into a curved
one, so that rendering it now combines multiple perspectives into one. The geometry in
the background is from a low elevation view point, while the geometry in the foreground
appears as seen from a high elevation viewpoint (refer to Figure 4.3).
We compute the curved surface as follows. First, we curve the hypothetical base
plane of the original terrain (at sea level) into a curved base plane via a power function
y = a ∗ x2 , for the given view. The arc length of the curved plane in the direction of
view is kept the same as the distance between the nearest and furthest points on the
hypothetical base plane of the terrain, d, in the direction of view, to maintain the original
34
original base plane
view direction
image plane
curved base plane
Figure 4.3. When the base of the terrain is curved, the image will include both
nonobscured features in the foreground, as well as horizon and sky in the distance.
(u, v) parametrization. The scalar value a is determined by a = b/d, where b is the
desired vertical offset. x stands for the value representing the distance along the line of
sight to the point on the grid we are currently curving, normalized to [0, 1].
To find the new 3D curved position for each heightfield grid point, we first find the
3D position on the curved base surface that corresponds to the same (u, v). The new
position is then an offset h = f (u, v) in the normal direction to the curved plane at the
(u, v) location. The new offset surface places all the geometry within view without selfintersection or obvious foreground occlusion.
Modifying the base plane of the geometry requires computing new surface tangents
(as the mapping from u, v to 3D has changed) and updating the gradients and normals
accordingly. As pointed out previously, the only requirement is that the surface is still a
parametric surface of two variables; the mapping from (u, v) to points in cartesian space
is much more complicated now, but the fall lines are still expressed in the same manner.
For a comparison among leaving the geometry as is, geometry curving, and moving
the view point to a higher elevation, refer to Figure 4.4. Since rotating the geometry is
equivalent to rotating the view, we keep the view the same for all the comparisons. We
simulate the view from higher elevation by instead rotating the geometry.
4.2.5
Preliminary Terrain Image
We use a raytracer to render an image of the triangulated surface model for a specified
light position (single distant point light) and viewing direction. We treat the terrain
surface as perfectly diffuse. If the pixel is in light (N · L > 0), we use simple direct
35
Figure 4.4. Comparison of the same terrain rendered from the same viewpoint and
camera parameters, but with different geometry modifications: leaving the geometry
as is (top), our geometry curving method (middle), rotating the geometry (instead of
rotating the view) to show corresponding view from above (bottom). Geometry curving
preserves the 3D properties of the geometry, while allowing us to see both the horizon
and the terrain features of interest, that would have otherwise been lost due to occlusion.
36
lighting as our resulting luminance value. Otherwise, we compute a 1 bounce diffuse
path tracing solution with 25 samples to lighten up the otherwise perfectly black pixels.
If we hit a triangle that is classified as water, we send reflection rays to determine the
values of the reflected point. Because of the curved geometry, we find that we need to
lower the reflected ray (by increasing the angle between the reflected ray and the normal
by 10-15 degrees) so that the reflection values match our visual expectations.
For each pixel in the rendered image, we store six variables: luminance, normal, 3D
gradient projected into 2D screen space, depth, classification type and the light/shade
mask. This results in 33 bytes per pixel assuming single precision floating point values.
4.2.6
Classification Boundaries
As a result of the rendering projection, the boundaries of the classification types in
image space are not smooth, especially if the resolution of the terrain is lower than the
resolution of the final rendering. We find that smoothing these boundaries improves the
visual quality of the final rendered image.
We smooth the boundaries by treating each pair of bordering classification types
independently. We assign 0.0 to the first type, 1.0 to the second, and then run an averaging
filter. All pixels with value below 0.5 get assigned to the first classification type. All
pixels with value above 0.5 get assigned to the second classification type. We repeat
until all pairs of bordering classifications are smooth.
4.3 Base Shading Stage
In our analysis, we concluded that the shading tone should have an even distribution
of light, medium, and dark values. This step ensures that we extend the luminance value
ranges in a manner independent of the brightness of the light source or the original image
statistics of the raytraced image. In effect we abstract the terrain shading so it has the
minimum amount of detail, while we retain important geometric and shape information.
Refer to Figure 4.5 for a rendering of Yellowstone National Park using only Berann base
colors.
Based on this convention, we now describe a method for terrain base shading. First,
we label individual pixels as being one of three tonal types - Light, Medium, or Dark. Our
37
Figure 4.5. In this rendering of Yellowstone National Park, textures are using only the
base Berann colors.
algorithm relies on luminance, classification type, and light/shade mask. To determine
the light/medium/dark choice, we take all pixels for a given classification and mask types,
and compute a three-bin histogram equalization. This step ensures that we have an equal
distribution of light, medium and dark tones.
Then, we compute the color for each pixel in the scene. Per classification type, the
user specifies 12 base shading palette colors corresponding to the combinations of (Light
/ Medium / Dark Tones) x (In Shade / In Light Masks) x (Near / Far Variations). For a
tonal type and mask type, we blend between the Near color and the Far color given by
the palette. The blend factor is the fraction computed by dividing the depth stored in the
current pixel by the maximum depth of the terrain for the given viewpoint. The blend
factor also determines how much aerial effects should be added to the pixel. Colors in
the far distance have an added bluish cast and are further desaturated to match the effect
of Aerial Perspective. Finally, we composite a sky image underneath our base shading.
38
4.4 Fall Lines
All classification types in the panorama paintings except for the grass type have
additional textures applied on top of their base shading. How should such textures be
created automatically?
As discussed in Chapter 3, fall lines are an appropriate guiding direction for applying
painterly strokes or for the placement of painterly textures. Figure 4.6 compares the
placement of tree textures along fall lines to placing them via jittered sampling or random
sampling. We note that fall lines are clearly superior at communicating geometric and
perceptual properties.
Since not all classification types require surface textures (e.g. grass), and since we
would like to give the user control over the length of the texture strokes, we build fall
line guided paths for each classification type independently.
For each pixel in the image, pi , we store the image-space negative gradient of the
visible geometry, vi = −∇ f . From this discrete vector field V , where vi ∈ V , we need
to reconstruct a set C of smooth image-space curves that follow these gradients (the fall
lines projected in image-space). ci ∈ C, is the path starting at pixel pi , with desired length
Figure 4.6. Our method of tree distribution uses placement of trees along fall lines
(center). As a comparison, you can see the same region with trees placed using jittered
sampling (left) and random sampling (right).
39
l, and has the same classification type as pi .
Each curve is the solution to the initial value problem, specified as follows:
dci
= g(t, ci (t)) = −∇ f ,
dt
ci (t0 ) = pi .
To compute ci we must solve this differential equation, and can do so with any
method. In our implementation, we use a 4th Order Runge-Kutta method for its accuracy
and stability.
After building all the paths for our image, we are likely to have far too many paths.
More importantly, many of these paths will converge and diverge so that many paths
go through the same pixels. These are unnecessary as only one final set of strokes can
actually be drawn within a single pixel. To prune the resulting paths, we iterate over all
pixels that contain more than one path and only allow the longest one to continue. The
other paths are clipped to stop before the current pixel. This keeps longer, flowing paths
instead of many short paths (see Figure 4.7).
Because of numerical precision issues with projecting into integer pixels, paths may
go through the same pixel twice. We detect these loop cases and trim the paths accordingly. Once done with this step, we have unique paths that run through every pixel in
each classification region.
4.5 Surface Textures Stage
Once we have painted all the base colors, we need to place the visual elements in
order to enhance the panorama perceptually and stylistically. Each classification type is
handled individually, as the visual elements are different. To examine the effect of stroke
Figure 4.7. Paths before (left) and after (right) pruning.
40
textures, compare Figure 4.5, where we only have base shading, to Figure 4.1 where we
have the complete rendering.
4.5.1
Forest Textures
Since forests are 3D textures formed by fairly regular 3D texture elements, we try
to maximize their perceptual effects by following the rules we developed about image
space texel size, texel distributions, and texel overlap (see Section 3.3.1). We make their
effects even stronger, by combining the texture cues with surface contour directions,
forming forest meta-strokes.
As mentioned in Chapter 3, we would like to place our tree strokes so that they are
reasonably spaced, follow the fall lines of the terrain, and give the illusion of occlusion.
We choose to place our trees by first computing “seed” positions in image space by
walking along the fall lines by an amount determined by the slope of the surface projected
into image space. Projecting the slope values is a useful approximation of texel spacing
foreshortening. To keep tree spacing reasonable, we also mark pixels surrounding a
chosen seed point as unavailable (see Algorithm 1). We use the longest paths first to
ensure that the most important fall lines are most visible.
Algorithm 1 PlaceTreeSeeds (paths, treeSpacing, radius)
mark pixels ∈ FOREST as AVAILABLE
for all paths ∈ FOREST do
for all currPath ∈ paths, starting with max length do
length = 0
while length < max path length of currPath do
pixel = currPath[length]
if CircleAvailable (pixel) then
mark pixel as SEED
mark circle of radius pixels around pixel as USED
else
length += 1
continue
stride = treeSpacing
stride *= Dot (-viewDir, surfNormal[pixel])
length += stride
41
Once all the seed positions have been chosen, we paint the actual tree strokes in a
“back-to-front” order. Using back-to-front ordering provides the illusion of 3D occlusion. Our tree brushes are defined as alpha values and we pick the per pixel brush color
by multiplying each of the brush alpha values by the desired color.
The desired stroke color is determined by the (Light / Medium / Dark Tones) x (In
Shade / In Light Masks) x (Near / Far Variations) combination. This is very similar to
what we did for the Base Shading Stage, except now the chosen color corresponds to
surface brushstrokes, not background. To add additional variety to the colors (as Berann
does in his strokes), we stochastically choose a blending factor (between 0% and 30%
percent) and use it to blend between the chosen color and the next darkest tree color. This
forms the final color for each tree stroke.
To allow the user control of the perspective scaling of brushes, we provide a single
scale factor that describes how small texels in the farthest distance should be relative to
the original brush size. At points in between the nearest point and the farthest point, we
use the relative distance (dist/maxdist) to scale the user parameter.
4.5.2
Cliff and Snow Textures
Surface textures for cliff and snow enhance the perceived terrain structure. As discussed in Chapter 3, fall lines follow the essential structure of terrain. Therefore, using
the fall line paths as imaginary lines along which brush strokes for cliff and snow are
placed will make that geometry more structurally expressive.
The application of strokes is made independently in each of the classification regions
(snow and cliff). The user has control over four parameters - stroke color, stroke width,
stroke overlap, and stroke image definition.
Similarly to forest strokes, the user provides 12 stroke colors per brushstroke type
(cliff or snow). Again, the tonal classification and the mask type indicate the appropriate
color for Near and Far. The two palette colors are blended using a weight related to
the relative depth of the pixel we are shading, to form the selected color. Color variety
is incorporated by blending the selected color and its closest darker color (here blend
weights vary from 0% to 50%). The most significant difference for these strokes is that
the color selection is made based on the luminance segmentation for the starting point
42
(origin) of the path, and not on the current position along the path, as the path’s tonal
classification can change. This is necessary to maintain the visual integrity of individual
strokes.
Stroke overlap and stroke width determine the visual density of strokes. Overlap specifies the number of pixels we allow strokes to overlap on. Width is randomly selected,
per stroke, from the set of widths specified by the user. The less dense the strokes are, the
more the background will come through. We use the width and overlap settings to resize
the stroke footprint for each point along the stroke path. If the current stroke footprint is
still unoccupied in the image, we mark it as occupied, and indicate that we should draw
part of the stroke there. If it is already occupied, we move on to the next path.
The stroke image definitions are similar to the skeletal strokes of Hsu et al. [49]. For
each stroke, we read the brush stroke image definition and resample it to match the width
of our desired stroke and the length of our currently selected path. We then walk along
each pixel that makes the current path. When the footprint is marked as available for
drawing we alpha blend the selected brushstroke color with the base shading, based on
the weights provided in the brushstroke image.
4.5.3
Water
Water surfaces reflect light from its surroundings, providing the observer with a
distorted and muted image of the terrain as well as the sky. Such reflections undoubtedly
contribute to the appeal and natural appearance of the water body. Water reflections were
also painted in both panorama maps we analyzed in Chapter 3.
In order to simulate surface variation, we adopt Berann’s approach of adding strokes
horizontal to the image plane (refer to section 3.4.2). We render textures for water
surfaces by generating strokes of varying lengths in the horizontal direction. Based on
the luminance of the base water color, we pick a stroke color that is a blend between the
lightest and darkest color of the sky. To add some variation to the brush appearance, for
each brush, we modify the luminance value of the brush pixels by 10%. Then, as we did
for the other textures, we alpha-blend the brush with the background computed in the
Base Shading stage, resulting in water strokes.
43
4.6 Results
As evidence that our proposed algorithms are successful at producing panorama maps
reminiscent renderings, we render two panorama maps of Yellowstone National Park.
The first is in the style of Heinrich Berann (see Figure 4.1), the second in the style of
James Niehues (see Figure 4.8). The renderings allow us to do a direct comparison
with the original work. Compare the similarities between our renderings and the actual
paintings (see Figure 3.1 for the original paintings).
For the Berann-style rendering of Yellowstone, we used Berann-style tree strokes
and samples of his colors for near and far distance (refer to Figure 3.9). Similarly, for
the Niehues-style rendering, we used Niehues-style tree strokes and color samples. Cliff
and snow strokes used are shown in Figure 4.9. Stroke length is set to 50 pixels for tree
strokes and 70 pixels for cliff and snow. Stroke width for snow and cliff strokes for both
images varies between 3 and 6 pixels. In the application of the tree strokes, we pad the
Figure 4.8. Yellowstone National Park, rendered with rock, snow, water, and two kinds
of tree textures with Niehues-style strokes and colors.
44
(a)
(b)
Figure 4.9. Cliff strokes used by our system for (a) Yellowstone (b) Rocky Mountain
National Park.
shape by 3 pixels, to control spacing.
We render our images on an Intel Mac Pro 3.0 GHz with 8GB of RAM. The image
size is chosen to match the physical dimensions of the original paintings (assuming 150
dpi). For our chosen image size, rendering consumes from 1.5 to 2.4 GB RAM. For
rendering statistics, refer to Table 4.1.
We generate another example panorama map to demonstrate the applicability of our
algorithm for a different type of terrain - White Sands National Monument (refer to
Figure 4.10). We treat sand as we would treat cliffs. We view the terrain from the east,
again from a distance, and with a narrow field-of-view. We reuse the Berann strokes
(refer to Figure 3.10), as well as the cliff and snow strokes used for our rendering of
Rocky Mountain NP. Stroke length is set to 50 for both tree, sand, and cliff strokes. Tree
spacing is set to 4 pixels. Once again, we pick our 12 color palette for base and stroke
colors for each classification type from images of the region. We did not have to do
anything special to produce this panorama, except for categorize the sand as snow (so
that stroke handling for snow is applied to regions where there is sand). Our rendering
accentuates the mountains in the background and retains the appearance of the sand
Table 4.1. Rendering statistics for our example panoramas.
Panorama
Image
Datset
Rendering
Map
Dimensions Dimensions With Path
Tracing
Yellowstone 5284 x 3587 1889 x 1880 23.65 min
(Berann)
Yellowstone 5284 x 3587 1889 x 1880 24.03 min
(Niehues)
Rocky
5284 x 3699 927 x 767
12.05 min
Mountain
White Sands 6400 x 3050 661 x 657
5.73 min
Mt Rainier
6400 x 3054 3470 x 1470 9.53 min
Texture
Generation
4.83 min
Total
Rendering
Time
28.48 min
4.95 min
28.98 min
4.26 min
16.31 min
4.98 min
3.95 min
10.7 min
13.48 min
45
Figure 4.10. White Sands National Monument, rendered with brush tree, sand, and grass
texture. Colors were selected from real images of the terrain.
dunes.
We also compare our rendering to the image generated by Google Earth for a similar
viewpoint of White Sands (see Figure 4.11), as that is the terrain viewing application
most accessible to the public. Our rendering accentuates the mountains in the background and retains the appearance of the sand dunes.
Figure 4.11. White Sands National Monument as displayed by Google Earth.
46
We also produce a rendering of the area surrounding Mount Rainier and Mount St.
Helens (see Figure 4.12). We view the terrain from the west, in order to see the Pacific
Ocean and Columbia River with the mountains as a backdrop. We reuse Berann strokes
(refer to Figure 3.10) for cliffs and snow, and Niehues’ strokes for trees. Stroke length is
set to 50 for tree, sand, and cliff strokes. Tree spacing is set to 4 pixels. We mix cliff and
snow colors from the Berann palette with tree and water colors from Niehues.
To show the applicability of our algorithms to a different mountainous terrain, we
create a panorama map rendering of Rocky Mountain National Park, Colorado. We view
the terrain from a distance, but with a narrow field-of-view, to reduce the perspective
distortion. The tree strokes are based on the shapes and dimensions of the strokes we
sampled from the Berann Yellowstone Panorama (refer to Figure 3.10). The cliff and
snow strokes we use are shown in Figure 4.9. Each stroke is assigned randomly one of
the available shapes. Stroke length is set to 50 for tree strokes, and 70 for cliff strokes.
Stroke width for snow and cliff strokes for all images again varies between 3 and 6 pixels.
We control tree spacing using a 4 pixel stride.
We do not have a reference panorama map to compare to, and we do not have
colors that have been manually selected by artists. Therefore, we lazily select the 12
Figure 4.12. Mount Rainier and Mount St. Helens, rendered with tree, snow, grass, and
water texture. Colors were selected from a combination of Berann and Niehues colors.
47
color palette for base and stroke colors for each classification type from images of the
region. Unfortunately, this approach does not produce a very attractive panorama (see
Figure 4.13).
Despite the fact that our colors are sampled from real images, corresponding to the
same terrain, the colors of the rendered image are not satisfactory. Guessing the effects
of tweaking the base and stroke colors on the final rendering, in a predictable manner, is
a difficult task, aggravated by the effects of simultaneous contrast.
A possible solution to this problem can be achieved by performing a color transfer
of image statistics from a desirable image to the rendered panorama, using Reinhard et
al. [91]. Unfortunately, the suggested l αβ space nonlinearly scales the luminance of the
rendered image and forces about 22% clamping of values when converted back to RGB
(see Figure 4.14). Applying the transfer in RGB space produces even worse results due
to further color clamping and distortion (refer to Figure 4.15).
Figure 4.13. Rocky Mountain National Park terrain, rendered with rock, snow, water,
and two kinds of tree textures. Rendered with colors from images of the terrain.
48
Figure 4.14. Color transfer result for Figure 4.13 applied in l αβ .
49
Figure 4.15. Color transfer result for Figure 4.13 applied in RGB space.
In order to address this problem, we use a novel color space that is applicable for color
transfer applications (the space is discussed in detail in Chapter 6). We use the proposed
opponent color space, oRGB as the space where the color transfer is performed. The
result of the color transfer, using our original rendering as source and Berann’s panorama
map of Yellowstone as color target, can be seen in Figure 4.16.
4.7 Conclusion
We have demonstrated a novel method for landscape visualization for mountainous
terrain. Our main contributions are: an analysis of the functional and stylistic role
played by visual elements in panorama maps and an automatic technique that allows
us to generate images whose visual style is reminiscent of those in panorama maps of
mountainous terrain. We have chosen to operate mainly in image space because that is
the natural space in which to generate painterly strokes. The downside of using image
space is that an animated change of viewpoint is then difficult. This is an example of
50
Figure 4.16. Color transfer result for Figure 4.13 applied in oRGB space.
the classic tradeoff between image quality and frame coherence seen in almost all NPR
work.
The most obvious limitation of our work is that we have mostly concentrated on
the rendering, and only partially on geometry manipulation (and mostly for visibility
purposes). We have not modified the geometric information itself to make it clear and
easier to understand, and we have not proposed any ways in which we can do geometric
feature enhancements.
In the future, we would like to improve the speed of our algorithms, specifically
the path tracing part. We anticipate that making our system parallel and utilizing an
irradiance cache would speed up our system drastically. In addition, we would also like
to address modifications that might be necessary when rendering different types of terrain
or cultural features.
CHAPTER 5
AUTOMATIC VIEW SELECTION FOR
OUTDOOR SCENES
In the preceding two chapters we analyzed panorama map paintings, derived principles and heuristics for panorama map design, and proposed a rendering system that
allows us to create panorama maps of different terrain datasets. Our rendering system
assumed the user will provide the desired position of the camera and set the camera
orientation.
Sometimes, that is not possible, as the user has no easy way of previewing the
geometry. Other times, the scenes are so large (in fact quite commonly), that no existing
application relying on OpenGL, i.e., Maya, can load and display the dataset interactively.
Furthermore, selecting just any view is not useful, as we want to visualize as many of the
important features of the scene as possible.
However, rendering requires that a camera viewpoint be specified. In this chapter,
we present criteria and some practical solutions to the automatic viewpoint selection
problem. Our algorithm is specific to outdoors scenes, as we developed the work as a
way to automate the practical map design problem encountered by panorama map artists.
5.1 Good Views
There are infinitely many 2D images of a given 3D object or scene. How do we make
a good one? Intuitively, it seems that an image from a good viewpoint is one that provides
as much information about the world of interest as possible. In practice, this definition
is too general as “information” is application dependent. For example, in recognition, a
c Eurographics Association 2009; Reproduced by kind permission of the Eurographics Association,
°
from ”Automatic Views of Natural Scenes.” in the proceedings of Computational Aesthetics in Graphics,
Visualization, and Imaging (2009) with authors Margarita Bratkova, William B. Thompson, and Peter
Shirley.
52
good view, also referred to as a “canonical view” is one that aids people identify an object
in the least amount of time. A good view could also be the one that is more “aesthetic”
or subjectively visually pleasing, as opposed to a bad view (Figure 5.1).
We note that measures of view quality are ultimately subjective. However, we believe automatically generating good initial images of scenes, or selecting representative
viewpoints can be beneficial for meaningful scene visualization. Gaining insight from an
image is hard for large or complex scenes, especially when interactive exploration is not
possible. Therefore, our goal is to provide a general framework that computes automatic
viewpoints of natural scenes by taking into account important features, preserving perceptually relevant 3D information during the image projection, and presenting the result
in a visually pleasing manner.
In this chapter we present a set of criteria for viewpoint quality estimation that is
based on utility, perception, and aesthetics. We also describe a method that uses the
criteria to automatically produce viewpoints for outdoor terrain scenes.
5.1.1
Viewpoint Selection
Viewpoint selection is a widely researched topic in a variety of fields, including
computer vision, cinematography, geo-visualization, and image modeling. More recently
it has also gained momentum in the fields of computer graphics and visualization.
Most viewpoint selection methods in computer graphics use the heuristic that the
best view is the one that provides the user with the most information (depending on
(a) Good view
Figure 5.1. Good and bad view of an outdoor scene.
(b) Bad view
53
the application). For virtual world exploration and camera path planning, the goal is
to automatically determine camera positions which when connected produce camera
path trajectories that explore the space in the most informative manner (see [20] for
an in-depth overview). In scene understanding, it is desirable to choose individual viewpoints that are most representative and indicative of the scene [6], while in image-based
rendering, the intent is to find a minimal set of viewpoints that can see the object of
interest and allow its reconstruction [34]. For mesh-saliency, an attempt is made to find
views that emphasize the most important features of an object [33, 71, 88].
Viewpoint quality is commonly computed as a sum of visible surface quantities, i.e.,
projected area of nondegenerate faces or angle between the look direction and the face
normal [6, 60]. Viewpoint entropy is an information theory based measure that represents
distribution of visibility and is applied to the unconnected geometry faces. A good view
is one with high viewpoint entropy and representing high number of faces [99, 118].
Viewpoint mutual information, a different information theory measure, is calculated on
“information channels” between all viewpoints and all object polygons [33]. It is more
robust to polygonal discretization (unlike viewpoint entropy). Stoev and Strasser [110]
note that such methods generate top-down views that lack depth for scenes whose normals point in a similar direction, e.g., terrain models, and advocate the need for a measure
that accounts for scene depth.
More recent work explores the connectivity of the geometry of the visible surfaces in
addition to using primitive elements of the 3D model. Sokolov and Plemenos consider
the total curvature of visible surfaces as an amount of information appropriate for a
specific viewpoint [105].
Viewpoint selection criteria have commonly been aimed at determining the best view
of a standalone object, as opposed to scenes. Higher-level methods have also been
proposed, where not only the visibility of faces, but also the visibility of individual
objects has been deemed appropriate [106, 107].
The search for best viewpoint requires the evaluation of specific viewpoint parameters. Most commonly, viewpoint position is chosen from a discrete set selected on
the bounding sphere of the object or scene, with direction aimed at the center of the
54
sphere. The rendered view is given a weight, and the best view is the image with the best
criteria result. Some authors introduce heuristics that limit the search space. For objects,
Polonsky et al. assume an up direction, three-quarter views, and normal clustering [88].
For scenes, Sokolov and Plemenos choose a set of discrete camera positions, fixed at a
predetermined height, as well as a discrete set of view directions [106]. The algorithm
performs a greedy search through the discretized parameter space.
In Geographic Information Systems (GIS), it is often the visibility between points
on a terrain surface that is of importance. Viewshed analysis methods are commonly
employed to assess what areas are visible from specific locations in space. The increasing
size terrain data are difficult to handle by existing viewshed algorithms. A grid-based
solution proposed by Wu et al. [128] is practical for large scale terrains.
5.2 Viewing
Many rendering systems use a perfect pinhole camera and place the image plane a
unit distance away from the pinhole. Introducing bounds to the projection plane produces
a fixed dimension image of the world. Since a pinhole camera never projects along
parallel lines, we always have a perspective, as opposed to parallel, projection. While
pinhole cameras do not have focal length, f , for practical reasons focal length is used to
indicate the distance between the pinhole and the image. In graphics, pinhole cameras
usually have zero aperture, and therefore depth of field is rarely a parameter.
Geometry is typically specified in world coordinates. When projecting the geometry
into the camera image plane, it is useful to express it in the camera’s coordinate frame. A
standard rigid transformation described by a rotation matrix, R, and a translation vector,
t, takes a general world coordinate system frame, Fw = (Ow , Uw , Vw , Ww ), into the
pinhole camera coordinate system frame, Fc = (Oc , Uc , Vc , Wc ).
Parameters that specify the position (via translation w.r.t. the world frame), Oc , and
orientation (via rotation w.r.t. the world frame), Wc , of the camera are called extrinsic.
Constructing the camera frame assumes a standard up direction. The field of view of
the pinhole determines the bounds of the image in the image plane, as it controls what
portion of the scene projects to it. It is usually measured as an absolute angle in the
55
horizontal (hFov) and vertical (vFov) directions centered around the optical center of
the camera. Assuming unit focal length, the dimension of the image plane is w × h,
where w = 2 ∗ tan (hFov/2) and h = 2 ∗ tan (vFov/2). Similarly to wide angle lens in
photographic cameras, the wider the field of view, the bigger the portion of the world
that is recorded in the image.
5.2.1
Image Composition
Modifying the viewing parameters of a virtual camera will produce images of diverse
quality. Without doubt, aesthetics is subjective. Nonetheless, artists commonly use
heuristic compositional rules. And they can be especially helpful when automatically
creating images in computer graphics.
In the graphics literature, Gooch et al. [40] propose a simple system based on artistic
heuristics that determines the format, viewpoint, and layout for an image of a 3D object.
Grimm discusses composition techniques such as camera angle, screen placement, and
color adjustment as well as their use in a postrendering step [42].
5.3 Criteria for Automatic View Selection and Image
Composition of Outdoor Scenes
As there is no agreement among researchers for common criteria that determine
the “best view” of a complex scene, we will discuss three criteria that we believe are
important in the context of outdoor terrain scenes.
While we are interested in automatically determining good viewpoints for complex
scenes in terms of their usefulness (i.e., can we see the objects that matter), we think it
is also important that we aid the perceptual system in recovering depth from the image,
and finally - that the image produced is aesthetically pleasing.
5.3.1
Utility Criteria
When determining if an image presents a useful view of a scene, we feel it is important that objects and features of interest are clearly visible and recognizable. For outdoor
terrain scenes, such features could include any of the following types: topographic,
landmarks, or man-made.
56
Topographic features include ridges, valleys, peaks, etc. Such features are part of the
geology of the terrain. They are meaningful, as their visual recognition provides us with
vital information about the essence of the geometry of the space.
Landmarks are prominent or well-known markers that help us determine spatial directions and relations. A well-known mountain (i.e., the Grand Teton), or a famous peak
(i.e., Denali), are semantically useful. In fact, many people navigate through complex
terrain based on landmark features alone.
Man-made features include roads, buildings, etc. They are not naturally part of the
geology of the terrain, but are also important in navigation and self-localization.
5.3.2
Perceptual Criteria
If the only objective is to make sure that all objects of interest are in view, an
obvious solution is an orthographic aerial view from above. Such a view is indeed
useful in cartography, and has led to the development of the topographic map. However,
the view from above does not provide any cues that can be used to reconstruct the
three dimensionality of the real terrain. Since the goal of visualization is to provide a
meaningful and perhaps, enhanced, rendition of the geometry, in this work our goal is to
determine a view from which the image projection of the 3D geometry allows the viewer
to maintain the sense of depth present in the real scene.
We propose the following three metrics: occlusion, depth variability, and horizon
visibility, as they directly affect our depth perception of the geometry of a scene from an
image.
When foreground surfaces obstruct the view of background surfaces, our perceptual
system can interpret the occlusion cues as an indication of real depth. We need to be
careful not to take this to the extreme (e.g., having near features completely replace more
distant features in the image), as our goal here is the visualization of the entire scene, not
just a subset of it.
A complementary metric to occlusion is depth variability. The more occluded surfaces are, the smaller the range of scene depth for the objects visible in the image. Also,
the higher the depth variability between the visible near and far surfaces in the scene, the
57
more pictorial depth cues will be maintained later when 3D shading and texturing are
applied, making depth perception easier.
Finally, ability to see the horizon further helps establish depth, as objects near the
horizon are perceived to be farther away than objects away from it.
5.3.3
Aesthetic Criteria
Composition is the visual structural arrangement of elements in an image. Successful
compositions are achieved by selecting a meaningful viewpoint and then framing the
objects in the scene such that they form an image with a balanced visual layout [40].
Framing determines the image boundaries of the depicted scene and is usually specified by format and aspect ratio. The framing format deals with the proportions of
the final image. The appropriateness of the format depends on the visual elements
depicted [4, 76]. An object that has more horizontal extent will be better visualized in an
image that is of a landscape format, as it fits better in the image frame. Similarly, objects
that have more vertical extent will look better in a portrait format. The framing aspect
ratio, the ratio between the width and height of an image, is commonly determined from
existing standards. For example, 35 mm film constraints the aspect ratio to (3 : 2), while
HDTV uses (16 : 9). In art, however, the aspect ratio is not controlled by a standard.
Instead, the artist uses it to increase the emotional and compositional impact of the work.
Elongated aspect ratios are perceived to be more dynamic, while aspect ratios close to
one are more static [45]. Certain aspect ratios, however, seem to be preferred more than
others. The golden section, φ , is a commonly employed, aesthetically pleasing aspect
ratio that has been used in art and architecture since the Ancient Greeks [29], and perhaps
earlier. It is expressed algebraically as follows:
√
a a+b 1+ 5
φ= =
=
≈ 1.618
b
a
2
Psychophysical experiments provide evidence [32, 69] that common people also have
an aesthetic preference for the golden section proportions. Table 5.1 lists the ratios of
58
Table 5.1. Rectangle ratios, sorted in order of preference by Fechner. The golden section
is preferred in both studies.
Aspect Ratio Fechner Study [32] Lalo Study [69]
8:5
35.0 %
30.3 %
20.6 %
11.0 %
3:2
23:13
20.0 %
6.3 %
10:7
7.7 %
5.6 %
2:1
7.5 %
8.0 %
3.0 %
11.7 %
1:1
4:3
2.5 %
9.5 %
5:4
2.0 %
1.3 %
5:2
1.5 %
15.3 %
6:5
0.3 %
1.0 %
the most preferred rectangles as well as the percent of people that chose them [29]. The
most popular ratio, (8 : 5), very closely approximates φ .
Interestingly, it has been suggested that the visual elements of a composition influence the desirable format of the frame, since commonly chosen aspect ratios by artists
do not always correspond to the golden section [4].
Layout ensures the balanced distribution of important elements in the image plane.
There are no real rules that can be used here, only heuristics. A variety of approaches
have been recommended by artists [41, 66, 76, 100]. In its simplest form, creating a
balanced image requires that all elements balance their visual weight, which is commonly
determined from their luminance, color, and placement in the frame. We note that
brighter objects have a stronger perceived visual weight than darker objects. Similarly,
certain hues are perceived to be brighter than others, despite the same luminance. Among
photographers, equal division of the visual elements for placement in the frame is considered bad, and thirds and fifths are considered better than halves and fourths [21]. These
are the so called rule of thirds or rule of fifths [66, 76, 100], which work in the following
way: when the aspect ratio of the final image has been determined, the visual elements
are placed onto an imaginary 3x3 (for rule of thirds) and 5x5 (for rule of fifths) grid in the
final image. These rules are only a guide and not a silver bullet. In fact, it is suggested
that variety and unity [21] are essential to a good composition.
59
5.4 Automatic Viewpoint Selection and Image Composition
Our goal is to automatically determine viewpoints that form meaningful and wellcomposed images of terrain scenes. We do not want to modify the geometry of the
underlying scene in order to improve the composition, nor do we want to modify the
lighting and the surface materials and textures.
Though there are different ways one may approach this problem, our strategy is
to perform a global optimization through the multidimensional search space of camera
parameters. Let C be the space of all possible camera configurations, and F, the objective
function we are trying to maximize. Then, we can define the problem mathematically as
trying to find a camera configuration, c ∈ C, that maximizes an objective function, F, as
follows:
maximize F( f1 (c), f2 (c), . . . , fn (c)), s.t. c ∈ C
where the function fi measures the quality of an image for a specific criteria. In its
simplest case, F is a linear combinations of scalar weighted functions:
n
F ( f1 (x), f2 (x), . . . , fn (x)) =
∑ wi fi(x)
i=1
5.4.1
Finding a Solution
Optimization algorithms often require the availability of derivatives, as this determines the minimization direction in the local neighborhood. In our setup we cannot
express the objective function analytically, and therefore we can not directly find its
derivative. It is possible to express the function, as well as to approximate its derivative,
numerically. However, such an approximation would be a major bottleneck for the evaluation function, as each derivative estimation will require two evaluations per parameter
(assuming central differences).
Stochastic search techniques are useful when the structure of the search space is
not well understood, or is not smooth (as is our case). Simulated Annealing [61] is
60
a stochastic search technique that addresses these complications. It does not require
the presence of derivatives and is applicable to multidimensional, global optimization
problems. It is resilient to local minima, and converges to a plausible global solution in
a reasonable time.
5.4.2
Setup
Our camera setup uses a perfect pinhole camera positioned a unit distance away
from the image plane. The image is formed symmetrically around the optical axis.
Assuming a default up orientation, we specify the extrinsic parameters of the camera
via 6 independent degrees of freedom (DOF): 3 for the direction vector (using cartesian
coordinates), and 3 for the position. Our images are rendered using the golden ratio.
Vertical field-of-view is used to control the image formation. Therefore, each camera
configuration is controlled by 7 independent parameters.
Given a set of fixed weights for the different criteria discussed in Section 5.3, we
start with an initial set of camera parameters determined from the bounding box of the
scene. The camera is positioned at the maximum extent of the bounding box and is
oriented towards the center of the bounding box. The heightfield scene is rendered with
a ray tracer that uses a dynamic BVH implementation [122]. We wrap this in a Simulated
Annealing optimizer that employs the ray tracer to render an image for each set of camera
parameters and compute its corresponding objective function. Whether a rasterizer or a
ray tracer is used, however, is a matter of preference, and should not affect results.
5.4.3
The Objective Function
In our discussion of meaningful criteria for outdoor scenes in Section 5.3 we advocated that a “best image” objective function should satisfy, to some reasonable degree,
utility, perceptual, and artistic considerations. Specifically, the image produced should
maximize the normalized projected area of the desired features. The view should maximize scene depth, prefer views that provide greater depth variability, reward a certain
level of element occlusion, and make sure to include the horizon in the final rendering.
Finally, the image should evaluate the position and balance of the functional elements,
and should choose an image plane that can be framed according to compositional con-
61
straints.
In our evaluation we attempt to use a representative subset of these criteria. In
addition, all criteria metrics are normalized to the [0, 1] range, so that their behavior
can be meaningfully controlled by the desired weight factors.
As a matter of notation, an image produced by a camera configuration ci is indicated
by Ii . The quality of such an image is measured by F (U(Ii )), P(Ii ), A(Ii )), where U(Ii ))
represents utility, P(Ii ) perceptual, and A(Ii ) aesthetic criteria.
5.4.3.1
Utility Criteria
A simple binary test that determines whether a feature can be seen in an image is not
useful if the landmark projects to a single pixel. Instead, a quantitative measure should
be used to ensure that the object of interest is ”visible enough”.
Projected area is a direct, quantitative measurement of how visible a feature is. Since
some objects are quite large, their relative projected area is a more useful measure of
whether or not ”enough of the object” is visible.
Ridges and valleys are the interesting features of the terrain and accentuate peaks and
canyons, while landmarks mark areas of semantic importance to the user. Our utility
criteria is therefore evaluated by the normalized projected area of ridges (Rn ), valleys
(Vn ), and user-specified landmark features (Fn ), with weights wr , wv , and w f , respectively,
where:
U (Ii ) = wr ∗ Rn (Ii ) + wv ∗Vn (Ii ) + w f ∗ Fn (Ii )
5.4.3.2
Perceptual Criteria
Our images need to maintain a sense of depth. However, it is also desirable that they
show much of the extent of the terrain we are trying to visualize.
The depth variability metric addresses both requirements, as it rewards scene depth,
but also encourages seeing the scene in its entirety. Another useful depth-related metric
is occlusion. Since it is based on the properties of the topology, not simply the distance
to parts of the terrain, it nicely complements scene variability
62
We evaluate our perceptual criteria by using the normalized depth variability (Dn )
and occlusion (O) with weights wd and wo :
P (Ii ) = wd ∗ Dn (Ii ) + wo ∗
p
O (Ii )
Though O has a range of [0, 1], its values vary nonlinearly with respect to variety of
views and topologies. For example, terrain occlusion of 20% does not produce twice
as good a result as having only 10% of the terrain occluded. We try to make the effect
of each metric of the objective function as linear as possible. Therefore, we scale this
metric nonlinearly to correct for its contribution.
5.4.3.3
Aesthetic Criteria
Since our scenes are made of large terrains that typically have a wide horizontal
extent, it is natural to select a landscape format for our renderings. We choose not to
optimize for an aspect ratio, as artists commonly make that choice before producing a
final image. Instead we fix the aspect ratio to the golden section for all rendered images.
In the interest of reducing the complexity of our analysis, we currently ignore the
effects of illumination and color on the layout. Our images are produced by only controlling for frame placement, specifically - we use the rule of fifths to place the horizon.
The frame placement of the horizon is rewarded by filling the upper fifth of the image
with pixels marked as sky (Sn ), and filling the rest of the image with land (Ln ) with
weights ws and wl :
A (Ii ) = ws ∗ Sn (Ii ) + wl ∗ Ln (Ii )2
We feel people are especially sensitive to large gaps of open space, where land is
instead expected. To penalize such behavior, we nonlinearly scale Ln .
63
5.4.3.4
Weights
Clearly, not all of the individual criteria are of equal importance or interest. Their
weighting should be left as flexible input that can be set by the user of the optimization
system.
However, we believe that all three types of criteria should have similar importance.
Since we mix the objective function weights in a linear fashion, we expect each of the
three groups to contribute similarly. Our tests support this observation. Refer to Table 5.2
for a summary of weight ranges that worked fairly well for our scenes.
5.5 Results
Our system is implemented in C++ on a 3GHz Mac Pro System with 8GB of memory.
We use three different heightfield data sets with 30 m resolution data - Yellowstone NP,
Rocky Mountain NP, and Death Valley NP. All our images are rendered at resolution
900 × 557, and the optimizations run for 210 iterations.
We used the same weights for all optimizations (see Table 5.2) . The only user input
consisted of texture maps that marked features of interest for each of the three scenes.
The values of the individual metrics are evaluated automatically. For each dataset, we
ran 10 independent evaluations, with the same initial camera parameters. For rendering
statistics, please refer to Table 5.3. Our results show the image with the highest energy
at the final iteration.
Table 5.2. Criteria weight ranges that proved useful in practice, as well as weights used
for all the results in this paper.
Metric
Suggested Weight Weight Percentage
Percentage Range Used In Chapter
Ridges (wr )
8 - 12 %
8%
Valleys (wv )
7 - 12 %
7%
Features (w f )
10 - 15 %
10 %
12 %
Depth Variability (wd ) 10 - 25 %
Occlusion (wo )
12 - 25 %
28 %
Sky (ws )
15 - 20 %
17.5 %
Land (wl )
15 - 20 %
17.5 %
64
Table 5.3. Rendering statistics for our solution. Total time includes processing of the
dataset, building of the bvh, and rendering as well as evaluation of each of the 210
images.
Terrain
Dataset
Total
Dimensions Time
Dataset
Yellowstone (Fig. 5.2)
1889 x 1880 20.59 min
12.62 min
Rocky Mountain (Fig. 5.3) 927 x 767
Death Valley (Fig. 5.4)
2596 x 2537 21.27 min
Figures 5.2, 5.3, and 5.4 are renderings produced by our system. The image at top
is rendered with the initial camera setup. The one at the bottom - with the optimized
viewpoint solution. We feel our results demonstrate promising results and consistently
validate our framework.
Figures 5.5, 5.6, and 5.7 display the values of the objective functions, as well as
its individual components during the optimization (top), and the values of the camera
parameters (bottom) for Yellowstone National Park, Rocky Mountain National Park, and
Death Valley National Park, respectively.
The values of the objective function in Figure 5.5 reveal that the initial camera setup
(iteration 1) has an acceptable composition score and great depth variability. However, it
fails to visualize features of interest, and only minimally displays the ridges and valleys
present in the terrain. This is easy to see in the initial setup rendering (Figure 5.2, top).
After the optimization, our requested features of interest, the Yellowstone Lake (center)
and the Grand Tetons (bottom left) are clearly visible (Figure 5.2, bottom). The image
maintains a good sense of depth - we can see the near as well as the distant parts of the
terrain. Considering the finite dimensions of the terrain, the land is framed such that if
fills most of the bottom of the image, while leaving a nice area at the top for the sky.
In Figure 5.3, we can clearly see Grand Lake (top left), and the city of Estes Park
(center). The prominent mountain ridge is easy to spot, and the position of the camera is
well oriented so that we can see along the fork of the Big Thompson River (center right).
The image has a great sense of depth, allows us to see the extent of the terrain, and is
framed exactly to our expectations. Similarly, in Figure 5.4, the rendered image features
prominent ridges and valleys, a reasonable sense of depth, and meets our framing criteria.
65
(a) With Initial Parameters
(b) With Optimized parameters
Figure 5.2. Rendering of Yellowstone NP produced with initial camera parameters and
after optimization with energy 0.71.
66
(a) With Initial Parameters
(b) WIth Optimized parameters
Figure 5.3. Rendering of Rocky Mountain NP produced with initial camera parameters
and with optimized parameters with energy 0.63.
67
(a) With Initial Parameters
(b) With Optimized Parameters
Figure 5.4. Rendering of Death Valley NP produced with initial camera parameters and
after optimization with energy 0.66.
68
1
energy
features
ridges
valleys
depthVar
occlusion
composition
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
50
100
150
200
250
300
(a) Objective Function
450000
eyeX
eyeY
eyeZ
lookX
lookY
lookZ
vFov
400000
350000
300000
250000
200000
150000
100000
50000
0
0
50
100
150
200
250
300
(b) Camera Parameters
Figure 5.5. Value of the objective function and its individual components during the optimization of Yellowstone NP, as well as corresponding values of the camera parameters.
The horizontal axis represents number of iterations.
69
1
energy
features
ridges
valleys
depthVar
occlusion
composition
0.8
0.6
0.4
0.2
0
0
50
100
150
200
250
300
(a) Objective Function
250000
eyeX
eyeY
eyeZ
lookX
lookY
lookZ
vFov
200000
150000
100000
50000
0
0
50
100
150
200
250
300
(b) Camera Parameters
Figure 5.6. Value of the objective function and its individual components during the
optimization of Rocky Mountain NP, as well as corresponding values of the camera
parameters. The horizontal axis represents number of iterations.
70
1
energy
features
ridges
valleys
depthVar
occlusion
composition
0.8
0.6
0.4
0.2
0
0
50
100
150
200
250
300
(a) Objective Function
500000
eyeX
eyeY
eyeZ
lookX
lookY
lookZ
vFov
450000
400000
350000
300000
250000
200000
150000
100000
50000
0
0
50
100
150
200
250
300
(b) Camera Parameters
Figure 5.7. Value of the objective function and its individual components during
the optimization of Death Valley NP, as well as corresponding values of the camera
parameters. The horizontal axis represents number of iterations.
71
Figures 5.8, 5.9, and 5.10 show more example optimizations for the same three
scenes.
5.6 Limitations
As with any stochastic global method, we note that we cannot guarantee the image
produced by a specific optimization will be good. However, we can evaluate its goodness
and repeat the optimization, until a user-specified minimum energy is reached.
Global optimization is never fast, especially when we have terrain datasets as big as
ours. Occlusion calculations on large terrains with undulating topology are also quite
costly.
While speed has not been a great concern for us, optimizing and parallelizing our
raytracer, as well as utilizing the power of the GPU, will most certainly improve the
speed of the evaluation drastically. In particular, we prefer to consider the number of
frames necessary to find a good solution rather than the particular time necessary. Faster
rendering systems will therefore produce good views more quickly, but will still require
the same number of iterations. In addition, there are also a number of ways to speed up
the rendering. We can perform the evaluation on smaller images, or we can apply mesh
simplification on the terrain datasets.
5.7 Conclusion and Future Work
In this chapter we examine a set of criteria useful for automatic view selection and
image composition of outdoor scenes. Following that, we discuss how one may create an objective function formed by linear combination of scalar weighted functions
representing the criteria discussed and use it for solving the multidimensional camera
parameter optimization. We present results for three different large terrain datasets. The
results directly provide the viewpoint parameters that can be used by our Panorama Map
rendering system.
We believe the set of metrics we have advocated for is a good starting point, as it
produces fairly successful images with no manual intervention. Additional metrics are
likely to improve the quality of the results, most notably, addressing the effects of color
and lighting variations on image layout.
72
(a) With Initial Parameters
(b) With Optimized parameters
Figure 5.8. Rendering of Yellowstone NP produced after optimization with energy 0.66.
73
(a) With Initial Parameters
(b) With Optimized parameters
Figure 5.9. Rendering of Rocky Mountain NP produced after optimization with energy
0.68.
74
(a) With Initial Parameters
(b) With Optimized parameters
Figure 5.10. Rendering of Death Valley NP produced after optimization with energy
0.59.
75
Future work should investigate a faster method for rendering the geometry. It will
also be useful to perform a user study that validates our proposed criteria and approach,
as well as allow us to fine-tune criteria weights based on perceived importance.
CHAPTER 6
OPPONENT COLOR SPACE: ORGB
In Chapter 4 we encountered a situation where an image produced by our panorama
map system did not exhibit a good selection of color, despite our efforts. As a way of
improving the distribution of the colors, we suggested the use of color transfer [91].
Unfortunately, we noted that the transfered result exhibited undesirable color shift, especially of the sky. To address this problem, we developed a novel color space, oRGB,
which we now present. We show its applicability to a variety of graphics applications, in
addition to color transfer.
Like HSV, oRGB is a an invertible transform from RGB. The primaries of this model
are based on the three fundamental psychological opponent axes (white-black, red-green,
and yellow-blue). The model has three channels - one luma (see Section 6.1.5 for a
definition) and two chroma, as shown in Figure 6.1 . The nonlinearity of these channels
relative to intensity uses the nonlinear gamma encoding of the native RGB space. Thus
it has some of the advantages of nonlinear perceptual spaces such as CIE L*a*b*, while
maintaining a simple gamut and computationally simple transform to and from RGB.
Because the chroma axes of oRGB go from warm to cool (red to green and yellow
to blue), oRGB has a quantitative concept of color warmth as needed by some artistic
applications. This new space is reasonably simple, has intuitive axes, and is well-suited
for computational applications.
In the rest of this work, we first review opponent process color theory and some of
the color models used in computer graphics (Section 6.1). We then develop the details
c 2009 IEEE. Reprinted, with permission, from ”oRGB: A Practical Opponent Color Space for Com°
puter Graphics.” in Computer Graphics and Applications 29, 1 (2009) with authors Margarita Bratkova,
Solomon Boulos, and Peter Shirley.
77
(a) Original Image
(b) Luma Channel
(c) Yellow-Blue Channel
(d) Red-Green Channel
Figure 6.1. Visualization of the oRGB decomposition of the image in the top left into
luma, yellow-blue, and red-green channels.
of the oRGB model (Section 6.2), and show the application of this color space for color
adjustment, NPR lighting computation, color transfer, and gamut mapping (Section 6.3).
6.1 Color Models
Digital images are stored, manipulated, and transmitted in a variety of color spaces.
Many different color spaces have been proposed, with RGB, CMYK, HSV, and CIE
L*a*b* being especially popular (see [31, 50]). The RGB space is an additive color
model and is probably the most commonly used among computer graphics practitioners,
but it is not very intuitive. CMYK is a subtractive color model and is most useful in
color printing. A bijective transform of RGB space, HSV was introduced by Smith [104]
as a more convenient and meaningful color space, designed specifically for computer
graphics. At about the same time, the need for a device independent, perceptually linear
78
color space led to the development of CIE L*a*b*.
The recent interest in automatic image manipulation techniques such as color harmonization, color transfer, colorization, as well as the proliferation of work in computational
photography, suggest a need for an intuitive, yet simple and practical computational
computer graphics color space. While HSV has proven extremely useful in practice,
there are two characteristics it lacks from our standpoint. First, its primary colors (red,
green, and blue) are based on the mechanics of additive color mixing rather than the more
natural psychological primary colors. Second, HSV is well-suited for color selection, but
is not a good space for computational applications.
We now review the high-level differences between various color ordering systems,
paying particular attention to opponent color systems. We also establish several terms
and symbolic conventions as the study of color spans several disciplines and there are no
universally consistent uses.
6.1.1
Hue Spacing Systems
Most color spaces have one light-dark axis and two chromatic axes. The boundary of
the realizable colors in the chromatic dimension is often displayed as a circle or “wheel”.
The ordering of hues is the same in most color spaces, but the spacing between the hues
differs. We summarize these to clarify the relationship between opponent color systems
and more traditional hue ordering systems.
Four of the most popular types of color circles are shown in Figure 6.2. The two
circles to the left are based on three primaries and how they mix. The uniform color
circle, of which the one that visualizes the Munsell Color Space is perhaps the best known
attempt to arrange the hues so they are “evenly spaced”. Such uniform spaces often lack
a concept of “primaries” as uniformity precludes even spacing between named colors
such as red and blue. To the right is a Hering-style opponent circle with four primaries
on two natural axes. It emphasizes his concept of red-green and yellow-blue axes, while
maintaining the order of colors as in the other color circles.
79
subtractive
additive
uniform
opponent
Figure 6.2. Visualization of four common types of color circles. The ordering of hues
is preserved in each, while the position of the hues on the circle varies. The subtractive
circle is that used by painters while a cyan-magenta-yellow circle (not shown) is used by
most ink printing processes.
6.1.2
Hering’s Opponent Process Color Theory
The opponent process theory of color was first advocated by Ewald Hering in the
1870s. It took a sharp departure from the prevalent theory of the time, the trichromatic
theory of Young-Helmholtz, by proposing four hue primaries: red, green, yellow, and
blue, instead of the traditionally accepted three: red, green, and blue.
Interestingly, Hering was not the first to identify these as fundamental colors; Leonardo
da Vinci wrote in the fifteenth century:
The simple colours are six, of which the first is white, although some philosophers do not accept black or white in the number of colours, because one is
the cause of colours and the other is the absence of them. Yet, because
painters cannot do without them, we include them in their number and say
that in this order white is the first among the simple, and yellow is second,
green is third, blue is the fourth, red is the fifth, and black is the sixth [92].
While three primaries are needed to produce all possible colors, Hering noted that
our subjective experience of color suggests that perceptually there is an encoding for
the fourth. For example, while purple seems to be a mixture of red and blue, yellow
does not seem to be composed of any other primaries. Instead, it seems to be a “pure”
color, a primary itself. In addition, Hering observed that we do not seem to have a
phenomenological experience for a color that is both yellow and blue, nor red and green.
However, we have one for a color that is both yellow and red (orange).
These observations led Hering to hypothesize that human color perception uses two
chromatic axes: one that ranges from red to green, and another from yellow to blue.
80
If these axes are independent, then people will be able to see a red and yellow color
simultaneously (orange), but not a red and green color, nor a blue and yellow color, as
they share the same “channel” (see Figure 6.3).
Hering’s ideas were controversial at the time. However, there has been a wealth of
psychological and physiological evidence since then (Hurvich [53] brought the revival
of the theory) that suggests that the organization of human color perception is at least
partially based on opponent axes [22, 117]. Color scientists have moved more and
more towards including some form opponency into many of their color spaces, with
CIE L*a*b* and the Natural Color System (NCS) being notable examples.
6.1.3
Opponent Space Models
A variety of spaces are based on opponent axes, with the television encoding YIQ
being perhaps the best known. However, we make the distinction that most of these
spaces are not Hering-style opponent spaces in that their principal axes do not include a
red-green dimension (while some of them do include a blue-yellow). This is illustrated
in Figure 6.4 for YCC (a modern digital descendant of YIQ), l αβ (a color space based
on natural image statistics), and CIE L*a*b* (an approximately perceptually uniform
space).
Note that all of these models have more of a magenta-cyan axis than a red-green one.
An opponent space proposed for computer graphics by Naiman [82] is similar to YCC
(though it has a slightly different matrix, and operates on linear RGB), but it also has an
axis shifted toward magenta-cyan.
reds
blues
yellows
greens
Figure 6.3. The color circle divided into its opponent sides. The top part of the circle is
composed of colors that have a red tone in them, the bottom - green, the right - yellow,
and the left - blue.
81
(a) YCC Yellow-Blue Channel
(b) YCC Red-Green Channel
(c) l αβ Yellow-Blue Channel
(d) l αβ Red-Green Channel
(e) CIE L*a*b Yellow-Blue Channel
(f) CIE L*a*b Red-Green Channel
Figure 6.4. Visualization of the chroma channels for YCC, l αβ , and CIE L*a*b color
spaces. Compared to our new oRGB space (Figure 6.1), these spaces do not have a true
red-green channel.
The most well-known opponent space with a true red-green axis is the NCS. Unfortunately, there are no accepted color formulas to allow this space to be used for
computation [68], and it is therefore not well suited for computer graphics applications.
82
We are aware of only three quantitative Hering-style opponent models. The first is
that of Schwarz et al. [101] who embed their space in XYZ tristimulus space with nonorthogonal but true opponent axes. The second, introduced by Ware and Cowen [123],
places the opponent primaries at corners of the model. Unfortunately, the axes are not
opponent. The third is that of Shirriff [103] who developed an orthogonal red-green
and yellow-blue space, using HSV as an intermediary space. Each of these spaces were
designed for color selection, are defined in terms of linear RGB, and are not well suited
for computational color applications. We will explore this further in Section 6.3.
6.1.4
Limitations of Color Models
One approach to generating a color model is to proceed directly from existing scientific knowledge. While one might expect color science to be both a mature and
static field, this is far from the case. The details of the physiological origin of color
perception remain enigmatic despite the increasing understanding of opponent mechanisms [22, 117]. The psychophysical approach that is used in the construction of most
models also has limitations. There is unlikely to be a “right” choice, as different color
spaces have different goals and are all simplifications of the real experience [67]. For this
reason, existing color spaces are based at least partially on heuristics and differ largely
in their goals and how they manage trade-offs.
Opponent color theory presents many unknowns and asymmetries. For example, it is
believed that the opponent signals are not simple linear combinations of cone signals as
is often diagrammed in textbooks [19, 117]. In addition, the axes do not all combine in
symmetrical ways. The combinations of some axes create “important” colors with their
own names, while others do not [44]. For example, red and yellow make orange, while
green and yellow make chartreuse (which most English speakers associate with a pink
or purplish color). Fundamental colors such as blue still look “blue” when darkened,
while some may change color category such as an orange turning to “brown” when
darkened. In addition, the chromatic axes have a natural symmetry around grey while
the achromatic black-white axis is believed to be (and represented in most color models
as) just positive [67, 121].
These uncertainties and trade-offs suggest that there is no “true” color space. It is
83
important that one bases the choice of color space on its performance in the applications
of interest. Here we propose a color space that is suitable for computational applications
and is simple, intuitive, and practical.
6.1.5
Terminology and Notation
Most computer graphics images are stored in some form of an RGB color space.
While there are many different RGB spaces, we assume that for the most part, our
images are encoded in sRGB, an increasingly popular standard. An often confusing
notational issue is distinguishing between linear and nonlinear intensity red, green, and
blue components. A linear red channel stored with range [0, 1] for example, would
represent half the maximum physical intensity with 0.5. A nonlinear one, however, might
do so at 0.75, depending on the coefficients used.
We adopt the often used convention that a prime (′ ) is used to denote a quantity that
is nonlinear in intensity and ready for display. So the nonlinear R′ G′ B′ stored in most
files contains (R′ , G′ , B′ ), while a linear intensity RGB contains (R, G, B). For sRGB the
relationship between components is well approximated by R′ ≈ R1/2.2 . When converting
to linear photometric quantities such as (X,Y, Z), the linear (R, G, B) should be used.
Linear color spaces usually use luminance, a linear photometric standard for achromatic light response, and often denoted by the symbols L and Y . The value for luminance
can be computed as a linear transform of a color value stored in linear color space. For
example, for linearized sRGB values, the luminance is:
L = 0.213R + 0.715G + 0.072B
(6.1)
Computing luminance as a linear combination of (R′ , G′ , B′ ) is not possible. Instead,
a nonlinear analog of luminance called luma is often used. There is no single standard
for luma computation (see Poynton [89] for an excellent detailed discussion of different
luma computation standards). Sometimes the symbols L or Y are also used for luma, but
we use L′ to distinguish it from luminance. Typically, luma is a linear combination of
(R′ , G′ , B′ ). For example the YCC standard uses:
L′ = 0.299R′ + 0.587G′ + 0.114B′
(6.2)
84
The nonluma axes typically encode chromatic information, and are also linear combinations of (R′ , G′ , B′ ). The color quantities are usually called chroma.
The gamut of an RGB monitor is the set of colors it can physically reproduce. In
RGB space this gamut is the unit cube [0, 1]3 . When a color space has a well-defined
mapping from RGB, the RGB gamut within that space is the shape created by sending
the entire RGB cube through that mapping. For some color spaces the boundaries of
that RGB gamut are algebraically simple, e.g. the RGB gamut of the YIQ space is a
parallelepiped. For others such as CIE L*a*b* the surfaces on the RGB gamut boundary
are algebraically complicated. When color modifications take colors outside the RGB
gamut, some method must be used to map these colors back into gamut before they can
be stored in typical RGB files. Such a method is usually called gamut mapping (for an
overview see [81]) and component by component truncation is often used [111].
6.2 The oRGB Color Space
As our interest is an opponent color space that is ideal for RGB computation, we
begin with Alvy Ray Smith’s original observation in his classic HSV paper:
Full-blown color theory is a quite complex subject, involving physics, psychology, and physiology, but restriction to the RGB monitor gamut simplifies
matters substantially [104].
Smith went on to design HSV as a simple bijective transformation of RGB that is useful
for color picking, as it has natural axes of lightness, hue, and saturation. We would
like an analogously simple space that retains these intuitive notions, but also uses the
natural axes of light-dark, red-green, and yellow-blue as suggested by perceptual theories. Unlike HSV, however, we would like to be able to perform well in computational
applications.
Inspired by Poynton’s framework [89] for how different color models deal with
CIE tristimulus values, we note how two popular color spaces, CIE L*a*b* and l αβ
operate. Both of these spaces have been used effectively in computational settings, and
unlike other opponent-like color spaces, store quantities that are nonlinear in intensity.
However, both of these spaces have complicated pipelines as shown in Figure 6.5. Each
85
L*
polynomial
a*
# # #
# # #
# # #
b*
linear
transform
l
polynomial
polynomial
logarithm
α
# # #
# # #
# # #
logarithm
β
linear
transform
logarithm
X
Y
Z
# # #
# # #
# # #
linear
transform
L
M
S
R
G
B
R
# # #
# # #
# # #
linear
transform
G
B
polynomial
R'
polynomial
G'
polynomial
B'
polynomial
R'
polynomial
G'
polynomial
B'
Figure 6.5. The pipelines of R′ G′ B′ to CIE L*a*b* (top) and R′ G′ B′ to l αβ (bottom).
of them first “linearizes” the (R′ , G′ , B′ ) to (R, G, B), does a linear transform to an intermediate linear intensity space (either XYZ or LMS), applies a component-by-component
non-linear map (either logarithmic or polynomial), and then does a linear transform to
get into an opponent-like space.
Our goal is to design a space that is useful for graphics applications. Having a
nonlinear luma channel is more perceptually uniform than luminance, and we believe it is
simpler, improves computational speed, and also works better for practical applications
(see Section 6.3.3). Therefore, instead of going through a device-independent space and
reintroducing nonlinearity, we simply apply a linear transform directly to the nonlinear
R′ G′ B′ values to get to an intermediate linear space we denote as L′C′C′ . Like the
CIE L*a*b* and l αβ chroma axes, L′C′C′ does not have true Hering-style opponent
axes. Therefore, we introduce a final rotation around the L′ axis that achieves a true
Hering-opponent model - oRGB (see Figure 6.6).
6.2.1
oRGB Step 1: Linear Transform of R′ G′ B′ to L′C′C′
There is an existing family of color spaces that are linear transforms of R′ G′ B′ . These
are those used for the video encoding of television systems including YIQ and YCC.
In a similar fashion, oRGB first transforms the RGB cube into a parallelepiped via
a linear transform. This color space, L′C′C′ , has 3 axes: white-black, yellow-blue, and
86
L'
L'
C'yb
C'rg
nonuniform
rotation
around
L' axis
C'1
C'2
R'
# # #
# # #
# # #
G'
linear
transform
B'
′ ,C′ ).
Figure 6.6. The pipeline of R′ G′ B′ (R′ , G′ , B′ ) to oRGB (L′ ,Cyb
rg
magenta-red/cyan-green. The matrix for this transformation is:
 ′ 
 
L
0.2990
0.5870
0.1140 R′
C′  = 0.5000
0.5000 −1.0000 G′ 
1
′
C2
0.8660 −0.8660
0.0000 B′
(6.3)
and its inverse:
 ′ 
 ′
R
1.0000
0.1140
0.7436
L
G′  = 1.0000
0.1140 −0.4111 C1′ 
B′
1.0000 −0.8860
0.1663 C2′
6.2.2
(6.4)
oRGB Step 2: Non-uniform Rotation Around Luma Axis L′
from L′C′C′ to oRGB
Is there an affine transform of the RGB cube that moves blue opposite yellow and
red opposite green? Unfortunately, there is not. Figure 6.7 shows the RGB cube viewed
along the white-black axis. As can be seen there is no affine transform that will make
red, white, and green collinear. The closest we can come is to put a red-magenta opposite
a green-cyan.
Figure 6.7. Orthographic view of RGB cube looking along white-black axis (left).
No affine transform will put red-white-green in line (middle). The y-axis will have a
red-magenta and a green-cyan as its extreme values (right).
87
To create a true opponent space where red is actually opposite green and orthogonal
to the yellow-blue axis, we must apply some nonaffine transform. We use the simplest
one we can think of: a constant scaling of angle between the opponent directions (see
Figure 6.8). A smoother mapping was attempted, but it made no visual difference, and
so we chose the simplest method that worked well.
′ ,C′ ) is just a compresThe transformation from L′C′C′ (L′ ,C1′ ,C2′ ) to oRGB (L′ ,Cyb
rg
sion or decompression of angles depending on which quadrant the linearly transformed
point ends up in. For points above the yellow-blue axis, the angle in the linear chroma
plane is θ = atan2(C2′ ,C1′ ). The new angle in oRGB space, θo is:
(
if θ < π /3
(3/2)θ
θo (θ ) =
π /2 + (3/4) (θ − π /3) if π ≥ θ ≥ π /3
′ ,C′ ) in oRGB we simply rotate the (C′ ,C′ ) point:
To compute the point (Cyb
rg
1 2
· ′¸
· ′ ¸
Cyb
C
= R(θo − θ ) 1′
′
C2
Crg
The inverse transformation angle θ from oRGB to L′C′C′ is:
(
(2/3)θo
if θo < π /2
θ (θo ) =
π /3 + (4/3) (θo − π /2) if π ≥ θo ≥ π /2
(6.5)
(6.6)
(6.7)
These transformations are symmetric for points below the yellow-blue axis. The
effect of this rotation along the luma axis on the gamut of the color space can be seen in
Figure 6.9.
compress
stretch
compress
stretch
Figure 6.8. To take the red and green directions to the vertical axis, we compress angles
on the blue side, and expand them on the yellow one.
88
C'2
C'rg
1.0
0.87
-1.0
1.0
C'1
-1.0
1.0
C'yb
-0.87
-1.0
Figure 6.9. Visualization of the gamut of our color space seen along the luma axis before
(left) and after (right) nonuniform rotation. After rotation, the y-axis is a true red-green
axis while keeping the yellow-blue the same.
6.2.3
Properties of oRGB Space
oRGB is a simple R′ G′ B′ -based color space that is useful for computational applications, such as color adjustment, color transformation, and color transfer. It retains
the notions of hue and saturation presented in HSV, but adds a nonlinear perceptual
brightness (similar to HSB). This similarity to HSV/HSB makes oRGB a useful space
for color picking. Unlike HSV/HSB, it encodes perceptual color opponency and its
axes encode common color naming used by people [44]. It also provides a natural
computational framework for categorizing a color by its color temperature - i.e. warm vs.
cool. Finally, the simplicity of the intermediate space L′C′C′ allows for straightforward
gamut mapping that is critical for computational applications where the manipulations
of colors often produces out-of-gamut results.
6.2.3.1
Locations of Colors
′ and C′ range in [−1, 1], while the luma channel, L′ , ranges
The chroma channels, Cyb
rg
in [0, 1]. The [L′ , 0, 0] position on the chroma plane corresponds to a neutral grey color,
that is determined by the luma L′ . In this constant chroma plane, the hue of a color is its
angle in this plane, while its saturation is its distance from the neutral point.
89
The oRGB space places the Hering opponent primaries along the four horizontal and
vertical directions. In addition, it places the opponent secondary colors (orange, magenta,
cyan, and chartreuse) exactly between the opponent axes, thus creating an intuitive space
for color modification based on common color naming (see Figure 6.10).
Studies on basic color naming agree that people across cultures often refer to colors
as warm (red or yellow) and cool (blue and green) [8]. oRGB provides a straightforward
computational framework for color selection in terms of color temperature. Colors to the
′ axis) are warm,
right and top of the magenta-chartreuse diagonal (at 45◦ from the CRG
while color to the left and bottom are cool (see Figure 6.10). Colors towards the center are
temperate. We show how this simple categorization can be applied towards cool-to-warm
shading models in Section 4.
6.2.3.2
The oRGB Gamut
The gamut of R′ G′ B′ is the cube defined by its orthogonal basis vectors - pure red,
green and blue. Linear RGB and nonlinear R′ G′ B′ spaces have identical gamut - [0, 1]3 .
When we transform the R′ G′ B′ gamut into L′C′C′ space, its (L′ ,C1′ ,C2′ ) basis vectors are
simply the columns of the matrix in Equation 6.3. The transformed gamut space is thus a
parallelepiped. Because we use unequal coefficients for determining luma values, L′C′C′
is a true parallelpiped and not a rotated cube like HSV. Since black is 0, its mapping into
L′C′C′ is also 0. Therefore the corner of the gamut cube remains at 0.
s
m
ar
w
ls
o
co
Figure 6.10. Colors to the right of the magenta-chartreuse axis are warm, to the left are
cool, and in the center are temperate.
90
Colors that fall within the volume of the transformed gamut space map to in-gamut
colors when converted back to R′ G′ B′ space. Colors that fall outside the volume require
gamut mapping.
A good gamut mapping method should first try to preserve luminance and then
attempt to preserve hue [111]. In L′C′C′ , luminance is a separate axis, so luminance
modification and preservation are straightforward. Lines of equal angle also represent the
same hue, with distance from the grey point being akin to saturation. These properties
allow for simple gamut mapping in L′C′C′ .
While our preferred editing space uses a further rotation around the luma axis of
L′C′C′ , projecting along lines of equal hue (and therefore equal angle) is equivalent to
performing the same projection in oRGB. This makes performing gamut mapping for
oRGB a straightforward, two step process.
First, we ensure that our luma values are in the range of [0, 1]. To do so, we generate
a piecewise polynomial mapping that ensures that the mean luma value does not move,
while values to the right of the mean get compressed to fit within [µ , 1], and values to the
left of the mean fit within [0, µ ]:

iβ
h

µ + (1 − µ ) (L−µ )
(Lmax −µ )
L′ (L) =
iβ
h

µ − µ (L−µ )
(L −µ )
min
if L > µ ∧ Lmax > 1
if L ≤ µ ∧ Lmin < 1
where µ is the average luma, Lmin is the minimum luma value, and Lmax is the
maximum luma value. In practice we have found β = 2/3 to work fairly well.
Second, we need to determine whether the chroma values are out-of-gamut, and if
so how to bring them into gamut. We found two approaches that work well in practice clamping and scaling.
In clamping, we take any pixel values that are outside the L′C′C′ gamut boundary
(i.e., the parallelepiped) and project them to the boundary. To do so, we trace a ray from
hL′pixel , 0, 0i to the pixel value in question, hL′pixel C′pixel C′pixel i. The ray is traveling along
the chroma plane of equal luma for that pixel. If we hit the L′C′C′ parallelepiped before
we reach the value, we know that the pixel value is outside the gamut space and needs
to be clamped to the chroma value found on the intersected boundary. Clamping to the
91
boundary is already a great improvement over first transforming from L′C′C′ to R′ G′ B′
and then performing clamping before display (see Figure 6.11). This is because clamping
to the boundary in L′C′C′ preserves both the luma and the hue of a particular value, while
clamping an R′ G′ B′ triple to [0, 1] does not.
If many pixel values are outside of the L′C′C′ gamut boundary, however, clamping
in L′C′C′ will produce a very saturated image, as many values will be truncated to the
most saturated part of the space. For this case, we scale values towards the center, i.e.,
hL′ , 0, 0i, instead of simply clamping to the boundary. To do so, we first divide the
parallelogram into planes of equal luma. For each equal luma plane, we discretize the
plane into many angular slices. For each angular slice, we compute both the furthest
distance from the grey point and the closest point on the boundary for all pixel values
that fall within the slice. We then project all pixel values within that slice so that
the point furthest out will lie on the boundary of the parallelepiped. To avoid color
discontinuities we use about 3000 angular slices per luma plane which result in visually
smooth mappings of the colors.
Using a simple linear scaling provides excellent results in practice (see Figure 6.11),
though it is possible that some more complicated nonlinear method may be necessary in
some cases. Gamut mapping via scaling is used for all oRGB results in the paper, unless
otherwise noted.
6.3 Sample Applications
We now demonstrate the utility of the oRGB color space in four different applications:
color adjustment, cool-to-warm shading, color transfer, and gamut mapping. For color
adjustment, we show how oRGB’s natural opponent axes can be used to provide a meaningful Photoshop-style “variations” tool. As oRGB’s natural primaries and secondaries
are intuitive they can aid in color selection and can serve as a natural metric for color
temperature in cool-to-warm shading. For color transfer, we demonstrate that oRGB
performs at least as well as previously suggested computational spaces. Finally, we
examine gamut mapping in the context of one computational application - color transfer.
92
(a) Original
(b) Clamp in R′ G′ B′
(c) Clamp in L′C′C′
(d) Scale in L′C′C′
Figure 6.11. The original image is modified by doubling the standard deviation in the
red-green channel resulting in out-of-gamut red values but improved green variation.
Clamping in R′ G′ B′ leaves overly saturated reds. Both clamping and scaling in L′C′C′
preserve luma, while scaling also produces a more even saturation distribution. Both
methods retain the improved green variation.
6.3.1
Color Adjustment
Users want to perform simple, yet intuitive color adjustments. Here we show how
oRGB can be used for Photoshop-style “variations” as well as for selection of colors for
shading models.
Our color space is useful for simple color contrast adjustments as we can increase the
saturation of the red-green channel independently from the yellow-blue one. To do so,
we simply have to scale the standard deviation in each channel separately. This operation
provides useful and intuitive variations results (see Figures 6.12 and 6.13). Sometimes, it
is more useful to simply take an existing image and make it “warmer” or “cooler” in tone.
This operation is straightforward in oRGB, as the two chroma channels provide a direct
mapping into a color’s temperature. Since the two chroma channels are independent, we
can adjust them without introducing global image color casts (see Figure 6.14).
green
red
93
blue
yellow
Figure 6.12. Modifying color contrast through deviation adjustment. One step down/left
scales deviation by 1/α while a step up/right scales deviation in that channel by α .
The horizontal axis is a yellow-blue standard deviation change (left lowers and right
increases) and the vertical axis is a red-green change (down lowers and up increases).
green
red
94
blue
yellow
Figure 6.13. Another example of color adjustment using deviation scaling, similar to
Figure 6.12.
green
red
95
blue
yellow
Figure 6.14. Changing the overall tone of an image through mean shifting. The natural
axes of oRGB provide intuitive shifts along primaries (yellow, red, blue, and green)
and secondaries (orange, magenta, cyan, and chartreuse). The horizontal axis is yellow
(shifting right) and blue (shifting left). The vertical axis is red (shifting up) and green
(shifting down).
96
We can also apply the concept of “variations” to material parameters. Varying the
diffuse texture via mean shifting in the chroma plane generates meaningful alternative
materials (see Figure 6.15). Shifting up produces a redder tone, up and right - an orange
tone, right - a more yellow tone, and right and down - a chartreuse tone. Similarly,
shifting left and up produces a magenta tone, left - a bluer tone, left and down - a cyan
tone, and down - a greener tone.
6.3.2
Cool to Warm Shading
Gooch et al. [39] present a cool-to-warm shading model that maps surface normal to
color, with normals pointing away from the light being associated with cool colors, and
those that point toward the light being warm. They also present a model for choosing the
green
red
colors based on intrinsic surface color. Their model is:
blue
yellow
Figure 6.15. Instead of applying mean shifting to the final image, as in Figure 6.14, we
used the intuitive oRGB axes as a color “variations” selector for the diffuse term in a car
rendering.
97
C(cos) = (1 − f (cos))Ccool + f (cos)Cwarm ,
(6.8)
where cos is the cosine between surface normal and a light direction, f is a monotonically increasing function that maps [−1, 1] to [0, 1], and Ccool and Cwarm are empirically
chosen colors. They present results for a linear f and a variety of Ccool and Cwarm .
A difficulty that arises when using this model is that there is no metric that helps users
decide how “warm” or “cool” a color is. This issue can be directly addressed in oRGB
(see Figure 6.10), as a color is warm if its dominant direction is toward the red or yellow
axes, and cool if its dominant direction is toward the blue or green axes. Quantitatively
′ ,C′ ) · (0, 1, 1) > 0 and a cool color is one where that
a warm color is one where (L′ ,Cyb
rg
dot product is negative.
To determine whether the intuitive axes of the oRGB space are helpful in NPR applications, we first added a cool-warm test to an NPR shader that uses Equation 6.8. We
also explored changes to the interpolation rule in an attempt to make NPR shaded objects
less flat looking. If the three channels are interpolated separately, then we have:
 ′ 
 

f0 (cos)Lw
(1 − f0 (cos))Lc
L
c   f (cos)Cw 
′ 
Cyb
= (1 − f1 (cos))Cyb
+ 1
yb
′
c
w
Crg
(1 − f2 (cos))Crg
f2 (cos)Crg
(6.9)
c ,Cc ) and (Lw ,Cw ,Cw ) are the oRGB components of C
where (Lc ,Cyb
cool and Cwarm ,
rg
yb rg
respectively. This will be a valid cool to warm mapping provided the end colors are
warm and cool, and that all of fi are strictly nondecreasing and fi (−1) = 0, fi (1) = 1). If
the two chroma channels are handled as in Gooch et al. [39] and the luma channel is an
average between linear interpolation and cosine-based interpolation (Gouraud lighting),
then an NPR look can be retained while also having some visual character of diffuse
lighting (Figure 6.16). Our point is not that this is a better NPR model, but that oRGB
provides a space where reasoning about NPR model design is fairly natural.
6.3.3
Color Transfer
For color transfer, Reinhard et al. [91] demonstrated that the l αβ color space [94]
provides excellent results when compared to the RGB color space. While l αβ works
well in many situations, oRGB provides features we feel are lacking in l αβ . For example,
98
Figure 6.16. Retaining an NPR style while maintaining the diffuse lighting look. The
luma channel in oRGB is modified to partially resemble traditional Gouraud style shading while maintaining a cool-to-warm transition (top). Shading from Gooch et al. [39]
(bottom).
color transfer in oRGB allows for independent manipulation of the luma, pure yellowblue, and pure red-green channels. Thus, the statistical properties of the channels can be
transfered in a space that is meaningful in terms of natural primaries and secondaries.
We compare the color transfer results produced by oRGB (see Figures 6.17 and
6.18) with those of three different Hering-style opponent color models - Schwarz [101],
RBW [103], and RGBY [123]. We show oRGB results that use no gamut mapping or
use gamut scaling. In addition, we compare against Adobe Photoshop’s “Color Match”
function, as well as HSV, l αβ , CIE L*a*b* and YCC.
In oRGB shifting red towards blue produces purple, while retaining yellow (Figure 6.17), and shifting magenta towards red results in true red (Figure 6.18). The opponent spaces we compare against also have orthogonal axes, and the hue shifts in these
spaces seem to produce reasonable results. However, the saturation and brightness in
these spaces do not seem to perform as well, perhaps as a result of the fact that they
operate in linear space. In Figure 6.19 we show the importance of running in nonlinear
space for oRGB. The rest of the spaces do not have orthogonal red-green and yellow-blue
axes, and therefore independent shifting and scaling results in mixing some of the color
from the other axis (e.g., the magenta cast for l αβ in Figure 6.18).
It should be noted that orthogonality is not the full story for color transfer. RGB
99
(a) Original Image
(b) Target Image
(c) oRGB none
(d) oRGB scale
(e) HSV
(f) Photoshop
(g) l αβ
(h) Schwarz
(i) RBW
(j) RGBY
(k) CIE L*a*b*
(l) YCC
Figure 6.17. The statistics of the target image are transferred to the original image in a
variety of color spaces. Transferring red towards blue produces purple in oRGB instead
of yellows, reds, or browns produced by the other spaces.
100
(a) Original Image
(b) Target Image
(c) oRGB none
(d) oRGB scale
(e) HSV
(f) Photoshop
(g) l αβ
(h) Schwarz
(i) RBW
(j) RGBY
(k) CIE L*a*b*
(l) YCC
Figure 6.18. The statistics of the target image are transferred to the original image in a
variety of color spaces. A magenta shifted towards pure red becomes pure red in oRGB
instead of remaining magenta.
has orthogonal axes, but unfortunately for natural images, it has highly correlated axes
restricting its use for color transfer [91]. While l αβ produces decorrelated axes using
PCA, oRGB attempts to utilize opponent-color perceptual axes for meaningful computational applications.
Color transfer is a complex operation that is highly dependent on the specific image
distributions. We do not believe there is one color space that can consistently provide the
best results. However, we believe oRGB performs at least as good, and in many cases better, than other computational color spaces, based on the set of images we have tested.
101
Figure 6.19. Linear version of our model for color transfer (original and target images
are the same as the ones used in 6.17 and 6.18).
6.3.4
Gamut Mapping
Gamut mapping is a necessary operation for computational applications, as values
are often shifted and scaled, thus increasing the chance for producing out-of-gamut pixel
values. Because oRGB is a linear transform of R’G’B’ followed by a rotational distortion,
the gamut boundary is simpler than for most other spaces. While this does not necessarily
guarantee fewer out-of-gamut values, it allows us to correct such cases more easily than
in CIE L*a*b* or l αβ .
One computational application, color transfer, works by shifting and scaling groups
of pixels without regard to gamut boundaries. While any image used for transferring
statistics will have a well defined mean in the RGB gamut space, some values towards
the tails of the distributions may land outside the RGB gamut following transfer.
To provide a comparison metric for the out-of-gamut results produced by different
color spaces during color transfer, we did the following test. We took the top 12 images
produced by Google images, using “family vacation” as a search term. We then did color
transfers of the 12 photos to each other, producing 132 nontrivial transfer results per
color space. Table 6.1 summarizes the averaged results for each of the color spaces.
CIE L*a*b* and l αβ produce a similar number of out-of-gamut values to oRGB,
however, they tend to generate results that are further from the RGB gamut boundary.
For these spaces, simply clipping once the color transfer result is returned to RGB can be
disastrous. While oRGB has very low out-of-gamut values by comparison, we still seek
102
Table 6.1. Different color spaces tend to produce different out-of-gamut results. The first
column lists the color space tested. The second column shows the average percentage
of out-of-gamut pixels. The remaining three columns demonstrate the out-of-gamut
percentage range per RGB channel.
Color Space % Pixels
%R
%G
%B
oRGB
10.36
22.58
16.59
26.42
HSV
9.07 121.87 122.12
118.9
l αβ
10.46 104.54
40.56
25.02
16.45 >999.9 >999.9 >999.9
Schwarz
RBW
11.95 146.29 124.20 154.94
RGBY
13.67 296.99 234.07 174.70
CIE L*a*b*
10.89 117.82
66.05
38.33
YCC
11.24
22.99
15.99
25.91
to remove these remaining out-of-gamut values via gamut mapping.
Having the ability to perform gamut mapping should not be ignored. The l αβ and
CIE L*a*b* results in Figure 6.17 produce results that are wildly out of the RGB gamut,
thus resulting in unpredictable and unexpected color transfer results.
6.4 Conclusion
The oRGB color space we have introduced has two basic merits. First, it is a relatively simple transformation of R’G’B’, so it is efficient, easy to implement, and has a
reasonably simple gamut. Second, it has orthogonal axes that are aligned with Hering’s
opponent color pairs.
We have shown empirically that this space can be useful in some common computer
graphics computations. We suspect the space works well for three reasons. First, the
gamma encoding of (R′ , G′ , B′ ) values serves the same role as the non-linearities in
models such as CIE L*a*b*. Second, the simple gamut boundary reduces out-of-gamut
issues and allows any that remain to be easily corrected. Third, we believe that for
computer graphics applications, there is something natural about the Hering primaries
and their opponency.
There are several limitations of the oRGB space. As a computer graphics space, it
is not well-suited to traditional colorimetric applications where all possible colors, as
103
opposed to all displayable colors, are represented. In applications where the Hering primary colors are not central, it is unlikely to have advantages over linear transformations
of R’G’B’ such as YCC. Perhaps its biggest limitation is that it is not a pure bijection of
RGB, so it is not the perfect space for choosing colors for RGB displays.
CHAPTER 7
CONCLUSION AND FUTURE WORK
7.1 Conclusions
In this dissertation we derive principles applicable to the map design of the mountain
panorama map, a perceptually and artistically enhanced cartographic painting of terrain.
We codify a set of useful rules based on detailed analysis and comparison of two example
panorama map paintings of Yellowstone National Park. The most obvious limitation
of our analysis is the fact that we concentrate on the function performed by the visual
elements in the painting, but we do not discuss how the geometric features themselves
could be manipulated for enhanced visual clarity, both in terms of their abstraction, as
well as spatial positioning.
We also present a proof-of-concept rendering system for panorama map production,
relying on prior work in nonphotorealistic rendering as well as the rich knowledge and
heuristics developed by the art community. Our method uses a hybrid image-object space
approach, in that the actual rendering is done in image space guided by relevant 3D
information we collect from our geometric model. Image space is easier to use, allows
us to combine graphic elements with ease, and makes blending and overlaying of textures
and colors a very straightforward operation. Our method is guided by such operations,
because it is based on traditional landscape panoramic maps who were created in the
image space domain. By using image space, we do not have to be concerned with the
complexity and connectivity of the geometric data, nor do we run the risk of performing
computations which will later be discarded due to lack of visibility (common in terrains).
In addition, for large scenes, the computational complexity of manipulating data in
object space far outweighs that required of a fixed-size image space. The most obvious
limitation of this work is that we have not yet created panorama images that match
105
the visual quality of paintings by artists such as Heinrich Berann or James Niehues.
However, we believe our results show a lot of promise, and are a first step at addressing
the complex artistic, cartographic, and illustration principles involved in the creation of
the panorama map, considered by many one of the most difficult cartographic endeavors.
While choosing a meaningful viewpoint automatically is not required for the successful rendering of a panorama, it is a useful and practical extension to the panorama map
rendering system we presented. We proposed a set of criteria based on utility, perception,
and aesthetics, and showed how one may use the criteria to perform a multidimensional
camera parameter optimization. The results seem promising in selecting fairly reasonable views of the complex terrain scenes. The biggest limitation of that work is that
we have not yet performed a user study to validate the criteria and show the relative
importance of the different proposed parameters.
And finally, we resolved the complexity of picking appropriate colors for background, base strokes, and brushstrokes for novel panorama map rendering by applying
color transfer from a desired image in a novel color space, oRGB. We also demonstrated
the usefulness of the new color space for various computational applications, such as
color adjustment, cool-to-warm shading, and gamut mapping. Again, we note that this
color space is not well suited to traditional colorimetric applications, as it is a display
space by design.
7.2 Future Work
In this section we discuss several avenues for future research related to the automation
of panorama maps, as presented in this dissertation.
7.2.1
Geometric Manipulation
While maintaining the accuracy of the geometry of the terrain is often very important
to cartographers, there are cases when enhancing the legibility of certain recognizable
features is essential to improving the perceptual quality of the map. Making features
more visible often means making them larger relative to the features that surround them.
We see this to be somewhat similar to the principle of vertical exaggeration, where it is
the feature’s height that is increased in the direction of the normal in order to make the
106
feature more noticeable.
Deciding which features are important, and therefore are to be manipulated, should
depend on user input, as it the user that can assign them special meaning. It is applying
the geometric modifications judiciously and with the proper parameter settings that will
take the bulk of this future work. It is very probable that such geometry enhancement
can be automated, as long as one can establish what types of manipulations can be
meaningfully applied to specific feature types, and figure out how to apply them without
introducing unacceptable surface discontinuities.
For example, peaks are naturally manipulated via exaggeration. To show an example of this principle, we seamlessly exaggerated a peak situated on the south side
of Little Cottonwood Canyon, Salt Lake City, UT. We used 30m DEM dataset from
Utah’s Dromedary Quad acquired from the USGS, and wrote a program that modifies
the geometric data set in 3D. The boundary of the modified feature with the surrounding
terrain is perceptually smooth. The peak was exaggerated via the use of a nonlinear cubic
Bezier curve (see Figure 7.1).
Figure 7.1. At the top, we have a rendering of the original dataset. At the bottom, we
have a rendering that exaggerates one of the peaks in Little Cottonwood Canyon.
107
7.2.2
Enhanced Spatial Layout
Some panorama artists like Heinrich Berann often manipulate the overall spatial layout of the geometry of the panorama map in order to enhance the legibility of important
cartographic as well as man-made features. He would commonly modify narrow valleys
and make them wider so that the views are not obstructed. Furthermore, he would rotate
important mountains, or move them to more prominent location altogether.
A viable approach to accomplishing such manipulation of the spatial layout of height
fields is image warping. Image warping applies 2D geometric transformations given
feature correspondence constraints between two images. Then, a warping function over
the whole image plane can be derived, which can take the original image from one spatial
domain - the source shape, to another - the target shape. Feature correspondences can be
specified by pairs of points, curves, line segments, mesh nodes, etc.
There are many image warping techniques. Wolberg uses mesh warping [127], Beier
and Neely [7], Johan et al. [57], and Birkholz and Jackel [10] use field morphing, Arad radial basis functions [3], Litwinowicz and Williams - thin plate splines [75], Lee et al.
- energy minimization [70] and multilevel free-form deformations (MFFD) [72], Arad grid-distortion on non-rectangular grids [2].
The main contribution of this work will involve finding ways to set the warping constraints and specify the feature correspondences automatically, as feature specification
for image warping often involves fairly tedious manual input.
7.2.3
Landform Generalization
A further improvement to the panorama map generation will involve utilizing the
cartographic principle of landform generalization. In the cartographic literature, generalization refers to the process of reducing the amount of detail in a map such that
distinguishing characteristics of the terrain are preserved, while unimportant features are
removed, thus enhancing clarity [65].
Cartographers have suggested conceptual models in an attempt to solve the generalization problem. Weibel [124] discusses strategies for computer-assisted terrain generalization that is adaptive to different terrain types, scales, and map purposes. He
provides recommendations for appropriate parameter settings in relation to scale, terrain
108
character, as well as other controls related to generalization. McMaster and Shea [80]
also approach the practice of generalization and propose concepts that attempt to explain
why, when, and how should one generalize. The art and illustration literature also
has useful heuristics related to landscape generalization and landscape sketching and
painting [37, 41, 58].
To our knowledge, there hasn’t been much work related to developing automatic
algorithms based on these heuristics in the computer graphics literature, and especially
ones that will be applicable to the panorama map. An excellent starting point, however,
is the work by Tasdizen and Whitaker, where the two-step, variational and feature preserving smoothing method for terrain data produces results that appear similar to certain
artistic drawings of terrain [114].
REFERENCES
[1] A LOIMONOS , J. Shape from texture. Biological Cybernetics 58 (1988), 345–360.
[2] A RAD , N. Grid-distortion on non-rectangular grids. CAGD 15, 5 (1998), 475–
493.
[3] A RAD , N., DYN , N., R EISFELD , D., AND Y ESHURIN , Y. Image warping by
radial basis functions: Application to facial expressions. CVGIP: Graphical
Models and Image Understanding 56, 2 (1994), 161–172.
[4] A RNHEIM , R. The Power of the Center: A Study of Composition in the Visual
Arts. Berkeley, CA: University of California Press, 1982.
[5] A RT, D.
Winter in Flaine, France.
http://mappery.com/fullsize-name/Flaine-Map.
In mappery collection,
[6] BARRAL , P., D ORME , G., AND P LEMENOS , D. Scene understanding techniques
using a virtual camera. Computer Graphics Forum 19 (2000).
[7] B EIER , T., AND N EELY, S. Feature-based image metamorphosis. In SIGGRAPH
’92: Proceedings of the 19th Annual Conference on Computer Graphics and
Interactive Techniques (1992), pp. 35–42.
[8] B ERLIN , B., AND K AY, P. Basic Color Terms. Their Universality and Evolution.
Berkeley, CA: University of California Press. Reprinted 1991, 1969.
[9] B ERTIN , J. Semiology of Graphics: Diagrams, Networks, Maps. Madison, WI:
University of Wisconsin Press, 1983.
[10] B IRKHOLZ , H., AND JACKEL , D. Image warping with feature curves.
Proceedings of the 19th spring conference on Computer graphics (2003).
In
[11] B OARD , C. Report on the working group on cartographic definitions. Cartographic Journal 29 (1990), 65–69.
[12] B RADFORD ,
C.
James
http://www.skimaps.com/article6.html.
Niehues
talks
back.
[13] B RATKOVA , M., B OULOS , S., AND S HIRLEY, P. oRGB: A practical opponent
color space for computer graphics. Computer Graphics & Applications 29, 1
(2009).
110
[14] B RATKOVA , M., S HIRLEY, P., AND T HOMPSON , W. B. Artistic rendering of
mountainous terrain. ACM Transactions on Graphics 28, 4 (2009).
[15] B RATKOVA , M., T HOMPSON , W. B., AND S HIRLEY, P. Automatic views of
natural scenes. In Computational Aesthetics in Graphics, Visualization, and
Imaging (2009).
[16] B ROOKS , M. Two results concerning ambiguity in shape from shading. In AAAI83 (1983), pp. 36–39.
[17] B UCHIN , K., S OUSA , M. C., D OLLNER , J., S AMAVATI , F., AND WALTHER , M.
Illustrating terrains using direction of slope and lighting. In 4th ICA Mountain
Cartography Workshop (2004).
[18] C ABRAL , B., AND L EEDOM , L. C. Imaging vector fields using line integral
convolution. In SIGGRAPH ’93: Proceedings of the 20th Annual Conference on
Computer Graphics and Interactive Techniques (1993), ACM Press, pp. 263–270.
[19] C HICHILISKY, E., AND WANDELL , B. A. Trichromatic opponent color classification. Vision Research 39, 20 (1999), 3444–58.
[20] C HRISTIE , M., AND O LIVIER , P. Camera control in computer graphics. Computer Graphics Forum 25, 3 (2006).
[21] C LIFTON , J. The Eye of the Artist. Westport, CT: North Light Publishers, 1973.
[22] C ONWAY, B. Neural Mechanisms of Color Vision. Boston, MA: Kluwer Academic Publishers, 2002.
[23] C UTTING , J., AND M ILLARD , R. Three gradients and the perception of flat and
curved surfaces. Jounrnal of Experimental Psychology: General 113, 2 (1984),
198–216.
[24] DA V INCI , L. The Notebooks of Leonardo Da Vinci. Old Saybrook, CT: William
S. Konecky Associates, September 2003.
[25] D E C ARLO , D., AND S ANTELLA , A. Stylization and abstraction of photographs.
In SIGGRAPH ’02: Proceedings of the 29th Annual Conference on Computer
Graphics and Interactive Techniques (2002), ACM Press, pp. 769–776.
[26] D EUSSEN , O., H AMEL , J., R AAB , A., S CHLECHTWEG , S., AND S TROTHOTTE ,
T. An illustration technique using hardware-based intersections and skeletons.
In Proceedings of the 1999 Conference on Graphics Interface (1999), Morgan
Kaufmann Publishers Inc., pp. 175–182.
[27] D ORLING , D., AND FAIRBAIRN , D. Mapping: Ways of representing the world.
In Insights into Human Geography, P. Knox and S. Smith, Eds. Harlow, Essex,
England: Longman, 1997.
[28] E DDY, G. Summer in lake Tahoe, California, USA. In panorama-map collection,
http://www.panorama-map.com/Eddy/Eddylaketahosmall500.html.
111
[29] E LAM , K. Geometry of Design: Studies in Proportion and Composition. New
York, NY: Princeton Architectural Press, 2001.
[30] E LBER , G. Line art illustrations of parametric and implicit forms. IEEE Transactions on Visualization and Computer Graphics 4, 1 (1998), 71–81.
[31] FAIRCHILD , M. D. Color Appearance Models, 2 ed. Reading, MA: AddisonWesley, 2005.
[32] F ECHNER , G. T. Zur experimentalen Aesthetik. Leipzig: Hirzel, 1871.
[33] F EIXAS , M., S BERT, M., AND G ONZ ÁLEZ , F. A unified information-theoretic
framework for viewpoint selection and mesh saliency. In ACM Transactions on
Applied Perception (2008).
[34] F LEISHMAN , S., C OHEN -O R , D., AND L ISCHINSKI , D. Automatic camera
placement for image-based modeling. In Pacific Graphics (1999).
[35] F RANK , A. U., AND E GENHOFER , M. J. Computer cartography for GIS: An
object oriented view of the display transformation. Computers and Geoscience
18, 8 (1992), 975–987.
[36] G ARDING , J. Shape from texture and contour by weak isotropy. Journal of
Artificial Intelligence 64 (1993), 243–297.
[37] G AYTON , R. R. Artist Outdoors Field Sketching in the American West. New
York, NY: Prentice Hall, 1987.
[38] G IBSON , J. J. The Perception of the Visual World. Cambridge, MA: Riverside
Press, 1950.
[39] G OOCH , A., G OOCH , B., S HIRLEY, P., AND C OHEN , E. A non-photorealistic
lighting model for automatic technical illustration. In SIGGRAPH ’98: Proceedings of the 25th Annual Conference on Computer Graphics and Interactive
Techniques (1998), ACM Press, pp. 447–452.
[40] G OOCH , B., R EINHARD , E., M OULDING , C., AND S HIRLEY, P. Artistic composition for image creation. In Proceedings of the 12th Eurographics Workshop
on Rendering Techniques (2001).
[41] G OULD , T. An Introduction to Painting Landscapes: Techniques, Light, Color,
Composition, Style, reprinted edition; original edition 1994 by quintet publishing ed. Ediso, NJ: Chartwell Books, 2004.
[42] G RIMM , C. Post-rendering composition for 3d scenes. In Eurographics (2001).
[43] H AEBERLING , C. Selected design aspects and graphic variables for 3d mountain
maps. In Proceedings, 4th Mountain Cartography Workshop, Vall de Nuria, 30th
September - 2nd October 2004 (2004), no. 8, pp. 109–118.
112
[44] H ARDIN , C. L. Basic color terms and basic color categories. In Color Vision:
Perspectives from Different Discplines, W. G. K. Backhaus, R. Kliegl, and J. S.
Werner, Eds. Berlin; New York: Walter De Gruyter, 1998, pp. 207–218.
[45] H ERRING , J. The Painter’s Composition Handbook. Clint, TX: Poor-Henry
Publishing Company, 1971.
[46] H ERTZMANN , A. Painterly rendering with curved brush strokes of multiple sizes.
In SIGGRAPH ’98: Proceedings of the 25th Annual Conference on Computer
Graphics and Interactive Techniques (1998), ACM Press, pp. 453–460.
[47] H ERTZMANN , A., AND Z ORIN , D. Illustrating smooth surfaces. In SIGGRAPH
’00: Proceedings of the 27th Annual Conference on Computer Graphics and
Interactive Techniques (2000), ACM Press.
[48] H ORN , B. Hill shading and the reflectance map. Proc. IEEE 23, 1 (1981).
[49] H SU , S. C., L EE , I. H. H., AND W ISEMAN , N. E. Skeletal strokes. In UIST
’93: Proceedings of the 6th Annual ACM Symposium on User Interface Software
and Technology (1993), ACM Press, pp. 197–206.
[50] H UNT, R. The Reproduction of Colour. Chichester, West Sussex, England;
Hoboken, NJ: John Wiley & Sons, 2004.
[51] H URNI , L., DAHINDEN , T., AND H UTZLER , E. Digital cliff drawing for
topographic maps – traditional representations by means of new technologies.
Cartographica 38, 1/2 (2001), 55–65.
[52] H URNI , L., N EUMANN , A., AND H UTZLER , E. Digital cliff drawing for
topographic maps. In ICA Commission on Mountain Cartography (1999).
[53] H URVICH , L., AND JAMESON , D. An opponent-process theory of color vision.
Psychol Rev 64 (1957), 384–404.
[54] I MHOF, E. Zürich, Vorhof der Alpen. Zürich: Orell Füssli, 1963, ch. Zurcher
Kartenkustler und Panoramazeichner, pp. 105–138.
[55] I MHOF, E. Cartographic Relief Presentation. Berlin; New York, NY: Walter De
Gruyter, 1982.
[56] I NTERRANTE , V. Illustrating surface shape in volume data via principal directiondriven 3d line integral convolution. In SIGGRAPH ’97: Proceedings of the 24th
Annual Conference on Computer Graphics and Interactive Techniques (1997),
ACM Press, pp. 109–116.
[57] J OHAN , H., KOISO , Y., AND N ISHITA , T. Morphing using curves and shape
interpolation techniques. In Pacific Graphics (2000), pp. 348–358.
[58] J OHNSON , C. The Sierra Club Guide to Sketching in Nature, 2 ed. New York,
NY: Sierra Club Books and Crown Publishers, 1997.
113
[59] J ONES , C. Geographical Information Systems and Computer Cartography. Harlow, Essex, England: Longman, 1997.
[60] K AMADA , T., AND K AWAI , S. A simple method for computing general position
in displaying three-dimensional objects. Computer Vision, Graphics, and Image
Processing 41, 16 (January 1988), 43–56.
[61] K IRKPATRICK , S., G ELATT, C. D., AND V ECCHI , M. P. Optimization by
simulated annealing. Science 220, 4598 (1983).
[62] K NILL , D. Perception of surface contours and surface shape: From computation
to psychophysics. JOSA A 9, 9 (1992).
[63] KOENDERINK , J. J., AND VAN D OORN , A. J. The structure of relief. Advance
in Imaging and Electron Physics 103 (1998), 65–150.
[64] KOLLER , L. Summer in Ober Egadin, Switzerland. In panorama-map collection,
http://www.panorama-map.com/Europeans/Koller/kolleroberengadin1000.html.
[65] K RAAK , M. J., AND O RMELING , F. J. Cartography - visualization of spatial
data. In 7th International Symposium on Spatial Data Handling (1996), pp. 53–
54.
[66] K RAGES , B. Photography: The Art of Composition. New York, NY: Allworth
Press, 2005.
[67] K UEHNI , R. G. Color Space and Its Divisions. Hoboken, NJ: J. Willey & Sons,
2003.
[68] K UEHNI , R. G. Color: An Introduction to Practice and Principles, second ed.
Hoboken, NJ: J. Willey & Sons, 2004.
[69] L ALO , C. L’Esthetique Experimentale Contemporaine [Contemporary experimental aesthetics]. Paris: Alcan, 1908.
[70] L EE , C HWA , H AHN , AND S HIN. Image morphing using deformation techniques.
Journal of Visualization in Computer Animation 7 (1996), 3–23.
[71] L EE , C. H., VARSHNEY, A., AND JACOBS , D. W. Mesh saliency. In SIGGRAPH
’05: Proceedings of the 32nd Annual Conference on Computer Graphics and
Interactive Techniques (2005).
[72] L EE , S.-Y., C HWA , K.-Y., AND S HIN , S. Y. Image metamorphosis using snakes
and free-form deformations. In SIGGRAPH ’95: Proceedings of the 22nd annual
conference on Computer graphics and interactive techniques (1995), pp. 439–448.
[73] L ESAGE , P. L., AND V ISVALINGAM , M. Towards sketch-based exploration of
terrain. Computers & Graphics 26 (2002), 309–328.
114
[74] L EUNG , T., AND M ALIK , J. On perpendicular texture or: Why do we see more
flowers in the distance? In IEEE Conference on Computer Vision and Pattern
Recognition (June 1997), pp. 807–813.
[75] L ITWINOWICZ , P., AND W ILLIAMS , L. Animating images with drawings. In
SIGGRAPH ’94: Proceedings of the 21st annual conference on Computer graphics and interactive techniques (1994), pp. 409–412.
[76] L ONDON , B., U PTON , J., S TONE , J., KOBRE , K., AND B RILL , B. Photography.
Upper Saddle River, NJ: Prentice Hall, 2005.
[77] M ACKANESS , W. Automated cartography and the human paradigm. In Cartographic Design Theoretical and Practical Perspectives. Hoboken, NJ: John Wiley
& Sons, 1996, pp. 55–66.
[78] M ALIK , J., AND ROSENHOLTZ , R. Computing local surface orientation and
shape from texture for curved surfaces. IJCV 23, 2 (1997), 149–168.
[79] M ARR , D. Vision: A Computational Investigation into the Human Representation
and Processing of Visual Information. San Francisco, CA: W. H. Freeman, 1982.
[80] M C M ASTER , AND S HEA. Generalization in Digital Cartography. Washington,
DC: Association of American Cartographers, 1992.
[81] M OROVIC , J., AND L UO , M. R. The fundamentals of gamut mapping: A survey.
Journal of Imaging Science and Technology 45, 3 (2001), 283–290.
[82] NAIMAN , A. C. Color spaces and color contrast. The Visual Computer 1, 3
(1985), 194–201.
[83] N IEHUES , J. The mapmaker: Jim Niehues, painting mountain majesties.
http://www.jamesniehues.com/article.htm.
[84] PALMER , S. E. Vision Science: Photons to Phenomenology. Cambridge, MA:
MIT Press, 1999.
[85] PARGETER. Northwest Passages, Washington, USA. In panorama-map collection, http://www.panorama-map.com/pargeter/pargeterpassage1000.html.
[86] PATTERSON , T. A view from on high: Heinrich Berann’s panoramas and landscape visualization techniques for the us National Park Service. Cartographic
Perspectives, 36 (2000).
[87] PATTERSON , T. Looking closer: A guide to making bird’s-eye views of national
park service cultural and historical sites. Cartographic Perspectives, 52 (2005).
[88] P OLONSKY, O., PATANE , G., B IASOTTI , S., G OTSMAN , G., AND S PAGNUOLO ,
M. What’s in an image? Visual Computer 21 (2005), 840–847.
[89] P OYNTON , C. Color science and color appearance models for cg, hdtv, and dcinema, 2004. SIGGRAPH course notes.
115
[90] P REMOZE , S. Computer generation of panorama maps. In 3rd ICA Mountain
Cartography Workshop (2002).
[91] R EINHARD , E., A SHIKHMIN , M., G OOCH , B., AND S HIRLEY, P. Color transfer
between images. Computer Graphics & Applications 21, 5 (September-October
2001), 2–9.
[92] R ICHTER , I. A., Ed. The Notebooks of Leonardo da Vinci, world’s classics ed.
Oxford: Oxford University Press, 1998.
[93] ROSENHOLTZ , R., AND M ALIK , J. Surface orientation from texture: Isotropy or
homogeneity (or both)? Vision Research 37, 16 (August 1997), 2283–2293.
[94] RUDERMAN , D. L. Statistics of cone responses to natural images: Implications
for visual coding. J. Opt. Soc. Am. A 15, 8 (August 1998), 2036–2045.
[95] RUSINKIEWICZ , S., B URNS , M., AND D E C ARLO , D. Exaggerated shading for
depicting shape and detail. ACM Transactions on Graphics 25, 3 (July 2006),
1199–1205.
[96] S AITO , T., AND TAKAHASHI , T. Comprehensible rendering of 3-d shapes.
In SIGGRAPH ’90: Proceedings of the 17th Annual Conference on Computer
Graphics and Interactive Techniques (1990), pp. 197–206.
[97] S ALISBURY, M., W ONG , M., H UGHES , J., AND S ALESIN , D. Orientable textures for image-based pen-and-ink illustration. In SIGGRAPH ’97: Proceedings
of the 24th Annual Conference on Computer Graphics and Interactive Techniques
(1997), ACM Press, pp. 401–406.
[98] S AUNDERS , J. A. The effect of texture relief on perception of slant from texture.
Perception 32 (2003), 211–233.
[99] S BERT, M., P LEMENOS , D., F EIXAS , M., AND G ONZ ÁLEZ , F. Viewpoint
quality: Measures and applications. In Computational Aesthetics in Graphics,
Visualization and Imaging (2005), L. Neumann, M. Sbert, B. Gooch, and W. Purgathofer, Eds.
[100] S CHULZKE , M. A Painter’s Guide to Design and Composition. Cincinnati, OH:
North Light Books, 2006.
[101] S CHWARZ , M. W., C OWAN , W. B., AND B EATTY, J. C. An experimental
comparison of rgb, yiq, lab, hsv, and opponent color models. ACM Trans. Graph.
6, 2 (1987), 123–158.
[102] S HEKHAR , S. Encyclopedia of GIS. New York, NY: Springer, 2007.
[103] S HIRRIFF , K. W. The RBW color model. Computers and Graphics 17, 5 (1993),
597–602.
116
[104] S MITH , A. R. Color gamut transform pairs. In SIGGRAPH ’78: Proceedings
of the 5th Annual Conference on Computer Graphics and Interactive Techniques
(1978), pp. 12–19.
[105] S OKOLOV, D., AND P LEMENOS , D. Viewpoint quality and scene understanding.
In VAST (2005), M. Mudge, N. Ryan, and R. Scopigno, Eds.
[106] S OKOLOV, D., AND P LEMENOS , D. Virtual world explorations by using topological and semantic knowledge. The Visual Computer 24, 3 (2008), 173–185.
[107] S OKOLOV, D., P LEMENOS , D., AND TAMINE , K. Viewpoint quality and global
scene exploration strategies. In GRAPP (2006).
[108] S TEVENS , K. The visual interpretation of surface contours. AI 17 (1981), 47–73.
[109] S TEVENS , K. A. Surface Perception from Local Analysis of Texture and Contour.
PhD thesis, Cambridge, MA: MIT Press, 1979.
[110] S TOEV, S. L., AND S TRASSER , W. A case study on automatic camera placement
and motion for visualizing historical data. In IEEE Visualization 2002, Boston,
MA (2002).
[111] S TONE , M. C., C OWAN , W. B., AND B EATTY, J. C. Color gamut mapping and
the printing of digital color images. ACM Trans. Graph. 7, 4 (1988), 249–292.
[112] S ULLIVAN , C. Drawing the Landscape. Hoboken, NJ: J. Willey & Sons, 1997.
[113] S UPER , B., AND B OVIK , A. Shape from texture using local spectral moments.
IEEE Trans. on PAMI 17, 4 (1995), 333–343.
[114] TASDIZEN , T., AND W HITAKER , R. Feature preserving variational smoothing of
terrain data. In IEEE Workshop on Variational, Geometric and LevelSet Methods
in Computer Vision (October 2003).
[115] U NKNOWN. Summer in Grachen, Switzerland. In alpineskimaps collection,
http://www.alpineskimaps.com/swit/grachen/summap.htm.
[116] U NKNOWN.
Winter in Alpbach, Austria.
In alpineskimaps collection,
http://www.alpineskimaps.com/htm/austria/alpbach/winmap.htm.
[117] VALBERG , A. Light, Vision, Color. Hoboken, NJ: John Wiley & Sons, 2005.
[118] VAZQUEZ , P.-P., F EIXAS , M., S BERT, M., AND H EIDRICH , W. Viewpoint
selection using viewpoint entropy. In VMV, Stuttgart, Germany (2001).
[119] V IELKIND , H. Panoramastudio vielkind. innsbruck., 1998.
[120] V ISVALINGAM , M., AND D OWSON , K. Algorithms for sketching surfaces.
Computers & Graphics 9 (1998), 213–228.
117
[121] VOLBRECHT, V. J., AND K LIEGL , R. The perception of blackness: An historical
and contemporary review. In Color Vision: Perspectives from Different Discplines, W. G. K. Backhaus, R. Kliegl, and J. S. Werner, Eds. Berlin; New York:
Walter De Gruyter, 1998, pp. 187–206.
[122] WALD , I., B OULOS , S., AND S HIRLEY, P. Ray tracing deformable scenes using
dynamic bounding volume hierarchies. TOG 26, 1 (2007).
[123] WARE , C., AND C OWAN , W. The RGYB color geometry. ACM Trans. Graph. 9,
2 (1990), 226–232.
[124] W EIBEL , R. Models and experiments for adaptive computer-assisted terrain
generalization. Cartography and Geographic Information Systems 19, 36 (1992),
133–153.
[125] W HELAN , J., AND V ISVALINGAM , M. Formulated silhouettes for sketching
terrain. In Theory and Practice of Computer Graphics (2003), IEEE, p. 90.
[126] W ITKIN , A. P. Recovering surface shape and orientation from texture. Journal
of Artificial Intelligence 17 (1981), 17–45.
[127] W OLBERG , G. Digital Image Warping. Los Alamitos, CA: IEEE CS Press, 1990.
[128] W U , H., PAN , M., YAO , L., AND L UO , B. A partition-based serial algorithm
for generating viewshed on massive dems. International Journal of Geographical
Information Science 21, 9 (2007), 955–964.