Imagery
CS 110
Fall 2005
Image Technology Always
Changing

Paint / pencil
• Cave drawings
• Perspective renditions
• Impressionism

Optics
• Photographs
• Photo volumes that shift lighting, time,
and space
Early Imagery: In the Mind’s Eye
Improving
Imagery

Equations dictate
image
Mastering Perspective
Lorenzetti (1342)
Rules for a New Era

Photorealism
Perspective, therefore, should be
preferred above all man’s discourses
and disciplines. (Da Vinci, 1400)
Image Generation Technology

Camera Obscura
• 15th Century
Camera’s Inhibit Art?



There are some who… trace outlines on a
surface of a transparent medium.
Such an invention is to be condemned in
those who do not know how to portray
things without it, nor how to reason about
nature with their mind.
They are always poor and mean in every
invention and in the composition of
narratives, which is the final aim of this
science.
(Da Vinci)
Can a Camera do This?

Artistic control
of vanishing
point
(Da Vinci, 1498)
Can a Camera do This?


Yes!
We’ll discuss
exciting new
technologies
(Da Vinci, 1498)
Photographs vs. Moments


Photograph: an (almost objective)
representation of a point in time and
space
Moment: a visual representation of the
subjective reality at some time and place
From M. Cohen, Microsoft Research
http://research.microsoft.com/~cohen/TheMoment.pdf
http://research.microsoft.com/~cohen/FindingMagicInAnImageStack.pdf
Photographs vs. Moments

Artists capture moments so well
Photographs vs. Moments



With introduction of cameras, art
became more abstract (to
compensate)
Have cameras lost the moment?
Can we recapture it?
Some Inspiration


The old days: 35mm film, $0.30 / pic
Now: Digital
• Disk storage = $1 / Gigabyte
• DVD = $0.50 / Gigabyte
• Images =~ 1 Mbyte
• 1 picture < $0.001 =~ free
From M. Cohen
Microsoft Research
Photos

Our roots: Edison’s 35mm film (1889)
• Chemical coating
reacts to light
• Specific chemicals
react to unique
wavelengths
Analog recording of visible wavelengths
and intensities (continuous data)
Images

An image is a 2D function I(x, y) that
specifies intensity for each point (x,
y)
From L. McMillan
MIT Comp. Graphics
Analog to Digital

Our goal is to convert the continuous
image to a discrete set of samples
Point Sampling an Image


Simplest sampling is on a grid
Sample depends
solely on value
at grid points
Point Sampling

Multiply sample grid by image
intensity to obtain a discrete set of
points, or
samples.
A to D, 2-D Example
Analog to digital conversion
Analog Signal
image slice
intensity

location on image slice
A to D, 2-D Example

Analog to digital conversion
Analog Signal
intensity
image slice
location on image slice
Digital Signal
intensity
samples
location on image slice
From Computer Graphics: Principles and Practice
Sensor Arrays

Capturing “samples” of light
Sensor Arrays

Storing amount of light contacting
squares on a surface
• Each square tuned to
sense red, green, or blue
• More green squares than
others
• Humans see green best
(lots of tigers in the grass)
Human Perception

We see these colors (mostly green)
Sensor Arrays



Very small
Lenses focus light
Light generates
electricity that is
sensed by a silicon
chip
Sensor Arrays



Pixel: each square in sensor arrays
Resolution: number of pixels
Color depth: number of bits that
describe the “reading” at each pixel
Sensor Arrays

Aliasing: a common problem when
discretizing analog data
Moire Pattern
Sensor Arrays

Noise: at low light, small errors in
pixel “readings” are very apparent
• Human perception is more sensitive to
small errors at low intensities

Digital cameras have more trouble
than film cameras in low light
Compression

Light intensity at all sensors can be
recorded
• Large: RAW and TIFF file formats

Small, local intensity variations can
be lumped together
• JPEG (most common of digital cameras)
• GIF (common web format)
Digital to Analog
New Display Technologies

Display Walls
• Princeton
• Stanford
• UVa – Greg
Humphreys
Interesting Display Technologies

Wood chips
Chrome spheres
Daniel Rozin – NYU: (movies) http://fargo.itp.tsoa.nyu.edu/~danny/art.html
Human Perception

A monitor can display these colors
• Note how limiting
they are
Human Perception

A photograph is a poor
model of realistic
lighting conditions
• Cannot convey shadows
and bright spots of real
scenes
• Artists use optical
illusions to fool us into
sensing lighting
conditions

Photographers have
limited abilities to
replicate artistic
features
Jack Tumblin, SIGGRAPH ‘99
Creating Moments


Automatically capitalizing on artistic
techniques to emphasize photos
Creating interactive photo techniques
Modern Technology
Stanford
New Results
Ng – Stanford
http://graphics.stanford.edu/papers/lfcamera/
New Results
Ng - Stanford
New Results

http://graphics.stanford.edu/papers/l
fcamera/lfcamera.wmv
Modern Technology
Mitsubishi Electric Research Labs
New Results

Shadows indicate edges
Artist Rendition
MERL Cam
New Results
New Results

http://www.merl.com/people/raskar/
NprCamera/nprMovie320short2.avi
New Results

Multi-Exposure
Durand - MIT
New Results
M. Cohen
Microsoft Research
New Results

Automatic Panoramas
M. Cohen
Microsoft Research
Modern Technology



10 Mpixel Sensor
200 fps
Buffers 5 seconds of capture
• 10 Gigapixel circular buffer
• Continuously “on”
M. Cohen
Microsoft Research
Merge Multiple Shots
New Display Technologies

Light Stage
New Display Technologies

http://gl.ict.usc.edu/research/LS5/LS
5_Siggraph_052005.avi