Stylization and Abstraction of
Photographs
Doug Decarlo and Anthony Santella
Outline
•
•
•
•
•
•
•
Introduction
Goals / Motivation
Image Structure and Analysis
Hierarchical Image Representation
Results
Video Summary
Conclusion
Introduction
• The human visual system is very powerful
• It masks the complex perceptual and cognitive
processing that is needed to understand
images
• Good information design makes it easer for us
to understand images
– Keep detail light in unimportant regions
– Have fine detail only in important areas
Example
• Invert these heuristics and…
Goals
• Paper goal: Present a method for stylizing and
abstracting photographs to clarify the
meaningful information in them with no artistic
ability required
Motivation and Contributions
• Artists have known
about abstraction for
years
Henri de Toulouse-Lautrec
Moulin Rouge-La Goulue
Motivation and Contributions (Cont)
• Many different heuristics have been created for
emphasizing certain parts of images
• But it is very hard to automatically detect
meaningful elements in a photograph
• Paper’s solution: Use eye movements to help
detect the important areas of a photograph
Algorithm Summary
• To transform an image:
– Instruct a user to look at the image for a short time,
obtaining a record of eye movements.
– Disassemble the image into its constituents of
visual form using visual analysis (image
segmentation and edge detection)
– Render the image, preserving the form predicted to
be meaningful by applying a model of human visual
perception to the eye-movement data
Image Structure and Analysis
• Edge Detection: The process of extracting out
location of high contrast in an image that are
likely to form the boundary of objects
Image Structure and Analysis
• Image Segmentation: Partitioning an image into
contiguous regions of pixels that have similar
appearance.
Image Structure and Analysis
• For image segmentation, colors were
represented in L*u*v space, to produce region
boundaries that were more meaningful for
human observers
Image Structure and Analysis
• Scale-space Theory: Provides a description of
images in terms of how content across different
resolutions is related
• This theory serves as the basis for their
hierarchical representation of the image
• It uses segmentation algorithms applied at a
variety of scales, and finds containment
relationships between their results
Visual Perception
• Eye movements give a strong indication of the
important elements in an image
• People can examine only a small area of an
image at a time, and therefore scan them in a
series of “fixations”
Eye tracker
• Use an eye tracking device to figure out the
important areas of a photograph
Hierarchical Image Representation
• Create an image pyramid of the input image
– The bottom image is the
original image
– Each layer up is a downsampled by a constant factor
– A segmentation algorithm
is computed at each later
• Edges are detected using the original image, a
process called Edge Tracking is used to smooth
them
Building the Hierarchy
• Construct a tree, a leaf is created for each
segmentation in the tree’s bottom image
Rendering with a Perceptual Model
• To create the line drawings, the hierarchal tree
that was created is “pruned” of leaf nodes in
areas where it is determined the user didn’t see
• Pruning is done
based on the
fixations that were
recorded and
perceptual cues
such contrast
sensitivity
Region Smoothing
• The detail level of boundaries is uniformly high,
since all boundaries derive from the lowest
segmentation
Drawing Lines
• After the segments have been drawn, we draw
the lines
• When drawing the lines, we ignore lines that
were not important (using factors like how close
they were to a fixation, how long they are, etc)
Results
Results
Video
• Video time…
Future Work
• Have the segmenter use a model of shading
• Find new ways to represent textures
Conclusion
• This paper presented a new visual style using
bold edges and constant color and a method of
interaction using eye tracking that helps find
important areas of a photograph for image
rendering
Questions?
• ???