Image Processing
and Computer
Vision
Outline
• Defining Image Processing and Computer Vision
• Emerging Technology
–
–
–
–
–
Digitization of documents
Digitization of images/photographs
Biometrics
Management of images on computers
Other: manufacturing, military, games, …
• Research in Image Processing and Computer
Vision
– Automatically Finding Faces and Cars
– Content-based Image Retrieval
Image Processing vs. Computer Vision
• Image Processing
– Research area within electrical engineering/signal processing
– Focus on syntax,
low level features
• Computer Vision
image
image
– Research area within computer science/artificial intelligence
– Focus on semantics,
symbolic or geometric
Faces
descriptions
People
Chairs
etc.
image
Optical Character Recognition (OCR)
• First patent in OCR in 19th century
• First applications in post-office and banks
• Documents easier to distribute, search, organize,
and edit in digital form
– Typewriter has been replaced by word processor
– Lots of legacy materials (the world’s libraries of books)
available only in print
• State of the art not perfect, but 99% accurate on
cleanly printed pages
• Examples of errors. . .
Heavy Print
Output from 3 commercial OCR systems
Light Print
Stray Marks
Typography
Processing Overlaid Text in Video
Text Area Detection
Video
The Video OCR (VOCR)
process used by the
Informedia research group at
Carnegie Mellon
Text Area
Preprocessing
Commercial OCR
ASCII Text
Text Area Detection
Video Frames
Filtered Frames
AND-ed Frames
(1/2 s intervals)
© Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
Carnegie Mellon
VOCR Preprocessing Problems
Augmenting VOCR with Dictionary Look-up
Handwriting Recognition
• Natural progression to OCR work for print
• Works if constraints on writer, e.g. palm pilot,
where user is asked to conform to specific
style or convention
Other Document Processing
• Not just for text. . .
• Examples:
– Engineering document to CAD file
– Maps to GIS format
– Music score to MIDI representation
Outline
• Defining Image Processing and Computer Vision
• Emerging Technology
–
–
–
–
–
Digitization of documents
Digitization of images/photographs
Biometrics
Management of images on computers
Other: manufacturing, military, games, …
• Research in Image Processing and Computer
Vision
– Automatically Finding Faces and Cars
– Content-based Image Retrieval
Digital Cameras = Convenience
•
•
•
•
Easy to capture photos
Easy to store and organize photos
Easy to duplicate photos
Easy to edit photos
• Rough Multimedia eCommerce class survey:
–
–
–
–
1999:
2000:
2001:
2002:
10% own digital cameras
25%
50%
??
Digital Camera Cautions
Via “Photo Industry Reporter” e-Magazine at:
http://www.photoreporter.com/2002/10-21/photokina_report_look_at_35mm.html
• Film cameras still outsell digital cameras by
almost three to one
• The household penetration of digital is at about
15%
• “But let’s face it: film’s days are numbered.
Anyone staying solely with film these days will
have a glorious buggy whip in a market that will
be clamoring for cars.”
Digital Camera Growth
• Photo Marketing Association on US digital camera
sales:
–
–
–
–
4.5 million in 2000
6.9 million in 2001
Projected 9.3 million for 2002
http://www.visioneer.com/About/press/june2402.html
• InfoTrends Research Group estimates that the U.S.
photo-enabled TV set-top installed base will grow from
less than 1 million units in 2002, to over 114 million
units in 2006. Household penetration will climb from
under 1% to around 85%.
• InfoTrends projects digital camera sales to grow at a
rate of 38% through 2003
State of the Art: Digital Cameras
• Film is currently better in resolution and color
– Professional photographers
• Digital for low quality newspaper advertisements
• Film for portrait photos
• Computer storage limitations: 1 high resolution digital image = 20-25
Megabytes
– http://pic.templetons.com/brad/photo/pixels.html
– 3500 line pairs/35 mm or about 5000 dots/inch, but grainy
– At 3:2 frame size, ~20 million pixels
– Conclusion: “a 5300 x 4000 digital camera would produce a shot
equivalent to a scan from a quality 35mm camera -- provided you can
get more than 8 bits per pixel. …A 3000 x 2000 digital camera would
match the 35mm for a good percentage of shots.”
• Printing: home printers not comparable to commercial printers
Future of Digital Cameras
• Improved resolution and color
• “Smart” cameras
• More programmable features
– Auto-focus on object of interest
– “Everything in focus” photo
– Capture photo when event X occurs
Outline
• Defining Image Processing and Computer Vision
• Emerging Technology
–
–
–
–
–
Digitization of documents
Digitization of images/photographs
Biometrics
Management of images on computers
Other: manufacturing, military, games, …
• Research in Image Processing and Computer
Vision
– Automatically Finding Faces and Cars
– Content-based Image Retrieval
Biometrics
• Technology for
identification
– Finger/palm print
– Iris
– Face
Fingerprints
• Minutae – spits and merges of ridges
Face Identification
• Not quite reliable yet.
– Performance degrades rapidly with uncontrolled
lighting, facial expression, and size of database
• Several companies exist:
–
–
–
–
–
–
–
Visionics (Rockfeller University spin-off)
Viisage (MIT spin-off)
EyeMatic (USC spin-off)
Miros (MIT spin-off)
Banque-Tec Intl (Australia)
C-VIS Computer Vision (Germany)
LAU Technologies
• Commercial systems installed in London and
Brazil to catch criminals
Automatic Age Progression
Original Image
(1962)
Computer-Aged
(1997)
Actual Photo
(1997)
Outline
• Defining Image Processing and Computer Vision
• Emerging Technology
–
–
–
–
–
Digitization of documents
Digitization of images/photographs
Biometrics
Management of images on computers
Other: manufacturing, military, games, …
• Research in Image Processing and Computer
Vision
– Automatically Finding Faces and Cars
– Content-based Image Retrieval
Management of images on computers
• Compression – reducing
storage size needed for
images
• Watermarking –
Protecting copyright
• Microsoft, Bell Labs, NEC,
etc.
Visible watermark
Photo Manipulation
• Adobe Photoshop, Corel
PhotoPaint, Pixami, PhotoIQ,
etc.
• Image editing: crop an image,
adjust the color, paint over
part of any image, airbrush
part of an image, combine
images, etc.
• Future: Applications of
computer vision, e.g.,
discriminating foreground
from background.
Online Digital Image Collections
• Stock photos of use to graphic designers,
artists, etc.
• Large collections of images exist
– Corbis 67 million images
– Getty 70 million stock photography images
– AP collects 1000s of digitized images per day
Outline
• Defining Image Processing and Computer Vision
• Emerging Technology
–
–
–
–
–
Digitization of documents
Digitization of images/photographs
Biometrics
Management of images on computers
Other: manufacturing, military, games, …
• Research in Image Processing and Computer
Vision
– Automatically Finding Faces and Cars
– Content-based Image Retrieval
Inspection for Manufacturing
• Occum – inspection of printed circuit boards
($100M / year)
• Cognex – Do-it-yourself toolkits for inspection
(400 employees)
Automatic Target Recognition (ATR)
• Finding mines, tanks, etc.
• Billion dollar a year industry
– Martin-Lockheed, TSR, Northrup-Grumman, other
aerospace contractors.
• Various types of imagery:
– Synthetic Aperture Radar (SAR), Sonar, hyperspectral imagery (more than 3 colors)
Aerial Photo Interpretation
• Also referred to as “automated cartography”
• Classification of land-use: forest, vegetation,
water
• Identification of man-made objects: buildings,
roads, etc.
Better Security Cameras
• Cameras that are responsive to the
environment
– Track and zoom on moving objects
– Automatic adjustment of contrast
Medical imagery
• Medical image libraries for study and
diagnosis
• Image overlay to guide surgeons
History
• 1980’s ~100 companies – manufacturing
applications mostly
• Early 1990’s less than 10 companies
• Late 1990’s ~100 companies – face
recognition, intelligent teleconferencing,
inspection, digital libraries, medical imaging
Outline
• Defining Image Processing and Computer Vision
• Emerging Technology
–
–
–
–
–
Digitization of documents
Digitization of images/photographs
Biometrics
Management of images on computers
Other: manufacturing, military, games, …
• Research in Image Processing and Computer
Vision
– Automatically Finding Faces and Cars
– Content-based Image Retrieval
Image Processing: Filtering
Enhancing an image’s quality for human
viewing, e.g., in medical imaging or in
telescopic views of space
Image Processing: Compression
• Lossless – No loss in quality: gif, tiff
• Lossy – Original image cannot be
reconstructed: jpeg
• New work on advancing lossy compression
strategies with fewer visual artifacts: JPEG
2000 and wavelet transformations
Image Processing: Watermarking
• Information hiding
– Protecting copyright
Image Processing: Transformation
• Transforming image can make it easier to
analyze
Wavelet transform of image
Wavelet Coefficients
Horizontal LP,
Vertical LP
Horizontal HP,
Vertical LP
Horizontal LP,
Vertical HP
Horizontal HP,
Vertical HP