Memorial University of Newfoundland
Faculty of Engineering & Applied Science
Engineering 7928
Computer-Aided Engineering
INTRODUCTION TO MACHINE VISION II
Prof. Nick Krouglicof
Memorial University of Newfoundland
4/9/2015
Presentation Outline
1. Machine vision systems for mechanical metrology
•
•
Algorithms for camera calibration
Development of 3D vision systems for MI Flume Tank
2. Industrial Applications of Machine Vision
•
•
High speed, line scan camera-based inspection system for the
food processing industry
Vision based inspection of liquid crystal display (LCD) modules
Memorial University of Newfoundland
4/9/2015
A systematic approach to the calibration of
machine vision systems for industrial metrology
•
In the field of machine vision, camera calibration refers to
the experimental determination of a set of parameters which
describe the image formation process for a given analytical
model of the machine vision system.
•
Ideally, camera calibration is performed without specialized
optical equipment, without modifications to the hardware,
and without a priori knowledge of the vision system.
•
Most calibration techniques are based on the observation of
planar (2D) targets with a large number of control points.
Memorial University of Newfoundland
4/9/2015
A systematic approach to the calibration of
machine vision systems for industrial metrology
The machine vision parameters
include :
a)
b)
c)
d)
e)
which must be identified
The scale factor
The frame buffer coordinates of the image center
The effective focal length of the lens-camera assembly
The radial and tangential lens distortion coefficients
The pose (position and orientation) of the camera
Parameters a) through d) are classified as intrinsic, e) as
extrinsic.
Memorial University of Newfoundland
4/9/2015
A systematic approach to the calibration of
machine vision systems for industrial metrology
Memorial University of Newfoundland
4/9/2015
A systematic approach to the calibration of
machine vision systems for industrial metrology
Memorial University of Newfoundland
4/9/2015
An Efficient Camera Calibration Technique Offering
Robustness and Accuracy Over a Wide Range
of Lens Distortion
Memorial University of Newfoundland
4/9/2015
An Efficient Camera Calibration Technique Offering
Robustness and Accuracy Over a Wide Range
of Lens Distortion
CALIBRATION ACCURACY AND THE LENS DISTORTION MODEL
Memorial University of Newfoundland
4/9/2015
Underwater 3D Vision Systems for MI Flume Tank
Memorial University of Newfoundland
4/9/2015
Prototype Underwater Stereo Vision System
Memorial University of Newfoundland
4/9/2015
Calibration Target for
Underwater Stereo Vision System
Memorial University of Newfoundland
4/9/2015
Development of an Intelligent 3D Vision System for
Underwater Environment through Actively
Manipulated Laser Triangulation
Memorial University of Newfoundland
4/9/2015
Development of an Intelligent 3D Vision System for
Underwater Environment through Actively
Manipulated Laser Triangulation
Memorial University of Newfoundland
4/9/2015
Industrial Machine
Vision Applications
Memorial University of Newfoundland
4/9/2015
Image Analysis Tools for Automated Inspection in the
Food Processing Industry: X-Ray Enhancement
Memorial University of Newfoundland
4/9/2015
Image Analysis Tools for Automated Inspection in the
Food Processing Industry: Multispectral Imaging
Memorial University of Newfoundland
4/9/2015
Image Analysis Tools for Automated Inspection in the
Food Processing Industry: Multispectral Imaging
Memorial University of Newfoundland
4/9/2015
High Speed, Line Scan Based Inspection System
Memorial University of Newfoundland
4/9/2015
Line Scan Based Inspection System: Specifications
•Objective: To remove defects (I.e. visible, dark particle larger
than 0.007”) from apple sauce
•System must be able to handle 12 metric tons per 8 hour shift
•System must remove 95% of visible defects
Memorial University of Newfoundland
4/9/2015
High Speed, Line Scan Based Inspection System
•2 distinct challenges:
•Detection
•Removal
Memorial University of Newfoundland
4/9/2015
High Speed, Line Scan Based Inspection System
Memorial University of Newfoundland
4/9/2015
High Speed, Line Scan Based Inspection System
Memorial University of Newfoundland
4/9/2015
Line Scan Based Detection System
•Detection system based on a high performance line scan
camera; 4096 pixels per line at 4800 lines per second.
•Image acquisition and processing functions implemented on
a Complex Programmable Logic Device (CPLD) as
opposed to a microprocessor or Digital Signal Processor
(DSP).
•The objective is to implement image processing functions
with dedicated logic gates (i.e. hardware) for real-time
performance.
Memorial University of Newfoundland
4/9/2015
What are CPLDs?
•Complex Programmable Logic
Devices (CPLDs) are a class of
programmable logic device that
are commonly used to implement
complex digital designs on a
single integrated circuit.
•Applications of CPLDs in the field
of computer engineering include
the implementation of bus
controllers, address decoders,
priority encoder and state
machines
Memorial University of Newfoundland
4/9/2015
Line Scan Camera-Based Detection System
Memorial University of Newfoundland
4/9/2015
Line Scan Camera-Based Detection System
Memorial University of Newfoundland
4/9/2015
Line Scan Camera-Based Detection System
Typical section of apple
sauce recorded with an
area scan camera
Typical Particle
Memorial University of Newfoundland
4/9/2015
Line Scan Camera-Based Detection System
LINESCAN DATA
120
Light Level
100
80
60
40
20
1
101
201
301
401
501
601
701
Pixel Number
Memorial University of Newfoundland
4/9/2015
Line Scan Camera-Based Detection System
Memorial University of Newfoundland
4/9/2015
Removal System
Memorial University of Newfoundland
4/9/2015
Removal System
Memorial University of Newfoundland
4/9/2015
Removal System: Flow Characterization
•Rheological nomenclature and
associated velocity profiles for steady
flow through tubes with circular cross
section.
Memorial University of Newfoundland
4/9/2015
Viscosity Measurement
Memorial University of Newfoundland
4/9/2015
Viscosity Measurement
Velocity profile can
be characterized by
power law!
Memorial University of Newfoundland
4/9/2015
Flow Profile of Apple Sauce
Memorial University of Newfoundland
4/9/2015
Particle Removal Window
Memorial University of Newfoundland
4/9/2015
System Timing Diagram
Memorial University of Newfoundland
4/9/2015
Aspiration Valve Characterization
Memorial University of Newfoundland
4/9/2015
Aspiration Valve Characterization
Memorial University of Newfoundland
4/9/2015
Particle Detection Rate Versus Flowrate
Memorial University of Newfoundland
4/9/2015
High Speed, Line Scan Based Inspection System:
Current Status
•Industrial partner is currently developing the production version
of the system.
•Packaging of the principle components (i.e., lenses, cameras,
electronics, light sources) remains a major challenge given the
environment.
•One possible solution is to integrate all the electronics in the
camera enclosure.
•Partner is anxious to explore applications in the pulp and paper
industry.
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
•Objective: to automate the
inspection of LCD modules in
order to improve quality control
•One step in the
implementation of a Six-Sigma
Program (“3.4 defects per
million opportunities”)
•The inspection must be
completed within 30 seconds
for 10 predetermined LCD
patterns
•System can “learn” new LCD
modules without modifying
software
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of LCD modules:
System Components
•Pulnix camera with macro
lens
•High frequency
fluorescent light sources
•Coreco Bandit integrated
image acquisition and
VGA accelerator
•Software developed using
with WiT graphical
programming environment
in combination with
Microsoft VB
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
Original Image Showing Error in Alignment
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
Thresholding Operation – Image Subtraction
with respect to an image with no segments
illuminated
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
Blob Analysis – Reference Points are Identified
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
Image Rotation and Translation
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
Pixel by Pixel Image Subtraction from
Reference Image – Thinning Operator
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules
Blob Analysis
Memorial University of Newfoundland
4/9/2015
Vision Based Inspection of Liquid
Crystal Display (LCD) modules: Conclusions
•Inspection system was installed at BAE Systems Canada Ltd.
where it was used to test between 200 and 600 LCD displays
per day.
•Number of defective modules that passed inspection was
basically reduced to zero.
•Occasional “false positives” proved to be technical problems
with the devices that previously went unreported.
•Applications for this technology are numerous given the number
of LCD displays produced annually.
Blob Analysis
Memorial University of Newfoundland
4/9/2015
QUESTIONS
?
Memorial University of Newfoundland
4/9/2015