Image Detection
Functionality+Open CV
Young S. Kim
Dankook University
June 02 2020
Source: http://img70.imageshack.us/img70/9950/serversnm2.jpg
Hardware/Software failure may result in corrupted images
https://www.youtube.com/watch?v=ORWYr1gx5eU#action=share
AI Reconstructs Photos with Realistic Results
Kinect for Windows Ski Shop Scenario Video
Exploration
• Image Processing overview/OpenCV
• Functionality
• Programming with Open CV
• OpenCV on CPU and GPU
Image Processing overview/
OpenCV
Image Processing Functionality/OpenCV
Image processing
•General Image
Processing
Transforms
Fitting
Optical Flow
Segmentation
Features
Depth Maps
Object detection
Video, Stereo, and 3D
Camera
Calibration
Pose estimation
4
Optical flow
OpenCV function : cvCalcOpticalFlow
Lucas-Kanade algorithm
It is a pattern of the coronal movement of an object
between two successive frames caused by the
movement of the object and the camera
Set each pixel window in one frame and find the best
match for this window in the next frame.
cv2.solvePnPRansac()
Camera Calibration
Intrinsic parameter
Extrinsic parameter
3D view of checkerboard
Un-­distorted image
Gary Bradski (c) 2008
Human Pose Estimation is defined as the problem of localization of human joints (also known as keypoints
- elbows, wrists, etc) in images or videos. It is also defined as the search for a specific pose in space of all
articulated poses
Pose estimation
PCK: percentage of correct keypoint
https://www.youtube.com/watch?v=L7iq69qWxAA&feature=youtu.be
Depth map
a depth map is an image or image channel that contains information relating to the distance of the surfaces of scene objects from a viewpoint.
import numpy as np
import cv2
from matplotlib import pyplot as plt
imgL = cv2.imread('./data/tsukuba_l.png',0)
imgR = cv2.imread('./data/tsukuba_r.png',0)
stereo = cv2.StereoBM_create(numDisparities=1
6, blockSize=15)
disparity = stereo.compute(imgL,imgR)
plt.imshow(disparity,'gray')
plt.show()
OpenCV Architecture and Development
5
CUDA의 Pros.
https://www.youtube.com/watch?v=P28LKWTzrI&feature=youtu.be&list=RDCMUCHuiy8bXn
mK5nisYHUd1J5g
FLANN(Fast Library Approximated Nearest Neighbors)
FLANN: SIFT + check=50
FLANN parameters FLANN_INDEX_KDTREE = 0
index_params = dict(algorithm = FLANN_INDEX_KDTREE,
trees = 5) search_params = dict(checks=50) # or pass
empty dictionary
flann =
cv2.FlannBasedMatcher(index_params,search_params)
matches = flann.knnMatch(des1,des2,k=2)
OpenCV License
•Based on BSD license
•Free for commercial and research use
•Does not force your code to be open
•You need not contribute back
–But you are very welcome to contribute back!
OpenCV sponsors
Where is OpenCV Used?
• Academic and Industry Research
• Security systems
• Image/video search and retrieval • Structure from motion in movies
• Machine vision factory production inspection systems
• Automatic Driver Assistance Systems
• Safety monitoring (Dam sites, mines, swimming pools)
• Robotics
3D Processing: PCL
Point Cloud Library
l
http://pointclouds.org/
PCL: Finding Normals
Functionality
How to choose which algorithms to
put into the library?
• Well established
• Cutting edge
• Works on any data • Works on Lenna
only
– Productized by a
commercial
company
– Patented
• Easy to reproduce
vs
• Have to hallucinate
the missing pieces
– And then it works
on Lenna only
Imgproc 1
General Image Processing Functions
Transforms
Robot support
Image Pyramids
Integral images
Imgproc 2
Segmentation
Shape analysis
Robot support
Features2d contents
Detection
Description
Matching
Matchers available
•
•
•
Detectors available
•
•
•
•
•
•
•
SIFT
SURF
FAST
STAR
MSER
HARRIS
GFTT (Good Features
To Track)
Descriptors available
•
•
•
•
•
•
SIFT
SURF
Calonder
Ferns
One way
HoG
BruteForce
FlannBased
BOW
Matches filters
(under construction)
• Cross check
• Ratio check
MSER + SIFT + FLANN + RANSAC +SVD
Feature detector
Descriptor
Descriptor Matcher
•BFMatcher
•FlannBasedMatcher
•FastFeatureDetector
•MSER
•SimpleBlobDetector
•GFTTDetector
Keypoint Detector + Descriptor= Feature2D
•BRISK
•ORB
•KAZE
•AKAZE
•SIFT(xfeatures2d)
•SURF(xfeatures2d)
• It should be easy to identify even if the shape, size, or position of the object changes.
• Even if the camera's point of view or lighting changes, it will be easy to find the point in
the image.
It should be easy to identify even if the shape, size, or position of the object changes.Even if
the camera's point of view or lighting changes, it will be easy to find the point in the image
Harris corner 1988
FAST(Feature from Accelerated Segment Test)
If there are n or more consecutive pixels (> p + t) brighter than a certain value than p, or n
or more consecutive dark (<p-t) pixels over a certain value, p is determined as a corner
point.
SIFT(Scale Invariant Feature Transform)2004
How to do image matching with SIFT? 1. Scale space, 2. DOG, 3. Key point,
4.Hessian Matrix, 5. Direction, 6. SIFT feature
SURF(Speedup Robust Feature)
Detects the points with the largest determinant of the
Hessian matrix as a feature point on the scale space.
import cv2
import numpy as np
filename = './data/butterfly.jpg'
img = cv2.imread(filename)
gray= cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
surf = cv2.xfeatures2d.SURF_create(50000)
kp, des = surf.detectAndCompute(gray, None)
print(len(kp))
img2 = cv2.drawKeypoints(gray,kp,None,(0,0,255),4)
cv2.imshow('img2', img2)
cv2.waitKey()
cv2.destroyAllWindows()
Difference between SIFT and SURF
While SIFT filter size is fixed and image size is reduced to detect points of interest,
SURF, as opposed to SIFT, fixes image scale, increases filter size and extracts points of
interest.
Pedestrian Detection: HOG Descriptor
• Object shape is
characterized by
distributions of:
– Gradient magnitude
– Edge/Gradient
orientation
• Grid of orientation
histograms
Magnitude
Edge
Orientation
36
HOG + SVM
Calib3d module
• Camera calibration : cv2.solvePnPRansac()
• 3D -­‐> 2D projection
• Homography in 2D and 3D
• PnP problem solver
• Stereo vision
• Fundamental matrix • Template detectors
Machine Learning Library (MLL)
CLASSIFICATION / REGRESSION
(new) Fast Approximate NN (FLANN)
Naïve Bayes
CART
Random Forests
(new) Extremely Random Trees
(new) Gradient Boosting Trees
Statistical Boosting, 4 flavors
SVM
Face Detector
AACBAABBCBCC
AACACB
CCB
CC
CLUSTERING
K-­‐Means
EM
(Mahalanobis distance)
CBABBC
AAA
B
CB
C
ABBC
B
A
BBC
BB
TUNING/VALIDATION
Cross validation
Bootstrapping
Variable importance
Sampling methods
34
C
Object detection with Deep learning
Haar Feature Selection
Creating an Integral Image
Adaboost Training
Cascading Classifiers
ML Lib Example:
In samples/cpp, see:
Multicascadeclassifier.cpp
Gary Bradski (c) 2008
Boosting: Face Detection with
Viola-­Jones Rejection Cascade2001
Boosting is one of the machine learning ensemble techniques. It is an algorithm
that improves prediction or classification performance by combining multiple weak
learners that are sequential.
SVM:Support Vector Machine : Decision boundary
Object detection
P. Felzenszwalb, D. McAllester, D. Ramaman. A
Discriminatively Trained, Multiscale,
Deformable Part Model. Proceedings of the
IEEE CVPR 2008.
Object tracking
2D
CamShift ();
CamShift();
MeanShift();
Tracking
KalmanFilter::
3D
Posit();
calcOpticalFlowPyrLK()
Also see dense optical flow:
calcOpticalFlowFarneback()
Gary Bradski (c) 2008
38
Ray tracing
Ray tracing is a rendering technique for generating an image by tracing the path of light as pixels in an image plane and simulating the effects of its encounters with virtual objects.
https://www.youtube.com/watch?v=KJRZTkttgLw&list=RDCMUCHuiy8bXnmK5nisYHUd1J5g&
index=2
Programming with OpenCV
Simple Image Processing
Histogram
I/O
Camera Calibration, Pose, Stereo
Object Recognition
Q&A
Refereces
• AI Technology & Systems
• SUALAB Research
• A Gentle Introduction to Object Recognition With Deep Learning
• Pyimage search
• OPEN CV
• 2D and 3D face recognition: A survey Pattern Recognition Letters
28 (2007) 1885–1906
• Texture Feature Extraction Methods: A Survey
IEEE Access ANNE HUMEAU-HEURTIER