OpenCV Training course By Theerayod Wiangtong Goals Develop a universal toolbox for research and development in the field of Computer Vision Why use OpenCV? Fast development time, more than 500 algorithms in OpenCV libraries C/C++ based programming Both Windows and Linux supported Open and free, BSD license Loads of developers using OpenCV Loads of information and documents Etc History of OpenCV Originally developed by Intel, currently maintained by Willow Garage 5 OpenCV - Features Cross-platform and extremely portable Free! for both research and commercial use Targeted for real-time applications Table Courtesy Learning OpenCV: Computer Vision with the OpenCV Library 6 OpenCV – Architecture & Modules CvAux Area for experimental algorithms: e.g. HMM, Stereo vision, 3D tracking, Bg/fg segmentation, camera calibration, Shape matching, Gesture recognition, .. OpenCV Comparisons Examples of Using OpenCV functions Click here OpenCV: Algorithmic Content OpenCV Functionality Basic structures and operations Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction (more than 500 algorithms!!) Image Thresholding Fixed threshold; Adaptive threshold; Statistics min, max, mean value, standard deviation over the image Multidimensional histograms Norms C, L1, L2 Multidimensional Histograms Histogram operations : calculation, normalization, comparison, back project Histogram Equalization Histograms comparison Image Pyramids Convolution in image The source pixel and its surrounding pixels are all mathematically merged to produce a single destination pixel. The matrix slides across the surface of the source image, producing pixels for the destination image http://beej.us/blog/data/convolution-image-processing/ Image Pyramids Gaussian and Laplacian Morphological Operations Two basic morphology operations using structuring element: erosion dilation Distance Transform Calculate the distance for all non-feature points to the closest feature point Two-pass algorithm, 3x3 and 5x5 masks, various metrics predefined Flood Filling •grayscale image, floating range •grayscale image, fixed range Feature Detection Fixed filters (Sobel operator, Canny operator, Laplacian, Scharr filter) Hough transform (find lines and circles) http://www.stevens-tech.edu/wireless/klin/EdgeDetection/EdgeDetectionInfo.htm Edge detection operators Simple 2 -1 -1 0 This means: pixel(i,j) = 2*pixel(i,j) - pixel(i,j+1) - pixel(i+1,j). Cross 0 1 1 0 -1 0 0 -1 Template 1: Template 2: pixel(i,j) = maximum(template 1, template 2) Edge detection operators Prewitt 1 0 -1 1 1 1 1 0 -1 0 0 0 1 0 -1 -1 -1 -1 Sobel 1 0 -1 1 2 1 2 0 -2 0 0 0 1 0 -1 -1 -2 -1 X-axis Template: Y-axis Template: pixel(i,j) = sqrt((x-axis template)^2 + (y-axis template)^2) Canny Edge Detector http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html Hough Transform Contour Retrieving The contour representation: Chain code (Freeman code) Polygonal representation Initial Point Chain code for the curve: 34445670007654443 Contour representation Hierarchical representation of contours Image Boundary (W1) (B2) (W2) (B3) (W3) (B4) (W5) (W6) Contours Examples Source Picture (300x600 = 180000 pts total) Retrieved Contours (<1800 pts total) After Approximation (<180 pts total) And it is rather fast: ~70 FPS for 640x480 on complex scenes Contour Processing Approximation: RLE algorithm (chain code) Teh-Chin approximation (polygonal) Douglas-Peucker approximation (polygonal); Contour moments (central and normalized up to order 3) Matching of contours Contours matching Matching based on hierarchical representation of contours Object Recognition: Eigen Image Object Recognition: HMM One person – one HMM Stage 1 – Train every HMM 1 … n Stage 2 – Recognition i Pi - probability Choose max(Pi) Motion Analysis and Object Tracking Background subtraction Motion templates Optical flow Active contours Estimators Background Subtraction Background: any static or periodically moving parts of a scene that remain static or periodic over the period of interest. How about waving trees, light on/off..?!? Background statistics functions Average Standard deviation Connect component Background Subtraction Example Motion Templates Object silhouette Motion history images Motion history gradients Motion segmentation algorithm MHG silhouette MHI Motion Templates Example •Motion templates allow to retrieve the dynamic characteristics of the moving object Object tracking Mean-shift Choose a search window (width and location) Compute the mean of the data in the search window Center the search window at the new mean location Repeat until convergence Cam-shift: Continuously Adaptive Mean SHIFT Region of interest Mean shift Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel Region of interest Mean shift Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel Region of interest Mean shift Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel Region of interest Mean shift Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel Region of interest Mean shift Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel Region of interest Mean shift Center of mass Mean Shift vector Slide by Y. Ukrainitz & B. Sarel Region of interest Mean shift Slide by Y. Ukrainitz & B. Sarel Center of mass Object tracking Particle filter Optical flow, LK Optical flow is the relation of the motion field. It is a 2D projection of the physical movement of points relative to the observer v2 p2 p3 p1 p4 Optical Flow I ( t 1) I ( t ), { pi } v1 v3 v4 {vi } Velocity vectors OpenCV shape classification capabilities Contour approximation Moments (image&contour) Convexity analysis Pair-wise geometrical histogram Fitting functions (line, ellipse) Using contours and geometry to classify shapes Given the contour classify the geometrical figure shape (triangle, circle, etc) Moments Here p is the x-order and q is the y-order, whereby order means the power to which the corresponding component is taken in the sum just displayed. E.g. m00 moment is actually just the length in pixels of the contour. Contour Not Hu moments (faster) applicable for different sizes, orientation invariants Image segmentation Separate image into coherent “objects” image human segmentation Segmentation Methods Edge-based approach Apply edge detector (sobel, laplace, canny, gradient strokes). Find connected components in an inverted image Color segmentation: histogram Calculate the histogram. Find the objects of the selected histogram in the image. OpenCV: Getting started 56 Getting Started Download OpenCV http://opencv.willowgarage.com/wiki/ There exists a short walkthrough video on YouTube at http://www.youtube.com/watch?v=9nPpa_WiArI Learning OpenCV: Computer Vision with the OpenCV Library by Gary Bradski and Adrian Kaehler http://proquest.safaribooksonline.com/9780596516130 OpenCV 2.1 with Visual Studio 2008 Download the OpenCV 2.1.0 Windows installer from SourceForge - "OpenCV-2.1.0-win32-vs2008.exe". Install it to a folder (without any spaces in it), say "C:\OpenCV2.1\". This article will refer to this path as $openCVDir During installation, enable the option "Add OpenCV to the system PATH for all users". Configure Visual Studio 2008 Open VC++ Directories configuration: Tools > Options > Projects and Solutions > VC++ Directories Choose "Show directories for: Include files" • Choose "Show directories for: Library files" • Add "$openCVDir\include\opencv" Add "$openCVDir\lib" Choose "Show directories for: Source files" • • • • Add "$openCVDir\src\cv" Add "$openCVDir\src\cvaux" Add "$openCVDir\src\cxcore" Add "$openCVDir\src\highgui" Configure your Project Open Project Properties: Project > %projectName% Properties... Open Linker Input properties: Configuration Properties > Linker > Input Open the "..." window to edit "Additional Dependencies" and on each line put: • • • • "cv210.lib" "cxcore210.lib" "highgui210.lib" And any other lib file, e.g, cvaux.lib, necessary for your project Your project should now build. If you get any errors try restarting Visual Studio and then doing a clean Rebuild. More info http://opencv.willowgarage.com/documentation/c/index.html http://dasl.mem.drexel.edu/~noahKuntz/openCVTut1.html http://sapachan.blogspot.com/search/label/Learning%20OpenCV http://www.shervinemami.co.cc/introToOpenCV.html http://note.sonots.com/OpenCV/Install.html Questions