Path and Obstacle
Evaluation Using Kinect
By: Ben Casey
Introductory
•
Help Robots “see” objects and free paths
Detect obstacles of unknown size and quantity
o Evaluate the ground in terms of free space
o
•
Need 3D points
o
o
stereo vision
RGB-D sensor
Kinect Sensor
Viewing
angle
Vertical tilt
range
43° vertical by 57° horizontal
field of view
±27°
Focal Length of 580 pixels
Depth values between 80 and
6000. -> Correlates to 0.8 meters
and 3.5 meters
Depth image resolution is 640X480
Color image resolution is 640X480
previous work
•
•
•
Obstacle Detection with the Kinect Sensor
o dealt with known obstacle size and known environment
o ground detection, obstacle segmentation, morphing, and
labeling
3D modeling of Indoor Environments
o RGB-D ICP, point cloud, and projection
o re-projection error metric > Euclidean error metric
Plane-based detection for stairs
o preemptive ransac after depth data change
o each plane is based off of ground plane
Challenges
•
•
•
•
•
Separating the floor from obstacles
Range of the sensor for depth values
Separating obstacles of variant:
o distances from camera
o size
o forms
Noisy images
Overlay manipulated data with color
Approach
1. Grab depth image and color image
2. process depth image
a. acquire 3D points and store them
b. Detect ground plane and store plane
c. Detect different objects the sizes and store them
3. manipulate color image
a. draw bounding boxes over stored objects
b. draw rest of ground plane
4. display color and depth images after manipulation side-byside
The Project
•
•
Software
o C++
o Nui
o openCV
step 1 -> Grab Data
HRESULT hr = m_pNuiSensor->NuiImageStreamGetNextFrame(hstream, 0, &pImageFrame);
INuiFrameTexture *pTexture = pImageFrame.pFrameTexture;
BYTE *pBuffer = (BYTE*)LockedRect.pBits;
cvSetData(Depth, (BYTE*)rgb, Depth->widthStep);
Basic Display
The Project (2)
•
step 2: process depth
o
how far away the point is can be in meters

o
o
meters = .000086*(realDepth^2) + .0009*realDepth -.01;
Get X,Y,Z from spherical coordinates
Calculate Plane given 3 points



ax+by+cz+d =0
RANSAC implementation with N = 2000
rand() in c++ only goes to 32000
transferring depth to meters
The Results
The Results cont ...
The Results cont ....
The project (3)
•
step 2: process depth cont…
o Calculate objects above the ground plane
 version 1: used simpleBlobDetector
 version 2: used cvBlobs
 version 3: detect edges via Y differences, then detect widths and
height on image
 save the objects widths, heights, and last point
The Results cont...
The Results cont...
The Results cont...
Future work
•
•
•
•
•
compare algorithm with PCL
decrease noise from depth to increase
accuracy
Hone in on X,Y,Z calculations
better accuracy with object recognition
increase frame rate
Questions?
Thank you!