Computer Vision:
Parallelize or Paralyze
Team Purple Threads
CSE Capstone 2012
April 2012
Abstract
•
•
•
•
•
•
•
•
Purple Threads
Project Description
Motivation
System Overview
First Steps
Target Drone Platform
Turret System
Final Status
Purple Threads
Top Row: Duc Tran, Aviral Shrivastava(Sponsor), Gabriel Silva
Bottom Row: Nicholas Moulin, Craig Hartmann, Anthony Russo, Nadim Hoque
Project Description
• This project is a real world system of
robots that visually demonstrates the
benefits of parallelization for computer
vision applications
Motivation
• Computer vision systems have a wide
variety of applications, but are very
processor intensive
• Parallelization allows the
implementation of more advanced
computer vision techniques by
removing the bottleneck on the
processor
System Overview
• Project consists of two different robots
– Target Drone Platform
• Detect & track projectiles
• Avoid projectiles
– Turret System
• Detect & track the target
• Hit target with foam darts
First Steps
• Requirement Elicitation
– High Level
– Low Level
•
•
•
•
•
Hardware/Software Specification
Risk Management
Project Timeline
Budget
Configuring Development Environment
Target Drone Platform
• Hardware
– Traxxas Slash VXL*
– ArduPilot Mega w/ IMU Shield
– ION Intel Atom Motherboard*
– Microsoft Xbox 360 Kinect Sensor*
*Other names and brands may be claimed as the property of others
Target Drone Platform
• Software
– EMGU CV 2.3.0
• Open CV Wrapper (C#)
– ArduRover
• Code for ArduPilot
– Microsoft Robotics Developer Studio 4*
– Kinect for Windows SDK v1.0*
*Other names and brands may be claimed as the property of others
Target Drone Platform
• Implementation
– Assemble Target Drone
• Hardware Components
• Power System
– Setup Wireless Access to Drone
– Configure Drone Software Stack
– Implemented Platform Services through
RDS
Target Drone Platform
• Implementation
– Obtain RGB & Depth Image Data From
Kinect*
– Projectile Detection
• Color Threshold
– Projectile Tracking
– Collision Avoidance Algorithm
*Other names and brands may be claimed as the property of others
Target Drone Platform
• Setbacks
– Foam dart too small to track accurately
– Hardware too heavy for original shocks
– Depth frame coordinates are offset from
RGB frame coordinates with no translation
function
– Dr. Shrivastava crashed the car into a tree!
Turret System
• Hardware
– USB Missile Turret
– Microsoft Xbox360 Kinect Sensor*
– Arduino Uno
– Servos
• Software
– OpenCV 2.3.1
– Libfreenect
– Ubuntu 10.04*
*Other names and brands may be claimed as the property of others
Turret System
• Implementation
– Obtain RGB/Depth Images from Kinect*
Sensor
– Object Detection
• Color Threshold
• Cascade Classification
– C USB Turret Driver
– Program Arduino to Control Servos
– C NetDuino Driver
*Other names and brands may be claimed as the property of others
Turret System
• Implementation
– Equations Created to Track Target
• Depth Disparity to Depth(ft)
– d_feet = (disparity-824.8)/15.2
– d_feet = (0.1236*tan((disparity/2842.5)+1.1863))*3.2808399
• Pixels per Foot depending on Depth
– Pixels/Foot = -55.91836*log(d_feet)+177.49225974457
Turret System
• Setbacks
– Original turret configuration not precise
enough
– Unable to implement libfreenect driver on
PS3*
– Turret caught on fire
– Turret mounting unstable
*Other names and brands may be claimed as the property of others
Turret System
• Turret Tracking and Hitting a Moving
Car
• http://youtu.be/r7ccvPGcY8U
Turret System
• Effect of Parallelization on Turret
System
• http://youtu.be/D5yTRKgOykY
Final Status
Major Deliverables
Deadline
Status
Project Plan
9/26/2011
100%
Unparallelized
Turret System
1/31/2012
100%
Unparallelized
Target Drone
2/7/2012
100%
Parallelized
Robots
2/20/2012
50%
Benchmark
Results
2/28/2012
50%
Product
Documentation
3/15/2012
100%
Final Delivery of
System
3/15/2012
75%
Questions
Backup Slides
Turret System
• Implementation
– Equations To Track Target
• Horizontal Rotation
• Vertical Rotation