Object Detection and Mapping
James Mussi
Goals and Purpose
One of the main purposes of this project is to design a UAS with object detection and
collision avoidance abilities. Object detection and navigation currently available and created by
previous projects was felt to be rather basic. Sonar sensors are the common method of
detection, but they are highly unreliable against angled surfaces, have no edge detection ability,
and relatively short range. The project’s goal was to create an advanced method of mapping
objects’ surfaces and edge detection from a distance, giving the craft the ability to plot a
smooth navigation path around objects.
Design Process
The design chosen from early on was to use a LIDAR scanning method with image
processing. The system uses a line laser being tracked by a camera. The pixel location of the line
laser on the camera can be translated to a distance away from the camera using projection
effects. This is the basic function of a 3D scanner. A 3D scanner will sweep its laser along an
object mapping the entire surface. Sweeping the laser is too unreliable and complex since the
craft is moving. Mapping all three dimensions of the surface is also unnecessary since craft is
only going navigate on a horizontal plane. It was decided to use a static line laser as that is
adequate to map surfaces on the 2D horizontal plane the craft will travel on.
Tracking the laser involves image processing with a camera. Image processing is a very
computer processing intensive. Many advanced UAS projects using image processing use
external computers communicating with the craft to perform the intensive calculations, then
relay the results back to the craft in real time. This project was not permitted to use any
external processing. All processing must be used on board the craft.
The difficulty of object tracking with image processing is reliably distinguishing the
object from other things in the image. A laser is especially difficult because other lighting
sources can diluted it’s reflection off of objects making it untraceable. This is solved by using a
laser in the infrared spectrum. An IR pass filter was used on the camera, which blocks all visible
light and only allows IR wavelengths. IR light from the environment is usually limited indoors
except from unfiltered incandescent bulbs. This makes the laser appear more prominent in the
image and also eliminates the need to filter a colored image. A brightness intensity image is all
that would be required, which is 1/3 the size of an RGB color space, and just the Y component
can be grabbed from YCbCr color space.
The team looked into devices which can perform image processing and have are small
enough to place on a hobbyist size quadcopter. The choices found by the team were Raspberry
Pi, Beaglebone Black, Intel Edison, and ODROID. The team decided it would be easiest to use a
Raspberry Pi due to the very widely available documentation and community support available.
It also has the lowest cost, although it has the lowest processing performance. Beaglebone
Black was also left as an option to convert to if the Raspberry Pi underperformed. The Intel
Edison was decided against because of its high price and 1.8V logic level, which would require
level shifting to interface with microcontrollers. The ODROID is very powerful, but lacks the
community support and documentation the others have, so the team did not feel comfortable
working with the ODROID in our limited time period.
The team planned on purchasing two Raspberry Pi devices; one for image processing
and the other for the navigation and system master processor.
In February of MSDII, the new Raspberry Pi 2 was announced and released. This new
version is vastly more powerful featuring a quad-core processor while retaining compatibility
with existing Raspberry Pi support and the same cost. The team was lucky enough to purchase
one before stock ran out. This solidified the use of a Raspberry Pi and reduced the need to only
having one Raspberry Pi on the craft, saving both weight and electrical power requirements.
The team needed to decide on which camera to use. The camera needed to be capable
of capturing IR spectrum and lightweight. All CCD and CMOS cameras can capture the IR
spectrum, but manufactures generally ad an IR blocking filter over the camera sensor. The basic
plan was originally to use some sort of USB webcam capable of 720p streaming resolutions and
the IR filter was either non-existent or easy to remove.
The Mobius camera was the original choice of purchase do to its popularity in the
hobbyist community, durability, and simplicity of removing the IR filter. Unfortunately, after
purchase, it was found that the Mobius camera has poor features when using as live stream
USB device. It was missing critical features such as exposure control and limited resolutions that
are available when using as a recording device. This was unfortunate and not something that
was well documented on the official site and hobbyist community forums. The team decided to
just use the Mobius camera as the image taking camera.
It was discovered that the Raspberry Pi had an official camera, the Pi NoIR camera,
which was intentionally designed to have no IR filter. The team chose this camera due to its
official Raspberry Pi driver support, high resolution, great IR detection, and extremely small size
and weight.
Algorithm Design
The basic principle behind the LIDAR scanning is using 3D projection effects when the
cameral and laser are offset from each other. The equation for translating pixels on a camera to
real world coordinates is
[𝑥
𝑦
1]=[𝑋
𝑌
𝑍
𝑅
1] [ ] [𝐾]
𝑡
where x and y are pixel coordinates; X, Y, and Z are real world coordinates; R is a 3x3
rotation matrix; t is a 1x3 translation matrix; K is a 3x3 matrix of the camera’s intrinsic
parameters. For the purpose of LIDAR scanning, R and t are derived from the camera’s relative
position to the laser. K needs to be obtained through camera calibration software, such as in
Matlab Image Processing Toolbox which the team used.
The top level process of the LIDAR algorithm is designed as follows:
Capture Image
Locate Line Pixels
Convert Pixels to
Realword
Coordinates
Inform Mapping
Function of Object
Coordinates
The image processing algorithm designed as follows:
Capture image at full
sensor pixel density
Create binary image by
applying a minimum
brightness threshold
Segment Image into
vertical blocks
Convert centroid pixel
coordinates to realworld
coordinates relative to
craft.
Filter out likely centroid
false positives
Apply blob centroid
finding to each block
indepentently
Convert relative
coordinates to absulote
using current craft angle
and location.
Plot new coordinates on
map.
Error check against map's
currently plotted points