Nathan Bossart, Joe Mayer, Bob Urberger
RASCAL ACIP
Team Introduction
http://acip.us
 Facilitator: Bob Urberger
 Computer Engineering majors
 Space Systems Research Lab

RASCAL Mission


“Rascal is a two-spacecraft
mission to demonstrate
key technologies for
proximity operations…”
“After the on-orbit
checkout, one 3U
spacecraft is released and
passively drifts away....
After a suitable distance,
the released spacecraft
will activate its propulsion
system and return to within
a few meters of the base.
The second spacecraft will
be released and the
process repeated...”
RASCAL ACIP
Imaging Payload

Awareness of Cubesat
Environment
 Computer Vision

Low-Level Processing
 Object Detection
 Distance Determination

High-Level Data
 Navigation
 Thruster Control
RASCAL ACIP
Functional Breakdown
Unique Face Identifier
Known
Pattern
Capture Images
Raw
Data
Transfer Data
Structured
Data
Process Images
High-Level
Data
Output/Store Control Data
RASCAL ACIP
Modules
LEDs
Unique Face Identification
Camera
Capture Images
Transfer Data
Computational
Hardware
Process Images
Output/Store Control Data
RASCAL ACIP
LEDs

Required to perform
classification

 Not enough detail
visible for other features

 Fails when face partially
visible
Three approaches
 Unique pattern of LEDs
for each face
 Unique combination of
colors for each face
 Both unique patterns
and colors
RASCAL ACIP
Unique pattern works
regardless of camera
spectrum

Color combinations
only work with visible
spectrum cameras
 Can classify cube
corners as well as faces
with well chosen color
patterns
Camera


Potential Camera Choices:
FLIR Tau 640


 640x480, 14-bit

 PCB integration
 Control signaling simple
 Low resolution,
Visible spectrum image
sensor
monochromatic
 16.1 MB/s input data rate @
30Hz
 2-5MP - 16 to 24 bit color

Parallel data output from
cameras
Component Requirements:
FLIR

Visible spectrum
 Requires lens fixture
 Complex control signaling
 High resolution, wide color
range
 2MP with 24 bit color:
○ 57.6 MB/s input data rate
@ 30Hz
RASCAL ACIP
Processing Hardware




Processing blocks in hardware
Caching and system control managed in software
Timing and Gate Consumption
Alternatives:
 Pure software implementation
 Pure hardware implementation
RASCAL ACIP
Imaging Functions
Image Processing
Distance
Data
Distance
Detection
Image
Data
Structure
PreProcessing
Objects
In Frame
Object
Detection
Object
Classification
Image
Edges
RASCAL ACIP
Preprocessing

Noise suppression

Color Conversion

Object enhancement

Image segmentation

Conversion and downsampling
RASCAL ACIP
Distance Detection


Identify depth from a
single image
Monocular Cues
 Relative size
Structured geometry
identified should be
easy to identify
scale
 Comparison of
 regular structure
imaged objects to
known shape scale at
particular depth
RASCAL ACIP

 Square or equilateral
triangle
Distance Detection in RASCAL

Square LED pattern on spacecraft face
 Critical point identification
 Homography estimation
 Projective transform
 Point correspondence for scale

Hardware Domain
 Parallel matrix multiplication
RASCAL ACIP
Object Detection

Identifying Objects
in an Image

Region or Contour
Based

Edge Detection
 Relies heavily on Pre-
processing
RASCAL ACIP
(Columbia University)
Object Detection in RASCAL

Hardware Domain
 Canny/Deriche
 Sobel Operator

Constraints
 Cubesat size
 Environmental
 Resolution
(Columbia University)
RASCAL ACIP
Objection Classification



Post-object
detection / image
segmentation
Support vector
machine (metric
space classification)
Assign a class
based upon preprogrammed control
data
RASCAL ACIP
Object Classification in RASCAL






RASCAL ACIP
Completed with baremetal software
ARM Assembly / C
Minimum distance
principle (efficient)
Multi-tiered and/or multidimensional space from
attributes given
Determine a number of
attributes with significant
differences between
faces
Testing: expect a very
high level (>95%) of
correct classifications
Constraints of Object
Classification



Must work with a
variety of
backgrounds (Earth,
Moon, Sun, Space,
etc.)
Ideally real time
(bounded) and low
latency
Updated at >=10 Hz



Must function with
different sizes
(patterns can vary
from a few pels to
larger than the frame)
Definitive
discrimination
functions with high
reliability
Alternative algorithm:
neural nets, fuzzy
logic
Output to Control



System will output calculated information about
placement, attitude, distance, etc.
In the future, a separate team will construct a
system to interpret data and convert to control
signals/data
Since this is out of the scope of our project, the
output format/setup is ultimately our choice
RASCAL ACIP
Functional Testing

Output Unique Pattern

Process Image
 Software Verification

 Hardware Verification
Capture Images
 Stream Images
 Verify Control Signals

Output/Store Data
 Buffer

Transfer Data
 Oscilloscope
 Frame Buffer
RASCAL ACIP
System Testing



Camera integration
Hardware timing constraints
Block connectivity verification
 Blocks signal each other as intended

Full pipeline simulation
 Blocks interact as expected

Physical synthesis testing
 Data produced from each frame
RASCAL ACIP
Timeline
10/28
Project Task
Data In and Out of System
Obtain Camera
Separate Processing into Blocks
Interface Camera with Hardware
Algorithm Verification in Software
Store Camera Data in Hardware
Preprocess Image
Achieve Block Functionalality
Merge Processing Blocks
Confirm Full Integration
Project Wrap-Up
RASCAL ACIP
11/17
12/7
12/27
Dates of Years 2013-2014
1/16
2/5
2/25
3/17
4/6
4/26
Estimated Costs
Designed for very low budget and small
amount of needed materials
 Largely out of SSRL funding

Function
Part
Obtain vision data
Camera
Low Estimate High Estimate Notes
$4,000
$10,000 SSRL funding
Camera Specification
$0
$10,000 SSRL funding
Display patterns
LEDs
$1
$10 SSRL funding
Algorithm Resources
Books
$0
$0 library & creative commons
Development Tools
Zedboard
$0
$0 donation
Xilinx Vivado
$0
$0 donation
Oscilloscope
$0
$0 provided
Desktop PC
$0
$0 provided
RASCAL ACIP
TOTAL COST:
$4,001
$20,010
Future Work for Integration




Thruster control
system
Placement into
spacecraft
Radiation, vibration,
space-readiness
We will provide
thorough
documentation for
future groups
RASCAL ACIP
Bibliography
http://cubesat.slu.edu/AstroLab/SLU-03__Rascal.html
 Jan Erik Solem, Programming Computer Vision with
Python. Creative Commons.
 Dr. Ebel, Conversation
 Dr. Fritts, Conversation
 Dr. Mitchell, Conversation
 Milan Sonka, Vaclav Hlavac, Roger Boyle, Image
Processing, Analysis, and Machine Vision. Cengage
Learning; 3rd edition.
 http://www.cs.columbia.edu/~jebara/htmlpapers/UTHES
IS/node14.html

RASCAL ACIP
Questions?
RASCAL ACIP