San Diego State University
College of Engineering
A Web-Based Mobile Robotic System for Control
and Sensor Fusion Studies
1
2
3
1
Christopher Paolini , Gerold Huber , Quentin Collier and Gordon K. Lee
1Dept
of Elect &Comp Engr 2Management Center
5500 Campanile Drive
Innsbruck
San Diego State University University of Applied
San Diego, CA 92182
Sciences
Universitätsstraße 15
6020 Innsbruck, Austria
3IUT
de Bethune
Networks & Telecomm Dept.
62408 Bethune, France
San Diego State University
College of Engineering
Outline of Presentation:
 The Mobile Robotic System Overview
 The ANFIS Algorithm
 Sensor Integration
 Graphics User-Interface
 Results
 Conclusions and Future Work
San Diego State University
College of Engineering
Goal:
Develop a mobile robotic testbed to investigate
several control algorithms and sensor fusion
techniques
Approach:
Use web-based video streaming and embedded
control architecture for flexibility and robustness
San Diego State University
College of Engineering
iRobot Create® Platform
• Single board computer
(SBC)
• Voyage Linux (Debian)
• Unibrain Fire-i™ digital
camera (IEEE 1394)
• Proxima 802.11g PCMCIA
adapter card with external
5db gain antenna
•
•
•
•
•
Single Board Computer (SBC)
Linux Voyage
Unibrain Fire-i™ Digital Camera
Linksys Wireless-G PC adapter
card
iRobot Create® platform
San Diego State University
College of Engineering
iRobot Create® with Sensor Arrays
Experimental iRobot developed at San Diego State
University
•
•
•
•
iRobot Create® based robot designed with several sensors
Unibrain Fire-i™ Digital Camera
Streaming video
Web teleoperation
IR Sensor
2.5dBi gain indoor omnidirectional antenna
ANFIS automation
Thermal Sensor
Arduino Mega MCU
9DOF Inertial Measurement
Unit
Migrus C787 DCF-P single
board computer with a 1.2GHz
Eden ULV Processor
360° Ultrasonic Sensor Array
802.11g PCMCIA
Transceiver
San Diego State University
College of Engineering
Sensors
3-Axis Rate Gyroscope
3-Axis Accelerometer
3-Axis Magnetometer
Inertial Measurement Unit Controller
San Diego State University
College of Engineering
Ultrasonic sensor array using an Arduino Mega 2560 microcontroller
San Diego State University
College of Engineering
Ultrasonic Sensor Array
MaxBotix LV-EZ1 Ultrasonic Sensor
An array of 10 MaxBotix LV-EZ1 sensors
suspended on two circular plates.
Each MaxBotix sensor provides a 36 degree FOV.
San Diego State University
College of Engineering
Thermal Sensor
How the iRobot Adjusts Its Heading


r1
Large radius  small curvature
Small radius  large curvature
r2
The iRobot changes its azimuth by sending a 16 bit
signed value in the range [-2000, 2000] mm that
defines a turning radius
The turning radius is a ray from the center of the
turning circle to the center of the robot



r > 0  robot turns left


r = 0xFFFF  robot turns in place clockwise
r < 0  robot turns right
Special cases: r = 32768 or 32767 (0x8000 or
0x7FFF) causes robot to move straight
r = 0x0001  robot turns in place counter-clockwise
How the Web GUI Determines the Turning Radius
 1
   tan

(x,y)
x
y 
In quadrant I and IV for >5
II
r  23.8  2142.79
I
In quadrant II and III for <-5

r  23.8  2142.79
III
IV
Assume r varies linearly
with 
Inertial Measurement Unit (IMU)
•
•
•
•
•
•
Robot has an onboard 9DOF IMU
Incorporates four sensors: LY530ALH single-axis
gyro, LPR530AL dual-axis gyro, ADXL345 tripleaxis accelerometer, and a HMC5843 triple-axis
magnetometer
Gives nine degrees of inertial measurement
LY530ALH: STMicroelectronics ±300 °/s analog
yaw-rate gyroscope
HMC5843: Honeywell HMC5843, a 3-axis digital
magnetometer outputs Euler X,Y,Z orientation vectors
and roll () and pitch () angles (tilt sensor) with 12- xh  x cos   y sin  sin   z cos  sin 
yh  y cos   z sin 
bit ADC at 10 Hz
MCU computes azimuth or “yaw” with an accuracy of Az  tan 1 yh
xh
1-2 º
How to Model Robot Dynamics while Turning?
•
•
•
•
We want the absolute bearing defined by the
remote user with the GUI to equal the absolute
bearing reported by the IMU
How to compensate for unknown system
dynamics?
Can define a neural network to model robot (plant)
dynamics and use training data to tune network
parameters
i
i
i
i
Define a set of 4-tuple training data: d d a a
which are the ith desired (from GUI) azimuth,
azimuth rate, actual (from IMU) azimuth and
azimuth rate, respectfully.
From the HMC5843 digital
magnetometer
From the LY530ALH yaw
rate sensor
 d1  d1  a1  a1 
 2
2
2
2
 d  d  a  a 


 n
n
n
n
 d  d  a  a 
San Diego State University
College of Engineering
ADAPTIVE AJAX-BASED STREAMING VIDEO SYSTEM
AJAX based Web interface for
telerobotic control of the iRobot Create
Encoding bit rate as a function of fps
Virtual Force Field Approach
for Obstacle Avoidance
The Certainty Grid
•
•
•
•
•
•
2-Dimensional array of cells
Each cell contains a Certainty Value (CV)
CV indicates the measure of confidence that an object
exist within a cell
Instantaneous map for obstacle representation
dx = dy = 15 cm
21 by 21 square cells represent the Certainty Grid
Front
Method for Updating Certainty Values
(1)
Each sensor corresponds to a particular angle Ө,
based on its position on the sensor assembly
At a given time, a sensor returns a distance d
(2)
Eq. 1 and 2 transform (d, Ө) → (x’,y’)
Obstacle Avoidance for Path Planning Task
Port Side
Front
San Diego State University
College of Engineering
Cells Located on the Acoustic Axis
San Diego State University
College of Engineering
(a) Histogram Grid ; (b) Snapshot of Video Camera
College of Engineering
San Diego State University
The ANFIS Architecture
layer 1
layer 4
layer 2
layer 3
xy
A1
x
A2

B1

w1
w2

w1

w2
layer 5
w 1f 1
w 2f 2
y
xy
B2
If x is Ai and y is Bj, then f i  p i x  q i y  ri
nr
f
 w ifi
i 1

f
College of Engineering
San Diego State University
Off-line Training
Forward pass: consequent parameters



Sia i1a T
S
i1 i
Si1  Si 
, i  0, 1, ..., P  1
T
1  a i 1Sia i 1

T
X i1  Xi  Si1 a i1 (b T

a
i1
i 1Xi )
Backwards pass: premise parameters
P
P
i1
i1
2
E   Ei   (y di  y i )
P 2
  ei
i 1
San Diego State University
College of Engineering
On-line Learning
Forward pass: consequent parameters
Backwards pass: premise parameters
San Diego State University
College of Engineering
San Diego State University
College of Engineering
Simulation Results
System Integration through an Arduino MCU
• Multiple sensors: thermal (person/fixture differentiation), ultrasonic
(collision avoidance and path planning), IR (automatic docking),
video (telerobotic control), and magnetometer and accelerometer
(orientation and position) have been tested and integrated into the
overall system architecture.



ANFIS controller is being implemented on an Arduino Mega that
uses the magnetometer and accelerometer output for path tracking.
Initial simulation studies have been conducted using a MISO
controller to test this proof of concept.
Then performed actual experimentation using one and two inputs to
the ANFIS controller
Initial Web Browser Interface
San Diego State University
College of Engineering
San Diego State University
College of Engineering
Experimental Results
Bearing Scenario
San Diego State University
College of Engineering
Bearing Scenario
San Diego State University
College of Engineering
Constant Turn
San Diego State University
College of Engineering
Following Scenario
San Diego State University
College of Engineering
Learning Scenario
San Diego State University
College of Engineering
ANFIS Responding Time
Inputs
Membership-functions
1
2
3-4
16-18
3
7-8
20-23
4
9-11
23-26
5
13-15
26-29
6
17-19
30-32
7
22-23
36-38
2
12-14
26-28
3
30-31
44-46
4
80-82
94-96
2
ANFIS Computation Time
Response Time
San Diego State University
College of Engineering
Conclusions and Future Work
•
A MIMO ANFIS controller has been designed and tested through simulation
and experimental studies
•
The desired controller can adaptively adjusting to system variations through
supervised and un-supervised learning.
•
Future tasks include extending the MIMO design to a multiple inputs, two
output structure and evaluate the performance of this MIMO implementation
using Player/Stage and experimentation.
•
We will add on-line learning functionality to our embedded MIMO ANFIS
that will effectively tune the parameters computed from off-line training data.
San Diego State University
College of Engineering
San Diego State University
College of Engineering