Tennessee State University
College of Engineering
ENGINEERING RESEARCH INSTITUTE (ERI)
Interdisciplinary Research in Robotics
Intelligent Tactical Mobility Research Laboratory (ITMRL)
Intelligent Control Systems Laboratory (ICS)
Center for Neural Engineering (CNE)
Computer and information systems Laboratory (CISE)
Mohan J. Malkani, Ph.D. (Director)
(615) 963-5400 Fax: (615) 963-5397
mmalkani@tnstate.edu
Research Projects in Robotics: Past and Present

Tele-Robotics jointly with Caltech funded by NSF (1997-2000)

Originally Funded by US Army TACOM, Warren, MI, under two research
grant contracts:
1. Development of an Integrated High-level Mobility Controller for Virtual
Tandem Robotic Vehicles, DAAE07-98-C-0029, (1997-2000)
2. Deliberative, Reactive, and Adaptive Task Planning of Intelligent
Cooperative Mobile Robots, DAAE07-01-C-L-065, (2001-2002)



Embodiment of intelligent behaviors on mobile robots using fuzzy-genetic
algorithms, funded by NASA/Ames Research Center (2000-2004)
Funded by DARPA Through Penn State Applied Research “Sensor
surveillance” under MURI-ESP Research Project, DAAD19-01-1-0504,
(2002-2003)
Funded by NASA/JPL, FAR Investigator Program, “Visual Telerobotic Task
Planning of Cooperative Mobile Robots”,
(2003-2006)
Research Focus Areas

Development of Advanced Control schemes that enable tactical team
cooperation of Intelligent Autonomous robots effectively and efficiently.

Test and evaluate performance of advanced control schemes under
different operational conditions and different sensory data modality
experimentally using high-fidelity computer generated simulation and
physical robotic test beds.

Technical Competency Areas Included:
 Behavior-based Distributed control of Cooperative Mobile Robots.
 Sensory data and image processing and fusion for fault tolerance
control of intelligent robots.
 Advanced control schemes based Soft Computing techniques, (Neural
Networks, Fuzzy Logic, Genetic Algorithms, …).
 High-fidelity world perception modeling of robotic systems.
 Man-machine development for Visual Teleoperation and Telerobotic
control of Cooperative Robots.
Theoretical and Experimental
Research Capabilities

Developed various behavior-based schemes for
intelligent deliberative, reactive, and adaptive
task planning of cooperative robots.

Developed various image processing techniques
for visual localization and target tracking of
robots.

Applied different soft computing methods for
target pattern recognition and classification.

Developed FMCell comprehensive robotic
simulation software for the purpose of manmachine interface development.

State-of-the-art physical robotic test bed
consisting of twelve heterogeneous robots.

Embodiment of intelligent behaviors on mobile
robots using fuzzy-genetic algorithms
Image Captured
By Anchor Robot
Noise
Reduction
Robots
Isolation
Image
Enhancement
Image
Windowing
Robots Pose Detection
Using Neural Nets
HD: 6.1
LD:-2.2
Ro:-90.0
ID : 1
Background
Elimination
Robot Identification By
Color Feature Detection
HD: 9.3
LD: 2.9
RO:-82.5
ID : 2
HD: 2.0
LD: 2.6
RO:-45.0
Robots
ID : 3
Localization
Intelligent Man-Machine Interface

Interactive Component Based Architecture for
rapid task deployment of cooperative robots.

Image and sensory data processing and analysis
capability for intelligent control of autonomous
robots.

Soft computing capability for deliberative,
reactive, and adaptive development of behaviorbased robot tactical schemes.

3D modeling and simulation tools for world
perception modeling and visualization of
cooperative mobile robots.

Built-in TCP/IP wireless communication
protocols for distributed client/server-based
control of remotely operating robots.

Experimental human-robot interaction
Robotics Research
Multi-Robot Cooperation
Telerobotics
Intelligent User Interfaces
Interoperability
Software Architectures
Human-Robot Interaction
Intelligent Control Systems
Robotics Research
•
Human-Robot Interaction
– Over the Internet; Via PDAs; Via Speech
– Via cellular phones (speech integrated)
– Human detection, recognition, and localization
– Social behavior modeling Interoperability for
Robotics
– Programming language and operating system
independent software architecture
•
Intelligent User Interface Design
– Adaptive - mission aware
– Multiple users – multiple robots
•
Heterogeneous Multi-Robot Cooperation
– Behavior-based approach
The Human Agent System
Human Agent
Human
Detection
Agent (motion)
Human
Detection
Agent (sound)
Monitoring Agent
Interaction Agent
Observer
Agent
Human
Affect
Agent
Affect
Estimation
Agent
Human
Identification
Agent (face)
Human
Identification
Agent (voice)
Identification
Agent
Social
Agent
To Self
Agent
Human
Intention
Agent
• The Human Agent
is a virtual agent
that serves as an
internal active
representation of
people in the
robot’s
environment.
Human Database
•As a representation, it is able to detect, represent and monitor people. The description active is used, much
as in describing active perception vision systems [Bajcsy 1987], to indicate that the system can take action
to make its representation richer.
Sensory EgoSphere (SES) for Mobile Robots
•
Peters redefined the
Sensory EgoSphere as a
sparse spatiotemporally
indexed short term memory
(STM).
•
Structure: a variable
density geodesic dome.
•
Nodes: links to data
structures and files.
•
Indexed by azimuth,
elevation and time.
•
Searchable by location and
content.
images
sonar
laser
Peters, R. A. II, K. E. Hambuchen, K. Kawamura, and D. M. Wilkes, “The Sensory Ego-Sphere as a ShortTerm Memory for Humanoids”, Proc. IEEE-RAS Int’l. Conf. on Humanoid Robots, pp. 451-459, Waseda
University, Tokyo, Japan, 22-24 Nov. 2001.
Experimental Design
• 2 training tasks with the
original and enhanced
interface
• 2 teleoperation tasks with
the enhanced and orignal
interface.
Telepresence Software Architecture (Over Internet)
USER
Internet Control (Client Side)
TCP/IP
(Internet)
Robot Control Programs
API
SERVERS
Hardware
Internet Control (ServerSide)
Research Motivations (Consumer Tele-presence)
Human
Commander
Soldiers
TCP/IP
Internet
Audio
Commands
Robot
Commander
Speech
Recognition
System Architecture
Final Decision
(Fuzzy Logic)
Fuzzy Decision-2
Fuzzy Decision-3
Fuzzy Decision-1
MANAGER
Grab
Robot-1 Grabs
an Image
Grab
Robot-2 Grabs
an Image
Grab
Robot-3 Grabs
an Image
Image
Image
Image
Process
(NN-Fuzzy)
Process
(NN-Fuzzy)
Process
(NN-Fuzzy)
Research Motivations (Development of Robot Behaviors, NASA, Phase-I)
FIRBA Implementation


Abstracts beeSoft:
Complex API protocols
are hidden
Object Oriented:
Abstraction, reuse by
inheritance.

Perception Sharing:
Common perceptions can
be shared

Action Suggestions:
Arbitration through MAL,
fuzzy inference and Defuzzification.

SONAR
CLASS
ODOMETER
CLASS
TARGET
CLASS
PATH
CLASS
SONAR
HANDLING
ODOMETER
HANDLING
TARGET
HANDLING
PATH
HANDLING
SONAR PERCEPTIONS
LEVEL 0 BEHAVIORS
TARGET PERCEPTIONS
MOTION
PRIMITIVES
LEVEL 1 BEHAVIORS
Overall Software Architecture.
Independent Behaviors
Research Motivations (Development of Robot Behaviors, NASA, Phase-I)
FIRBA – Robot Control System
Complexity • Robustness
• Multiple Sensors
• Multiple Methods
• Integration
• Incremental Development
• Software
SENSORS
Pre-Perception Processing
Perception Capabilities
Behaviors.
Action Capabilities
This complexity is handled by system
decomposition in terms of :
-- functional units
-- behavioral units
Action Execution
ACTUATORS
The FIRBA architecture.