Biomimetic Sensing for
Robotic Manipulation
Neil Petroff, Ph. D. Candidate
University of Notre Dame
Lerner Research Institute
Cleveland, OH
December 8, 2005
• Me on Me
• Grasping
Outline
– biology as motivation for current work
• Robotic Manipulation
– Nonholonomic motion planning
– Motion planning for stratified systems
• Open-Chain Manipulators
– Forward kinematics
– Inverse kinematics
• Biomimetic Robot Sensors
– Vision, touch
• Control Perspective on Deep Brain Stimulation
• The Rest of the Story
Hand Orthosis
Target Group: C5 - C7 SCI
• 3 Grasps
– Fingertip, key, cylindrical
• Increase Autonomy
• Mercury Orthotics
– Rehabilitation technology
• therapeutic
• quality of life
Grasping
• Interaction
• Creation
• Task Execution
Grasping
Hand Orthosis
Robotic Manipulation
Fuzzy Logic
Open-Chain Manipulators
Biomimetic Robot Sensors
Work to Date
Grasping
Robots
Humans
Poor at fine motion
good at fine motion
No feedback
vision, proprioception
structured
precise
rapid
strong
stamina
adaptive
robust
slow
variable
need to rest
Can we improve robotic manipulation by imbuing robots with
useful human characteristics?
Grasping
Hand Orthosis
Robotic
Manipulation
Fuzzy Logic
Open-Chain Manipulators
Biomimetic Robot Sensors
Work to Date
Biological Motivation
• Haptic Recognition
– Force feedback
• Compliance is Useful for Manipulation
• Brain Model
– Fuzzy logic
• Hierarchical Control
Grasping
Hand Orthosis
Robotic
Manipulation
Fuzzy Logic
Open-Chain Manipulators
Biomimetic Robot Sensors
Work to Date
Biological Control Loop
desired
task
motion planning
algorithm
trajectory
adjustment
inverse
kinematics
fuzzy
supervisor
encoder
counts
sensor
readings
PID
Robot
encoder
counts
current
configuration
Testbed
Robotic Motion Planning
• Steering Using Piecewise Constant Inputs
– This is a geometric analysis
– Provides a systematic approach for establishing controllability
– Applicable to underactuated systems with nonholonomic
constraints
– Exact for nilpotent systems of the form
x  g1 ( x)u1  g 2 ( x)u2 
 g m ( x)um
• Driftless
• Not all gi’s may exist
• a system is nilpotent if all Lie brackets greater than a certain order are
zero
– Lie bracket motions
• allows the system to move in a new direction
Lie Bracket Motions
Flow along g3 can be approximated by
flowing along g1 and g2
Higher order brackets can be generated, e.g.
g4   g1 , g3    g1 ,  g1, g2  
Example
Parallel parking a car
Example
Car equations

l
 x   cos  
  


y
sin

  


    1 tan   u1 
  l

    0 
  

g1
 0
 
 0
 0  u2
 
1
 

x, y 
g2
sin



l cos 2 
 0 
 cos 
 0 

 g   l cos2  

g 3   1  , 4 
 0

 l cos2  


 0 
 0



Extended System
x  g1u1  g2u2  g3v1  g 4v2
Car Simulation
Why Didn’t it Work?
• The Car Model is not Nilpotent
– g5 points in the same direction as g3
– Motion along lower order brackets induces motion
along higher order brackets
• Solution
– Iterate
– Feedback nilpotentization
• Other Drawbacks
– Small Time or Small Inputs
• obstacle avoidance
– Open Loop
• highly susceptible to modeling errors
• no error correction
Stratified Systems
• Extends motion planning algorithm to systems with discontinuities
– Intermittent contact
• locomotion
• manipulation
Neither finger
in contact
S2
g2,2
finger 2 in
contact
S1
M=S0
g2,1
-g1,1
-g2,1
g1,1
stratum
g1,2
finger 1 in
contact
S12
Both fingers
in contact
Control Architecture
Desired
task
motion planning
algorithm
Open-Chain Manipulators
Forward kinematics
A configuration is of the form
R
P

g



0
0
0
1


Product-of-exponentials formula
g st    e
ˆ11
ˆ66
e
g st  0 
P
s
T
Inverse Kinematics
The inverse kinematics solution is not unique
1
2  0
(1, 1)
(1, 1)
1
1
1  90
1  0
 2  90
1
Inverse Kinematics
• PUMA geometry makes an analytical solution tractable
e3 pw  pb  g d g st1 0 pw  pb
ˆ
Inverse Kinematics
14” diameter circle
Control Architecture
Desired
task
motion planning
algorithm
inverse
kinematics
fuzzy
supervisor
encoder
counts
PID
current
Robot configuration
current
counts
Biomimetic Sensing
Force Sensors
• Feedback at Finger/Object Junction
• Piezoelectric
– Used in biomedical testing
– Compliant
– Tend to drift under static load
• Flexiforce Sensor
Finding an Object
Control Architecture
desired
task
motion planning
algorithm
trajectory
adjustment
inverse
kinematics
fuzzy
supervisor
encoder
counts
sensor
readings
PID
Robot
encoder
counts
current
configuration
Summary
• So Far
– Built a closed loop system to perform robotic
manipulation
• stratified motion planning
• inverse kinematics solution
• force feedback
• To Do
– Manipulation
• Currently working on simulation
• apply to robots
Control Perspective on DBS
(or “What the heck am I doing here?”)
• Underlying manipulation technique is a geometric
approach to nonlinear controls
• Nonlinear control lies at the forefront of modern control
methods
• One of the most intriguing aspects of nonlinearity is that
of chaos
• Nonlinear control techniques have been used to
suppress cardiac arrythmia, a chaotic process
• Is neuron transmission chaotic?
– at the heart of successful treatments using deep brain
stimulation is the ability to control chaos
• Robust and nonlinear control techniques provide an
analytical foundation on which to study such systems
Open Questions on DBS
• By approaching DBS from a control Theory
Standpoint, Can We
– Control with external stimulation locally?
• Filter the signals?
– Characterize which signals cause which disruptions
• stimulation can suppress dyskinesia
• tremors tend to lessen during movement
• Keep symptoms from returning with fatique?
– Muscle spasticity
• Completely eliminate meds?
The Rest of the Story
•
54,000 SCI
–
Additional 2,800 / yr at C5 – C6 level
•
Parkinson’s affects 750,000 – 1 million people in the U.S.
•
Other Pathologies
–
–
–
Hemiplegic stroke
Multiple sclerosis
Muscular dystrophy
•
Rehab
•
Funding
–
•
Competition for startup money
Who Can Pay?
–
Hand Mentor from KMI
•
•
•
–
$3,950
Coverage from private insurance companies in only 2 states
Currently no medicare coverage
State of Indiana Home and Community Based Care Act
•
•
Provides funding for community and home-based care
2002: 84 / 16
•
•
Medicaid savings of $1,300 per client per month
Savings on the order of 3:1 when compared with institutional care
My Plea
• As researchers, I believe we have a responsibility
to pursue noble goals
• Obligation of the Engineer
– “… conscious always that my skill caries with it the
obligation to serve humanity …”
• Hippocratic Oath
– “I will remember that I do not treat a fever chart, a
cancerous growth, but a sick human being, whose
illness may affect the person's family and economic
stability. My responsibility includes these related
problems, if I am to care adequately for the sick.”
– “will remember that I remain a member of society, with
special obligations to all my fellow human beings,
those sound of mind and body as well as the infirm.”
On a Lighter Note