Grand Challenges in Neural
Engineering
Restoring Voluntary Function in
Artificial Limbs
Todd A. Kuiken, MD, PhD
Neural Engineering Center for Artificial Limbs
Rehabilitation Institute of Chicago
Department of PM&R, BME and Surgery, Northwestern University
May 2010
Body-Powered Prostheses
Developed in the Civil War – refined in
WWII
Moving shoulders forward pulls on a
bicycle cable
Bicycle cable operates hook or hand and
elbow
Myoelectric Prostheses
“Myo” - muscle
When muscles contract, they generate
electric signals called “myoelectric
signals”
Electrodes on the skin over muscles can
pick up these signals. The signals are
then used to tell a motorized arm what to
do.
We Need a Neural Interface…
We Need a Neural Interface…
To acquire motor control data
To stimulate the afferent system
Options:
– EMG from residual limbs
• current standard
• Limited data available with high level amputations
– Direct peripheral nerve recording
• Compelling method to get both motor control data and be able to
stimulate afferents. Exciting demonstrations in humans
• Technically very challenging—not seen clinical deployment yet
– Spinal Cord
– Brain machine interfacing
– Targeted Reinnervation
• A pragmatic use of muscle as a biological amplifier of peripheral
nerve motor signals and portal to cutaneous sensation
Targeted Muscle Reinnervation
TECHNIQUE
– Residual nerves transferred to spare muscle
and skin.
– Muscle acts as a ‘biological amplifier’ of the
motor command
ADVANTAGES
– Additional control signals for
simultaneous control of
more degrees-of-freedom
– Control signals are
physiologically appropriate
• More natural feel
• Easier, more intuitive
operation
– Shoulder still available for
controlling other functions
– No implanted hardware
required
– Can use existing
myoelectric prosthetic
technology
DISADVANTAGE
– Requires additional surgery
• unless it is done at time of
amputation
Motion During Contractions
Blocks and Box Test
Original Prosthesis
Nerve Transfer Prosthesis
(Used more than 20 months)
(Used about 2 months)
Blocks and Box Test
Original Prosthesis
(Used more than 8 months)
Nerve Transfer Prosthesis
(Used about 2 months)
Sensory Reinnervation Studies
Jesse Sullivan Sensory Map
Paul Marasco
Targeted Sensory Reinnervation
TECHNIQUE
– Denervate residual limb skin to allow the hand
afferents to reinnervate this skin
– Stimuli detected by sensors in prosthetic hand can be
applied to reinnervated skin
Creates a portal to sensory pathways
POTENTIAL ADVANTAGES
– Provides physiologically appropriate sensory
feedback
– Provides anatomically appropriate sensory feedback
CONTROLLER
TOUCH SENSORS
TACTOR
Targeted Reinnervation
Functional Outcomes
Functional Outcomes of 1st six
patients
2.5-7 times faster on Block
and Box test
50% faster on Clothes Pin
test
Improvement in speed on all
Wolf Motor Functions tests
Significant improvement in
AMPs testing
University of Alberta TMR subject
Transfer sensation in four
patients
One Unsuccessful
Transhumeral Surgery
In OR, radial nerve atrophy
discovered
Likely brachial plexopathy
40-50 patients worldwide
Vienna
University of Washington
Walter Reed Army Medical Center
Brook Army Medical Center
Edmonton, Canada
96% Surgical success rate in
producing usable EMG
signals
Two prosthetic arms systems commercially available
Liberating Technology—Boston Elbow
Otto Bock—TMR Dynamic Arm
Advanced Signal Processing
Techniques
Pattern Recognition Results
Linear Discriminant Analysis (LDA) with time domain
feature sets and a combination of autoregressive
features and the root mean square (AR+RMS) feature
sets were used.
Bipolar Electrodes
Subject
Time
Domain
AR+RMS
BSD*
98.4±0.7
97.8±1.1
STH**
90.3±2.9
87.6±2.9
LTH1
97.1
95.5
LTH2
98.3
99.2
Average
96.0±3. 95.0±5.
* average of 3 experiments9and 3 different bipolar
2
electrode configurations
** average of 2 experiments and 3 different
bipolar
Kevin Englehart
electrode configurations
UNB, BME
How Many Electrodes Do We Need?
Classification Accuracy (%)
Electrode Channel
Reduction Analysis
100
90
80
70
60
50
40
30
20
10
0
P1
P2
P3
P4
1 2 3 4 5 6 7 8 9 10 11 12 … >300
Number of Bipolar Electrodes
16 classes
– 2 elbow
– 4 wrist
– 10 hand
Courtesy of JHU-APL and RIC
Grand Challenges in Neural Engineering
Richness of Neural Interface
We want as much motor
control data as possible
– Need lots to control more
degrees of freedom
– Need separable data to control
multiple DOFs simultaneously
Back to source separation
problem
Closer to the source, generally
the better the signal
separation
Grand Challenges in Neural Engineering
Smart Decoding and Control Algorithms
Need to decode of signals
robustly
– Extract as much info as possible
– Need to ‘learn’ the patient and the
task
Constant tension between
‘smart’ devices and human
control
– Example: slip sensors
Potential solutions
• Better ‘information fusion”
• Consider time-history systems
• Adaptive algorithms
Grand Challenges in Neural Engineering
Signal Stability
– Surface EMG signals are
problematic
• Location different each
time prosthesis is
donned
• Electrodes shift with
prosthetic use
– Potential solutions
• Developing new
surface EMG interfaces
• Hoping for implantable
EMG system
Grand Challenges in Neural Engineering
Robustness of Neural Interface
Amputees are very active
– System needs to withstand repetitive
deformation
• Prosthetic sockets dig in
– System needs to withstand high
force impacts
– The flying kid test
Potential solutions
– External devices are easier
• Replaceable
• Can incase in socket
– Internal devices:
– Need to be small and tough
– And/Or they need to be
compliant like tissues
Grand Challenges in Neural Engineering
Need Multidimensional Sensation Feedback
TR can provide some cutaneous feedback
– Not enough room for electrodes and tactors
– Can’t control reinnervation process
Proprioception necessary for complex limb
system
– TR can’t provide proprioception
– Proprioception poorly understood
– Direct nerve, spinal cord and cortical
stimulation hold more promise
Sensory substitution does not work
– Can’t rely on ‘neural plasticity’ too much
– Need physiologically and anatomically
appropriate feedback
Grand Challenges in Neural Engineering
Mechatronics Challenges
Need lighter devices for amputees
– This is what patients complain about!
Need more robust devices
– They breakdown all the time…
Need more dexterous devices
– As we develop the ability to control more
DOF’s, we need more dexterous devices
– Functionally, multi-degree-of freedom wrists
are particular important
Grand Challenges in Neural Engineering
Need better attachment
systems
– Stability of control and
mechatronics depends on
mechanical fixation
– Powered orthotics are
equally (or more)
challenging
Potential Solutions
Osseointegration (direct
skeletal attachment) is
very promising for
prosthetics
From http://www.branemark.se/osseointegration.htm
Grand Challenges in Neural Engineering
Psychological Challenges
for Patient with Disabilities
Many of our technologies will be seeing deployment in humans
for the first time soon
People with severe traumatic disabilities have high incident of
psychological difficulties
– Depression, PTSD, Anxiety disorders, adjustment disorders
Recommend careful psychological screening
Recommend no media until a trial is finished and successful
– Having media follow a new patient puts too much pressure on
the patient and the clinical team
– Of course, disclose problems/failures with patients’ identity
protected at meetings, in papers and all reports.
Collaborators and Support
NECAL Team
– Todd Kuiken, MD, PhD
– Aimee Schultz, MS
– Blair Lock, MS
– Bob Parks, MBA
– Dat Tran, BS
– Kathy Stubblefield, OTR
– Laura Miller, CP, PhD
– Levi Hargrove, PhD
– Robert Lipschutz, CP
– Jon Sensinger, PhD
Northwestern University
– Gregory Dumanian, MD
– Richard Weir, PhD
– Jules Dewald, PhD
Previous Post Docs
– Nikolay Stoykov, PhD
– Madeleine Lowery, PhD
– Ping Zhou, PhD
– Helen Huang, PhD
– Paul Marasco, PhD
Collaborating Institution
– University of New
Brunswick
– Liberating Technologies
Inc
– Otto Bock, Inc.
– Deka Research, Inc
– Johns Hopkins Applied
Physics Lab
– Kinea Design
This work supported by:
The National Institutes of Health
– Grant #1K08HD01224-01A1
– Grant # R01 HD043137-01
– Grant # R01 HD044798-01
– Grant # NO1-HD-5-3402
Defense Advanced Research
Projects Agency
The National Institute of Disability
and Rehabilitation Research
US Army
Generous Philanthropic Support