Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
Artificial Intelligence in Brain-Machine Interface
Henrik Månsson
831028-3513
henma508@student.liu.se
1
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
2
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
Abstract
This paper reviews research in the field of Brain-Machine Interfaces (BMIs) and the existing
Artificial Intelligence (AI) in Neuroprosthetics to the purpose of examining the future for
BMI in particular regard to AI. The conclusive remarks are that this research is expanding a
lot at the moment and the potential for this field can result in rapid improvements in the lives
of humans in need of prostheses. One of the main factors of this development is the
implementation of AI in prostheses, something that has barely even started.
3
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
Contents
1 INTRODUCTION .............................................................................................................5
1.1 INSPIRATION ..................................................................................................................5
1.1 AIM ...............................................................................................................................5
1.3 LAYOUT ........................................................................................................................5
2 TYPES OF BMIS ..............................................................................................................6
2.1 NEURON CULTURES ........................................................................................................6
2.2 NON-INVASIVE BMIS .....................................................................................................7
2.3 INVASIVE BMIS .............................................................................................................8
2.3.1 Experiments ...........................................................................................................8
2.3.3 Cochlear Implants [13] ..........................................................................................9
2.3.4 Visual Aid ............................................................................................................11
2.4 A PERIPHERAL INVASIVE CLOSED-LOOP BMI [17] .........................................................11
2.4.1 Procedure ............................................................................................................11
2.4.2 Result...................................................................................................................12
3 AI IN NEUROPROSTHETICS ......................................................................................13
3.1 SOFTWARE IMPROVEMENTS..........................................................................................13
3.2 THE INTELLIGENT HAND [20] .......................................................................................15
3.2.1 The Intelligent Foot [22] ......................................................................................15
4 SUMMARY .....................................................................................................................16
4.1 DISCUSSION .................................................................................................................16
4.2 SUMMARY ...................................................................................................................17
5 BIBLIOGRAPHY............................................................................................................18
4
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
1 Introduction
In recent years a new field of research called Brain-Machine Interface has evolved rapidly.
The aim of this research is to develop functional communication between the human nervous
system and artificial computers or machines. It is probably considered science fiction by
many but the research is growing and showing tangible results. Not-very-modest researchers
envisage prostheses for amputees and quadriplegics controlled directly by the brain and
cyborgs; artificially enhanced humans with built-in night-vision, artificial super-strong
“muscles” etc. This brings hope to many people with impairments of all kinds of sorts. The
most prevalent aid so far is the Cochlear implant, which have helped over a hundred thousand
deaf people, giving them partial hearing. In this paper the cochlea implant will be reviewed,
as well as a series of other BMI advancements.
1.1 Inspiration
The inspiration for this paper was taken from an online article called Brain vs. Machine
Control [1]. The article suggests that the field of BMI will spawn a new kind of interactivity
between men and machine in the form of a cyborg. The cyborg-construction will inevitably
result in direct neural connections between human brains and artificial intelligence (AI). In
this case, it involves AI in the form of additional metal arms belonging to a science fiction
character. How much science and how much fiction this idea contains remains to be seen.
This particular article is of course but one among many sources, more or less fictional,
containing this idea.
1.1 Aim
The artificial extensions being researched in the field of BMI are called neuroprosthesis and
this field of research is therefore also often called Neuroprosthetics. The aim of this paper is
to summarize and analyze some important developments in BMI research and the existence of
AI in neuroprosthesis to determine the likelihood and manifestation of the development in the
above-mentioned article.
1.3 Layout
Firstly, different types of BMIs will be dissected and explained. Then some different types of
BMIs most important to the aim of this paper will be explained using a few examples. Next,
AI in neuroprosthetics will be examined and lastly follows a summary and discussion.
5
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
2 Types of BMIs
There are several different types of BMIs and neuroprostheses. The first type of BMI is
neuron cultures. Neuron cultures are simply cultures of neurons grown in labs and connected
to a computer. The purpose of neuron cultures is to examine biocompatibility, neuronal
growth in interaction with artificial materials and artificial feedback and overall neuronal
functionality. The second type of BMIs are non-invasive BMIs – they utilize EEG recordings
performed outside of brains, which has great benefits for people who suffer from spinal cord
injuries or similar and cannot employ their peripheral nervous system beyond the brain. The
third group of BMIs is invasive BMIs. They build on the knowledge of the neuron cultures,
neuroscience and essentially a whole range of research fields to produce neuroprosthesis that
can interact directly with the neural structures of humans and other creatures.
2.1 Neuron cultures
Recently Dr Steve Potter at Georgia's Institute of Technology, Atlanta, and Guy Ben-Ary at
the University of Western Australia, Perth on different sides of the Pacific together connected
a culture of rat neurons resident in Atlanta to a robotic arm in Perth. Dr Potter grew 50,000
neurons from a rat in a petri dish and connected the culture to 64 electrodes. These electrodes
picked up the electrical activity of the ‘brain’, translated them into a binary code in a
computer, which sent the code over the Internet to the lab in Perth where it was transmitted to
a robotic arm that responded to the input by moving its 3 colored markers across a canvas.
The motion of the robotic arm was being registered by the computer and sent back to Atlanta
and resulted in stimulation of the neuronal mass in the petri dish through the electrodes,
creating a closed-loop entity that would, in theory, be able to respond to the environment. [2]
At first, the drawings that the 50,000 neurons generated were chaotic, but progressed into
more stable, although meaningless paintings, implying that a responsive neural network was
at work, adapting to the outside stimulation it received. However, its communicative skills
have not progressed beyond meaningless and is not likely to do so either. This, of course, in
part due to the fact that the neurons do not have a goal with their lives; they have nothing
meaningful to communicate. This is however a philosophical issue – debating how a
meaningful life arises and from what – so that will be left for others to contemplate. The
importance of this arts-project is that it proved that it was possible to create a closed-loop
connection between a computer and a biological neural network using electrodes.
Another similar experiment involving fish neurons has also been conducted recently.
Scientists at Northwestern University, Chicago, US, connected a part of the brain of a fish – a
lamprey – to a robot with motor functions and visual sensors. The part of the brain that was
used was the part of the brain that keeps the fish in an upright position while swimming in the
water, using the sunlight as guidance for its balance. It was expected that any light source
would mimic the electrical patterns representing a sun and the part of the brain adapted to
reacting to sunshine would respond to any input by correcting its position according to the
stimulation and thus, by feeding the electrical patterns using a robot that sensed light, the
brain-part was expected to react to the stimuli, as though it would have reacted to natural
sunlight while swimming in the water. [3]
6
Artificiell Intelligens, HKGBB0
Henrik Månsson
Fördjupningsuppgift, HT –05
henma508@student.liu.se
Examinator: Arne Jönsson
The result was as expected; the brain learned to interpret the signals from the robot sensors
and learned which signals to return to the robot to move it in the direction of the light. This
goes to show the no less than incredible ability of a brain to adapt to artificial sensory input
and be able to adapt its own sensory output to machines. These abilities are very important for
future creation and perfection of neuroprostheses, which will be discussed later in the paper.
In another experiment giving support to the other findings, Thomas DeMarse at the University
of Florida grew 25,000 neurons from a rat on a grid with 60 two-way electrodes. The neural
network was hooked up to a computer program simulating a plane that could tilt to the left
and to the right. These neurons, unlike the neurons in Atlanta, did seem to be goal-oriented,
trying to keep the plane level and adapting to the different tilting stimulations and reacting to
them by sending output to alter the angle of the wings. This again shows that neurons can
learn, even when not assembled in an orderly fashion, as is the case in the brain grown
according to the DNA of an animal. Important to remember here is that learning is enabled
through the flexibility of the neural network to change and restructure itself; restructuring is
learning. [4]
2.2 Non-invasive BMIs
Other recent findings in BMI include experiments with live (whole) rats and monkeys, the
perhaps most notable ones conducted by Miguel Nicolelis, neuroscientist at Duke University,
Durham, North Carolina, and his co-workers. At first, they implanted electrodes into the
brains of rats to record their neural activity [5]. The electrodes were picking up the electrical
activity of networks of neurons, i.e. listening to several neurons processing together, instead
of just one at a time and through these experiments important novel discoveries were made.
The cortex operates using several parallel neural processes, which combined produce the
output from the brain and Nicolelis et al. found that one needed not listen to all the parallel
processes of the entire brain to retrieve vital information from it; instead registering electrical
activity from only one or a few processes was sufficient to predict the output of the entire
processing. This led to the continuation of experiments using implanted electrodes that were
connected only to a few neural compositions, without having to search the entire brain for a
specific process related to a specific output (or specific action). The implants that are used in
these kinds of experiments consist of an array of electrodes that float around in the brain
tissue registering electrical impulses without interfering or damaging the brain’s neurons and
for Nicolelis’ first tests an implant was inserted into a rat-brain to monitor as little as 46
neurons.
Similar experiments [6-8], one involving an owl monkey pressing a lever was advanced to a
new level, when a robotic arm was produced and connected to the monkey to reproduce the
motion of the simulation in a physical artificial arm [9]. The effect was that the prosthetic
limb mimicked the actual arm of the monkey to a high degree. When measuring the activity of
100 neurons the artificial arm was 70% accurate in its movement and when measuring the
activity of 500 neurons the result was 95% accuracy. In the search for functional prostheses
for disabled humans, the research has come quite far. So far the achievements, as have been
presented in this paper, involve measuring brain activity more accurately, with less intrusion
and with materials accepted by the brain, and knowledge of the required amount of neurons
needed for more accurate prediction of the output of the motor cortex, useful algorithms for
interpreting neural activity patterns and adequate physical prostheses, fine enough to replicate
the motions researched upon in this experiment.
7
Artificiell Intelligens, HKGBB0
Henrik Månsson
Fördjupningsuppgift, HT –05
henma508@student.liu.se
Examinator: Arne Jönsson
Ordinary prostheses works by myoelectrical readings, i.e. registering contractions of a
functional muscle and conveying those signals to the operation of a prostheses. But the
problem is that the muscle reading does not correspond to the prosthesis movement directly or
even to the initially intended action of the muscle from the input, instead, the patient needs to
learn to control the prosthesis by altering its way of maneuvering the muscle. Recently Todd
Kuiken displayed how to better make of use the old nerve paths that are left in patients who
have lost any of their limbs [10]. The nerve path to a hand that has been lost is still able to
forward neural information and if it is moved to another muscle then thinking of moving the
hand will move that muscle instead. Thus doing myoelectric readings of that muscle for a
hand prosthesis will make the patient move the hand by actually thinking of moving the hand,
making it all much more natural and a patient with 2 arm prostheses utilizing this technique is
called the Bionic Man.
2.3 Invasive BMIs
All neuroprostheses existing today involve transmitting sensory input data into the brain, and
the most used of these is the cochlear implant, which will be granted an in-depth dissection in
this chapter. But work has begun on similar devices meant to help the blind see again, using
the same principal of sending sensory information directly into the synoptic (and auditory
nerve respectively) bypassing the normal instruments of sight and hearing, eyes and ears.
Lastly, some speculation about the possible prostheses that might arise in the future and the
progress in closed-loop (motor cortex – somatosensory cortex) prostheses will be discussed.
2.3.1 Experiments
The experiment was to see if the rat could interact with an artifact, ultimately controlling it
with thought alone. The artifact in question was a lever that, if pressed, would grant the rat a
sip of water. A computer registered the neural activity of the 46 neurons when the lever was
pressed and then the lever was disconnected and water was distributed based only on the input
from the implant displaying the before-used neural pattern, which led to the rat learning that it
did not have to move its arm to get water, it simply had to active the same process in these 46
neurons as it automatically did when moving its arm, but this time, without moving its arm,
concentrating on only the prerequisite processing for moving the arm and not the whole
process leading to the signaling of arm movement. Whether the neurons were disconnected
from the rest of the arm moving process to be used only for the specific task of constructing
the neural pattern as a way to get water or the neurons were still an integral part of the motion
process and could be used in both ways was not revealed by this experiment but leads to
intriguing questions about what would be required of the brain of a human controlling an
artificial limb with thoughts alone.
Quadriplegic humans have also been tested upon using the same techniques [11]. A part of the
skull in a patient was removed, revealing an entry point to the motor cortex for insertion of an
implant that would be able to register the neural activity of the patient’s thoughts, concerning
motor functions. An algorithm was again developed to interpret the signals deriving from an
area of the brain that the patient had little use for, being almost completely paralyzed. His
brain was not damaged, and the motor cortex was intact although fairly unused. However,
with a computer interpreting voluntary activity of the motor cortex into cursor movement on a
computer screen, the patient learned to control and use the cursor proving the feasibility of
electrode mapping of the brain in humans as a means to control prostheses, taking the
research one step closer to constructing functional neuroprostheses for people suffering from
e.g. damage to the spinal cord.
8
Artificiell Intelligens, HKGBB0
Henrik Månsson
Fördjupningsuppgift, HT –05
henma508@student.liu.se
Examinator: Arne Jönsson
The next principal matter in this field is to enable closed-loop feedback from the prostheses
and tests aiming for this have been performed. In 2002 Chapin et al. implanted probes into
rats. 2 probes were inserted in the somatosensory cortex of the brain where the input normally
is received from stimuli of the left and right whisker respectively [12]. 1 probe was implanted
in the medial forebrain bundle. The rats were trained to navigate using input through these 3
probes, where simulated touch on the left whisker represented a left turn, and simulated touch
on the right whisker represented a right turn, and stimulation of the 3rd probe indicated
moving forward. This was learned through guiding the rats through a maze stimulating the
rats using the wireless probes that received input from the scientists observing from afar, thus
creating a remote control over the rats. After the rats had learned how to be guided through
the maze, they were tested in other environments, successfully. This is proof of the basic
feasibility of communicating with the brain through electrical stimulation.
2.3.3 Cochlear Implants [13]
The cochlear implant, also referred to as the Bionic Ear, in 1978 became the first neural
prosthesis ever to be implanted in a human. The purpose was to bypass the auditory system
transmitting sound waves to the auditory nerve to help people with hearing disorders be able
to hear speech again. The hearing disorder that can be helped through the use of a cochlear
implant includes all forms of damage to the inner hair cells. Most people with hearing
disorders suffer some kind of damage to the outer hair cells that are responsible for
amplifying the sound waves making them easier to recognize and this condition is helped with
a regular hearing aid that boosts the sounds going into the ear. The inner hair cells react by
sending an electrical impulse via the auditory nerve when stimulated by a sound wave.
Depending on the amplitude of a sound, it is passed on a certain length of the inner ear
through the spiral-shaped cochlea and sound will hit separate hair cells, at different intervals
and at a difference of depth into the cochlea making the sound distinguishable and
interpretable. Damage to the inner hair cells is therefore more complex to circumvent and
requires a device that can distinguish sounds and transfer that information as electrical
impulses to the auditory nerve.
9
Artificiell Intelligens, HKGBB0
Henrik Månsson
Fördjupningsuppgift, HT –05
henma508@student.liu.se
Examinator: Arne Jönsson
Graeme Clark, the pioneer and creator of cochlear implants, found that to mimic the cochlea it
would be necessary to send impulses of different pitches to different parts of the auditory
nerve, thus an electrode array with multiple channels was needed. The electrode array is
inserted, as the picture shows, surrounding the cochlea, connecting its electrodes to different
areas of the auditory nerve. The whole system works as follows: A sound is intercepted by a
microphone which sends the audio information to a computational device processing the
information in binary code, translating it to send it to the right part of the cochlea, with the
right intervals and pitch of the stimuli, simulating normal sound processing to the best of its
ability. Although the information is limited in resolution due to the tens of electrodes,
compared to the tens of thousands of hair cells, the brain can extract enough useful
information from the input to be able to interpret ordinary speech.
Over the years Clark and others have improved the techniques surrounding the principle;
matters worth recognition are: finding suitable materials to produce the artificial prosthesis in,
materials that would not be rejected by the immune system of the body, as well as the
discovery that, just like a fully hearing person can determine distance to the source of a sound
using both ears, when operating 2 cochlear implants a person can receive enough auditory
information to establish the source of the sound, and also progressively discovering better
ways, better formulas for interpreting the input of the microphone into efficient binary coding.
2.3.3.1 Improving the Cochlear Implant
One thing to notice is the adaptability of the brain, which can interpret the new signals almost
as well as it could once interpret sound normally, and studies conducted by Clark et al. have
shown that children born deaf in many cases develop the same language skills as children
with normal hearing. Although the brain can perhaps do a lot better with more electrodes, that
is a surgically and technically challenging task to approach and improvements to the implants
are instead more likely with the improvement of the algorithms used to interpret the sound
waves.
A recent study addresses the problem of experiencing music with current cochlear implants.
The main factor of this deficiency is said to be the lack of spectral information due to
limitations in channels. Compared to a fully functional ear, a cochlear implant only provides
one thousand of channel information or less. Sound is therefore reduced to a few amplitude
channels, which works well when there is only one clear source of sound in hearing range.
However, low-resolution amplitude is not sufficient in noisy space and not for music either.
The answer is fine-tuned frequency information to complement amplitude. Using the Hilbert
transform frequency, amplitude and time variables can be extracted and separated in the
software of the prosthesis. This meant that information could be sent temporally much more
accurate; highly resolved, down to milliseconds. [14]
The result of the study was that subjects could hear normal speech in quiet surroundings just
as good as with preexisting cochlear implants, barely better in noisy surroundings, but music
recognition raised from 30% and 40% to >90% in noisy and quiet surroundings respectively
[14]. This suggests that the potential for improvements in sound computing software is still
great. The limitations of 24-channels implants can be circumvented using cleverer computing,
although it is also obvious that finer implants would provide more potential for improved
algorithms.
10
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
2.3.4 Visual Aid
The Bionic Ear has been a success in aiding people with hearing disorders and currently
research is being carried out investigating the possibility of creating a similar device for
people with impaired vision [15-16]. Following the same principle, researchers hope to
bypass damaged retinal cells, which hold the equivalent function for visual perception as does
the inner hair cells of the cochlea hold for auditory perception. Scientists are testing a device
connected to the retina via 16 electrodes, stimulation the optic nerve. The device consists of a
range of photon receptors able to pick up light, transmitting the visual information to a
decoder that determines the amount of stimulation the electrodes should send out. As this is
all very fresh, the algorithmic developments in the cochlear implant has not been paralleled
and early results are still far from satisfying; but initially it works by principal and software
and hardware advances should follow making it possible for blind patients to see, just as it
was made possible for deaf patients to hear. However, a photon receptor is fairly simple in its
construction, like any solar cell, compared to a device for spectral analysis, which would be
required if the patient was to be able to see colors.
2.4 A peripheral invasive closed-loop BMI [17]
One of the more prominent advocates for cyborgs is Kevin Warwick. His scientific research
along with his works of fiction and science fiction prophecies has resulted in much media
attention. Warwick “becoming a cyborg” is an interesting landmark in BMI research, despite
his many critics; possibly even thanks to his many critics, given the amount of media attention
it brought to the field. In 2002, Warwick et al. conducted an experiment inserting an electrode
array into his own body connecting to the median nerve of his lower left arm.
2.4.1 Procedure
Each of the 100
electrodes in the array
was 4 m in diameter at
the platinum coated tip.
The array was inserted
into the median nerve of
the arm and the wire
bundle and electrical
connector pad was
fastened externally on the
arm. The BMI fed signals
to a multi degree of
freedom hand prosthesis
being controlled by
voluntary opening
involuntary closing
(VOIC). This means that
input tells it to open, but
Figure 1. The microelectrode array
it closes on its own. The
hand is programmed to
adjust the grip when closing using sensors to detect the force needed to grip objects, prevent
them from slipping etc. All this is done by the prosthesis autonomously. The prosthesis sent
tactile information back through the electrodes closing the loop.
11
Artificiell Intelligens, HKGBB0
Henrik Månsson
Fördjupningsuppgift, HT –05
henma508@student.liu.se
Examinator: Arne Jönsson
The signals from the brain was filtered and amplified and decoded in software by a leaky
integrator. One feed gives the signals frequency in 10-bit resolution and another records the
temporal activity of the signals. The software is not described in any more detail in the paper.
2.4.2 Result
Trials were conducted where the test subject learned to operate the hand prosthesis and
although the feed-back sensory information from the prosthesis to the brain cannot be
evaluated, control on the prosthetic hand was clearly discernible, as seen in Figure
2.
Figure 2. Success rate for subject trying to control the grasp of the hand prosthetic
Clearly, the subject has learned to control the prosthesis to a certain degree and with the help
of the hand itself, it is a possible replacement for a natural hand given some time for training
and sophisticated programming allowing the hand to correct for human error or failure in the
neural communication. To quote the paper: “it has been shown that it is possible to decode the
neural activity into distinct control commands, such that, in co-operation with the on-board
‘intelligence’ of a prosthetic hand, the hand flexion can be controlled.”
12
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
3 AI in Neuroprosthetics
3.1 Software Improvements
Most of the software in neuroprostheses is concerned with interpreting the brain signals into
binary and electronic information. Several different methods of interpretation have been
tested, like Artificial Neural Networks and fuzzy logic algorithms.
In one experiment power spectral density (PSD) values where extracted from mental tasks
using autoregressive methods and processed through a fuzzy neural network. The PSD values
were used to train the device to produce letters. The information was converted into a tri-state
morse code – dot, dash and space – which could then be translated into letters using morse
alphabet; this to improve computational time. This experiment used EEG-signals and was able
to conclude that this is a possible way for paralyzed humans to communicate given the low
error counts and fast computation times. [18]
“Prediction algorithms are at the heart of all of the closed loop BMI studies. The two broad
classes of prediction algorithms that are used are regression and classification. Regression
algorithms attempt to map inputs to a continuous space of output variables… Classification
algorithms, on the other hand, map inputs onto a set of discrete classes usually with no
implied ordering.” [19] These two classes are illustrated in Figure 3.
13
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
Figure 3. “Regression and classification approaches to BCI control of two-target and fourtarget applications. For the two-target application (targets are up and down triangles), both
approaches must determine the parameters of a single function. In contrast, for the four-target
application (targets are up and down closed and open triangles), the regression approach still
needs only a single function while the classification approach needs three functions, one for
each inter-target boundary.” [20]
Figure 3 illustrates the advantage of a regression approach in a BMI and for each new study
there is almost as many new approaches and algorithms. There have not been many closedloop neuroprosthesis studies but here follows a brief review of a recent one.
Four rats were surgically implanted with electrodes in the motor cortex. Using a support
vector machine (SVM) to decode neural activity, indicative activity was more easily
categorized and separated. The rats were trained to press levers for a food reward and
received predominantly visual feedback. Once the rats were sufficiently trained the levers
were no longer in use and the SVM would instead predict intended lever-pressing based on
neural activity preceding the motion. The SVM was able to predict the motion significantly
better than a comparative naïve Bayesian algorithm would. [19]
14
Artificiell Intelligens, HKGBB0
Henrik Månsson
Fördjupningsuppgift, HT –05
henma508@student.liu.se
Examinator: Arne Jönsson
The success of the experiment led to the following statement: “Our success with determining
directional control signal has inspired us to look further into the idea of a supervisory closedloop system directing a vehicle. We have begun to expand the system into actual vehicle
control and soon hope to allow for full velocity control using an asynchronous system
interacting with a highly sophisticated vehicle capable of sensing its surrounding and
interpreting the supervisory commands in light of such sensor readings”[19]. In effect we
would get rat-steered intelligent vehicles, research the likes of DARPA are likely to advance
through funding.
3.2 The Intelligent Hand [20]
Current hand prostheses usually employ myoelectric signals from arm muscles. They also
receive feedback visually alone. Ideally a neuroprosthesis would be connected directly to the
nerves and feeding them with electronic feedback for proprioception, touch, warmth etc. The
prosthesis in this study has sensors for force, joint angle and slip, which is to a microprocessor
in the hand. It is not sent back to the brain, however, but the microprocessor processes that
information on its own, which is both the clever and easy solution, as well as the natural order
of thinking: “the conscious part of the brain usually makes only strategic decisions about the
task, and the lower levels of the brain co-ordinate the joints in the fingers and arm to present
the hand in the best way to perform the operation”. Only difference is that here the lowerlevel brain functions are replaced by the artificial intelligence of the hand prosthesis and the
only feed needed is from motor cortex to prosthesis.
Considering that “the processing power devoted to control the hand, as measured by the
amount of motor cortex devoted to it, is as much as the legs and trunk combined” [21] a lot of
computational power has been rerouted from the brain to the prosthesis. The Oxford hand, as
it is also known as, has been tested and evaluated showing good results. The hand takes care
of grip strength in response to slip and quality of the object and also controls rotation at the
joint while the human user of the prosthesis can focus on only the opening of the hand. This
symbiosis is not complete though, as even smarter hands can be envisioned that would be able
to handle many more hand functions. Personally I imagine someone juggling.
3.2.1 The Intelligent Foot [22]
The PROPRIO FOOT is another more recent development. In October of 2006 NY Times
presented the “‘intelligent’ prosthesis that closely mimics the action of a human foot” [23].
Similar to the intelligent hand, the intelligent foot is fitted with AI capable of controlling
some lower-level functions of the prosthesis. The PROPRIO FOOT tracks its movement in
space, remembers how the walk is comparing the walk to stored profiles to identify what type
of gait is in action and what type of terrain is being trodden. When the foot is in walking
mode, the toe lifts when the foot is lifted for more ground clearance, as would a natural foot
do and similarly seats the toe faster when the foot comes down to the ground. The foot can
adjust to slopes and stairs and even has a mode of operation for standing up from sitting.
15
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
4 Summary
The research in the field of BMI is still scattered into many fractions directed at a whole range
of approaches of the best way to assist humans in need of sophisticated prostheses. The
following discussion aims at comparing some of the inconsistencies of approaches.
4.1 Discussion
One matter of discussion is the need for AI in BMIs where the essential parts of the nervous
system is intact and the prosthesis is only replacing the sensory neurons of the body.
Prostheses simulating neurons would interact with the human brain without any need for
additional computational power beyond that of the brain itself. The cochlear implant is a good
example for this discussion. This device is currently employing a computational algorithm for
extracting the most important input to forward to the brain out of all the auditory information
it receives. There are a few restrictive variables, like the fact that the cochlear implant uses
only 16 or 22 electrodes while the ear contains billions of nerve endings. Today this means
that a cochlear implant is not able to replicate complete auditory function, but with only a few
electrodes it is impressively efficient. This means that the possibility for more efficient
algorithms in combination with improved technical devices or new technology such as
nanotechnology could result in auditory prosthesis that would be superior to human ears, as
long as the brain is ready to adapt.
The need for AI in motor-controlled prostheses is already upon us. The intelligent foot
described earlier is the first of its kind and will most likely be followed by many more similar
prostheses. Right now it is impossible to obtain perfect interaction between a brain and a
neuroprosthesis and therefore it is essential that the prosthesis can operate autonomously to an
extent without input from the brain. This reduces the information flow needed in a closedloop device to the essential, basic commands and responses needed for walking, running etc.
A lot of muscle movements are involuntarily following directly on triggering movements,
such as lifting the toes while lifting the foot. If the foot automatically raises the toes when the
foot it lifted, then the brain only has to signal for the lifting of the foot and the foot takes care
of the rest. This reduction simplification is both a smart solution to the limitations of today’s
neuroprostheses as well as a potential for increasingly advanced foot functions that could go
beyond human’s foot functions.
The research on peripherally connected neuroprostheses in contrast with BMIs connected to
the brain itself generates the question of which is better. Two approaches are available. Either
the prosthesis is connected to the stump of the amputee, or to the brain. If the device is not
connected to the brain, then it can be myoelectrically controlled, or receiving input from cuff
electrodes, but preferably it would be connected to the nerve endings at the end of the stump
as this would be the most practical place for surgery and wouldn’t involve any artificial
devices on the body except for where the prosthesis follows. If instead connected to the brain,
the prosthesis would still be physically connected to the stump, but the device would utilize a
wireless connection to an implant connected directly to the motor cortex. There’s a lot of
scientific research demonstrating the capabilities of BMI connected to the cortex but a
peripheral neuroprosthesis seems the more natural alternative, natural not only to the
physiological functionality but also to the already existing brain and nervous system structure.
16
Artificiell Intelligens, HKGBB0
Henrik Månsson
Fördjupningsuppgift, HT –05
henma508@student.liu.se
Examinator: Arne Jönsson
One last note in the discussion is that neuroprostheses have the potential of relieving some
computational power from the brain to the prosthesis. A lot of brainpower and a lot of the
physical brain are devoted to computing e.g. movement, which a sufficiently intelligent
device could do autonomously instead. It also offers the possibility of prosthetic system
capable of far more than what is naturally given to humans; superhuman strength and senses
for example. The philosophical debate on the implications of this is left for others to indulge
in.
4.2 Summary
This paper concludes that there is a great deal of progress to be expected in the following
years concerning BMIs and in particular the advancement of AI in neuroprosthesis. In the
field of BMI new results mean better-suited biocompatible materials and more functional as
well as user-friendlier hardware. In addition to this, better algorithms for signal decoding and
processing is expected accompanied by improvements in closed-loop feeds, where the
ultimate goal of totally replaceable neuroprostheses still lies far off. Lastly, developments in
AI and compatibility between artificial and human intelligence in brain-machine interfacing
will be a challenge for the coming of generations in neuroprosthetics. Although science is
closing in, sci-fi is still ahead by a number of years.
17
Artificiell Intelligens, HKGBB0
Fördjupningsuppgift, HT –05
Examinator: Arne Jönsson
Henrik Månsson
henma508@student.liu.se
5 Bibliography
[1] http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=535572
[2] http://news.bbc.co.uk/2/hi/science/nature/3096973.stm
[3] http://news.bbc.co.uk/1/hi/sci/tech/1043001.stm
[4] http://www.napa.ufl.edu/2004news/braindish.htm
[5] http://www.popsci.com/popsci/medicine/59b5c4522fa84010vgnvcm1000004eecbccdrcrd
.html
[6] http://news.bbc.co.uk/1/hi/sci/tech/1025471.stm
[7] http://student.biology.arizona.edu/honors2004/group22/results.html
[8] http://www.sciencedaily.com/releases/2005/05/050511073108.htm
[9] http://personal.bgsu.edu/~nberg/chilabs/bmi.htm
[10] http://www.ric.org/bionic/
[11] http://www.usatoday.com/money/industries/health/2004-10-10-braingate-cover_x.htm
[12] http://news.bbc.co.uk/2/hi/science/nature/1871803.stm
[13] http://www.bionicear.org/
[14] Sit, Simonson, Oxenham, Faltys, Sarpeshkar.”A Low-Power Asynchrounous Interleaved
Sampling Algorithm for Cochlear Implants That Encodes Envelope and Phase Information.”
IEEE Transsactions on Biomedical Engineering, vol 54 no.1, January 2007.
[15] http://www.usc.edu/uscnews/stories/8903.html
[16] http://www.usc.edu/uscnews/stories/8446.html
[17] Warwick, Kyberd, Goodhew, Hutt, Gasson. “Invasive neural prosthesis for signal
detection and nerve stimulation”. International Journal of Adaptive Control and Signal
Processing 2005; 19:365-375.
[18] Palaniappan, Paramesran, Nishida, Saiwaki. ”A New Brain-Computer Interface Design
Using Fuzzy ARTMAP”. IEEE Transactions on Neural Systems and Rehabilitation
Engineering, vol 10 no.3 September 2002.
[19] Olson, Si, Hu, He. ”Closed-Loop Cortical Control of Direction Using Support Vector
Machines. “IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol 13
no.1 March 2005.
[20] McFarland, Wolpaw. “Sensorimotor Rhythm-Based Brain-Computer Interface (BCI):
Feature Selection by Regression Improves Performance”. IEEE Transactions on Neural
Systems and Rehabilitation Engineering, vol 13 no.3 September 2005.
[20] Kyberd. “The intelligent hand”. IEE Review September 2000.
[21] Kyberd, Pons. “A Comparison of the Oxford and Manus Intelligent Hand prostheses”.
Robotics and Automation vol 3 September 2003.
[22]
http://www.ossur.com/shared/FileGallery/SharedFiles/Ossurcom/Bionics/Brohures/PROPRIO
_FOOT_Sept2006.pdf
[23]
http://www.nytimes.com/packages/html/science/20061003_FOOT_GRAPHIC/index.html
Additional resources:
Lebedev, Nicolelis. ”Brain-machine interfaces: past, present and future”. TRENDS in
Neurosciences vol 29 no.9 2006.
Spelman FA. “The past, present, and future of cochlear prostheses”. IEEE Engineering in
Medicine and Biology Magazine 1999; 18:27–33.
http://www.sciencedaily.com/releases/2004/10/041027143313.htm
http://www.ninds.nih.gov/disorders/deep_brain_stimulation/deep_brain_stimulation.htm
http://www.guardian.co.uk/life/feature/story/0,13026,1501763,00.html
18