B1 #6254 Disclaimer — This paper partially fulfills a writing requirement for first year (freshman) engineering students at the University of Pittsburgh Swanson School of Engineering. This paper is a student, not a professional, paper. This paper is based on publicly available information and may not be provide complete analyses of all relevant data. If this paper is used for any purpose other than these authors’ partial fulfillment of a writing requirement for first year (freshman) engineering students at the University of Pittsburgh Swanson School of Engineering, the user does so at his or her own risk. THE SIGNIFICANCE OF THE NEUROLOGICALLY CONTROLLED ARM Tim Bowers (teb50@pitt.edu) Jeremiah Hoydich (jsh70@pitt.edu) Abstract- Prosthetics have been around for centuries. Whenever we think of prosthetics we typically think about plastic molds resembling limbs that do not have many functions. Scientists now are working to change that. Prosthetics are now being designed and tested that use the power of the brain to bring the functions of the arm back to amputees and quadriplegics [1]. These neurologically controlled arm prosthetics will be a critical part of the future of prosthetics. Neurological prosthetics use electrodes along the natural pathways of the brain to control simple motions of a mechanical arm [2]. As the technology improves, so does the intricacy of the movements the arms are able to make. Currently, the only drawbacks with this technology are the relatively high cost and inability for patients to get the technology through health insurance and Medicare [3]. Key Words- Electrode Technology, Motor Cortex, Neuroprosthetic, Posterior Parietal Cortex, Prosthetics, Quadriplegic. THE IMPORTANCE OF THE NEUROPROSTHETIC ARM Each year in the United States an estimated 185,000 people have one limb amputated [4]. The number of amputees in the United States is projected to be 3.6 million by 2020 [4].The number of people who do not have function of their limbs only increases when you take into account the number of people with tetraplegia and other forms of degenerative spinal cord diseases. For these people, normal everyday life will likely never be a possibility again. This is especially true for people who have had arm amputations or loss of arm movement. The prosthetics provided today for arm amputees can only do the basic functions of grasping and arm movement. And people with degenerative diseases cannot even use these rudimentary prosthetics. Both of these people could be helped with the advent of the neurologically controlled prosthetic arm. This prosthetic would allow the user to control a fully functioning mechanical arm and hand the same way they controlled their old limbs: with their brain. They large scale introduction of these prosthetics could bring back the old everyday lives to these patients. HISTORY OF PROSTHETICS The history of the arm prosthetic starts in the historical writing of Pliny the Elder [5]. Pliny writes about a Roman general who had his arm amputated during the Second Punic War [5]. To keep fighting, the general had a metal arm cast with which he could hold his shield [5]. The next advancement happened during the Dark Ages when the hook was added to the prosthetic arm to give it some functionality in being able to pick up objects [5]. The first functioning hand prosthetic was crafted 1508 for a German mercenary [5]. This prosthetic could be made to grasp objects by manipulating it with your freehand [5]. Since then prosthetics have been better molded to the patient’s arm and have a better hook mechanism but these changes have not had a huge change in the area of arm prosthetics. The neuroprosthetic arm hopes to change this. The neuroprosthetic arm has been a long time imagined advancement in prosthetics. As most modern technology, the premonitions of this prosthetic started in science fiction. This prosthetic was popularized by Star Wars. After Luke Skywalker gets his hand cut off by his father Darth Vader, he gets a fully mechanical prosthetic arm and hand which are controlled with his mind. The actual science behind the neuroprosthetic arm was not figured out until 2002 when Dr. Kevin Warwick had electrodes implanted in his brain which allowed a neuroprosthetic arm to mimic the movements his natural arm was making [6]. From this point many companies and colleges, including the University of Pittsburgh, have joined in the process of creating the ideal neuroprosthetic arm. There have been many advancements. Here are two of the more intriguing advancements researchers have made for this type of prosthetic. First, a patient at Case Western University was successfully able to use two neuroprosthetic arms at the same time [4]. This is a huge advancement, as these people would be as close as possible to life before their amputation or illness. The other notable advance also takes place at Case Western. In this study, a man who lost a hand was able to feel when a cotton-ball was dragged across a neuroprosthetic hand. This advance is incredibly interesting as not only can the University of Pittsburgh, Swanson School of Engineering 2016-02-12 Timothy Bowers, Jeremiah Hoydich device decipher the electrical activity in a patient’s brain, but it can also relay information to the brain to trigger a biological response. These advancements were over a fourteen year time span. For science, this is a relatively short time which makes you wonder what heights this technology can reach if the advancements keep up at this speed. representation than the trunk or legs because the muscle patterns are relatively simple [7]. Secondary Motor Cortices The secondary motor cortices are involved in motor planning. These cortices are consisted of three parts: the posterior parietal cortex, the premotor cortex, and the supplementary motor area (SMA). The posterior parietal cortex is involved in transforming visual information into motor commands. It sends information to the premotor cortex and supplementary motor area. For example, the posterior parietal cortex is involved in determining how to steer the arm to a glass of water based on where the glass of water is located in space. The premotor cortex lies just in front of or anterior to the primary cortex. It helps to guide body movements by integrating sensory information. It controls the muscles that are closest to the body’s main axis. For example, the premotor cortex would help orient the body before reaching for the glass of water [7]. The supplementary motor area (SMA) lies above or medial to the premotor cortex and in front of or anterior to the primary cortex. The SMA is involved in planning complex movements and in coordinating movements involving both hands. Both the SMA and premotor cortex send information to the primary motor cortex as well as to brainstem motor regions [7]. THE ANATOMY OF MOVEMENT Cerebellum The cerebellum is a small grooved structure located in the back of the brain beneath the occipital lobe. It is involved in the timing and coordination of motor programs. Motor programs are generated in the basal ganglia. The basal ganglia is several subcortical regions that are involved in organizing motor programs for complex movements, such as learning to play a new sport or instrument. The basal output sends an output to other subcortical brain regions and to the motor cortex [7]. MOTOR CORTEX The motor cortex controls our body’s voluntary movements. It must receive various kinds of information from the various lobes of the brain. From the parietal lobe, it receives information about our body’s spatial awareness. The information on the goal at hand and appropriate strategies for obtaining it come from the anterior frontal lobe. The temporal lobe sends memories of past strategies to the motor cortex to perform the task at hand. The motor cortex then generates neural impulses that control the execution of movement. These signals cross the midline to activate skeletal muscles on the opposite side of the body. This process allows the right hemisphere to control the left side of the body and the left hemisphere to control the right side [8]. The motor cortex lies along the precentral gyrus in the rear portion of the frontal lobe, just before the central sulcus, or furrow. The central sulcus (furrow) separates the frontal lobe from the parietal lobe. The motor cortex is divided into two areas: the primary motor cortex and secondary motor cortices [8]. Corticospinal Tract The corticospinal tract is the main pathway for control of voluntary movement in humans. This is the only direct pathway from the cortex to the spine. Neurons in the brain give rise to over a million fibers in this tract. These fibers descend through the brainstem and cross over to the opposite side of the body. The fibers continue to descend through the spine until ultimately terminating at the appropriate spinal levels [7]. Cortical Control of Skeletal Muscles Signals generated in the primary cortex travel down the corticospinal tract, through the spinal white matter to synapse on interneurons and motor neurons in the spinal cord’s ventral horn. The ventral horn sends its axons out through the ventral routes to innervate individual muscle fibers. The ventral horn neuron, its axon, and myofibrils that it innervates are considered to be all a single motor unit. Motor neurons can innervate any number of muscle fibers, but each fiber is only innervated by one motor neuron. When the motor neuron fires, all of its innervated muscle fibers contract. The size of the motor units and the number of fibers that are innervated contribute to the force of the muscle contraction [7]. There are two types of motor neurons in the spine: alpha and gamma motor neurons. Alpha motor neurons innervate muscle fibers that contribute to force production of the muscle. Primary Motor Cortex The primary motor cortex forms a thin band along the central sulcus (furrow). Every part of the body is represented in the primary cortex in what is known as the motor homunculus. The amount of brain matter devoted to any particular body part represents the amount of control that the primary motor cortex has over that body part. For example, a lot of cortical space is required to control complex movements of the hands and fingers. These body parts have a larger 2 Timothy Bowers, Jeremiah Hoydich Gamma motor neurons measure the length, or stretch, of the muscle [7]. This complex process allows us humans to perform tasks effortlessly and learn new tasks everyday. Neurofeedback is the decoded motor intentions of the patient. These intentions are used in real-time to control an assistive device. A decoding algorithm is calibrated using neural signals collected during performance of an instructed set of real or imagined arm movements. The parameters of movement are tuned such that the output of the algorithm best predicts the direction of the training movements [2]. When the output of the algorithm during this phase does not influence the user’s behavior, it is known as one-loop decoding [2]. The decoding algorithm may be used for realtime brain control, during which the user receives some form of feedback from the device. This is known as closed-loop decoding. Proprioceptive information, information relating to stimuli that are produced and perceived within an organism, can improve the accuracy of the BMI-meditated movement. Once the brain is “in the loop” and connected to the decoding algorithm, patterns of neural activity change as the user masters the interface. Individuals learn voluntary control of a feedback signal that relays real-time information about specific brain activity. Practice using the BMI is accompanied by profound reorganization of cortical activity, such as changes in the directional tuning of neurons used by the decoding algorithm and reduction of the modulation of neighboring neurons [2]. Improvements to the neurofeedback could involve residual sensory models, such as vision or artificial sensory pathways provided by electrical stimulation to the brain. By incorporating the senses into the interface will allow the user to have not only the functionality to think to perform a task, but also the awareness to sense their surrounds to perform it more easily. Repetitive stimulation might induce long-term changes that increase the excitability of spinal circuitry and enhance the efficacy with which movements can be evoked [2]. INTERFACE WITH THE BRAIN Research has seen rapid progress in two examples of neuroprosthetics for spinal cord injuries (SPI). These examples are brain-machine interfaces (BMI) and functional electrical stimulation (FES). A BMI enables a patient to control assistive devices, such as robotic limbs, by using neural signals recorded directly from the brain. FES is used in the attempt to reanimate paralyzed limbs. [2] Brain-Machine Interfaces Implanted neurostimulators may find applications in modulating the excitability of spinal networks and guiding the activity-dependent processes that govern the formation of new motor circuits. The BMI record and decode signals from the brain enabling voluntary control of assistive devices. It modifies patterns of cortical activity through the process of neurofeedback [2]. Invasive techniques of direct brain stimulation have been used. BrainGate, a brain control system that uses an array of 96 silicon electrodes that penetrate 1.5mm into the upper-limb representation of the motor cortex to record firing from 50 or more neurons. Spiking rates of these neurons are then processed to provide control signals for various artificial effectors [2]. Noninvasive recording techniques, such as an electroencephalogram (EEG), are an alternative method to obtain signals for neural interfacing. An EEG is a test that detects electrical activity in your brain using small, flat metal discs (electrodes) attached to one’s scalp. One’s brain cells communicate via electrical impulses and are active all the time, even when one is asleep [2]. Unfortunately, as any imagined movement of a given limb produces the same general pattern, desynchronization over a wide area of cortex, interferes with the specificity of what signals are going toward the movement of which limb. The independent control of movements in multiple dimensions requires the patient to learn nonintuitive combinations of lefthand, right-hand, and foot movements. The EEG is generally poor at capturing the high-frequency bands. This is possibly due to the spatiotemporal filtering that is inherent in scalp recordings. Scalp recordings use spatiotemporal filtering to determine the active areas in the brain when given a task to perform. Also, low frequency EEG signals may sometimes be confounded by eye movements that cause occipital lobe activity. This activity can interfere with the intention of recording motor signals [2]. MECHANICS OF THE PROSTHESIS Robotic arm prosthetics come in many different shapes and sizes. Often a design team will sacrifice design appeal to achieve greater function. The relative importance of the appearance as opposed to the functionality is dependent on the patient. The components of a full arm prosthetic are the wrist, forearm, upper-arm, elbow, and shoulder (scapular). The most common actuator for electrically powered prosthesis is the permanent magnetic dc electric motor with some form of transmission [9]. The prosthesis can experience motion in either two or three dimensions dependent upon the parameters set. Two dimensional motion allows the prosthesis to only move on one plane, whether it be vertical or horizontal. Three dimensional motion of a full arm prosthetic includes: wrist flexionextension, wrist rotation, wrist abduction-adduction, upperarm flexion-extension, upper-arm rotation, upper-arm Neurofeedback 3 Timothy Bowers, Jeremiah Hoydich abduction-adduction, elbow flexion-extension, shoulder abduction-adduction, and shoulder elevation-depression [9]. An experiment done by the Department of Physiology and Pharmacology at State University of New York Downstate Medical Center in Brooklyn, New York involved the interconnection of a 3-layered cortex, composed of several hundred spiking model-neurons, which display physiologically realistic dynamics, to a two-joint musculoskeletal model of a human arm. The model of the human arm is composed of realistic anatomical and biomechanical properties. The virtual arm received muscle excitations from the neuronal model and fed back proprioceptive information, forming a closed-loop system. The virtual arm muscle activations responded to motor neuron firing rates, with virtual arm muscle lengths encoded via population coding in the proprioceptive population. The researchers calibrated the robotic arm to reproduce the same trajectories in real time and compared the dimensionless jerk measures between the musculoskeletal arm and a simple arm design [10]. The virtual arm includes rigid bodies (bones), joints, muscles, and tendons. The kinematics are governed by a set of ordinary differential equations (ODEs) that compute muscle activation, length, force, as well as arm motions and forces at millisecond resolution. The cortical model was interfaced with the virtual arm by exciting the arm muscles using a spiking output from motor neuron output. The proprioceptive information from the muscle lengths provided activation for a proprioceptive neural population. Arm joint angles were also fed back to the biomimetic model and was used to calculate the error signal during the reinforcement learning-based training phase [10]. The kinematics of each joint and the force-generating parameters for each muscle in the system in a biomechanical model of the upper extremity musculoskeletal system have been derived by anatomical and psychological studies represented an average size adult human male. The model captured the primary features of the upper extremity geometry and mechanics. This includes the complex joint coupling effects, where the mechanics of a given joint depends on the posture of the adjacent joints. The model included the following rigid bodies where the muscles are anchored: ground, thorax, clavicle, scapula, humerus, ulna, radius, and hand [10]. This experiment only allowed for two degrees of motion, which was shoulder and elbow joint rotation in the horizontal plane. The major active muscles in shoulder and elbow motion are: the posterior deltoid, infraspinatus, lattisimus dorsi, and teres minor (shoulder exterior muscles); anterior deltoid, pectoralis major, and corachobrachialis (shoulder flexor muscles); triceps (elbow extensor muscles); biceps and brachialis (elbow flexor muscles). Other muscles were set for joint stability, these included: lateral deltoid, anconeous, brachioradialis, extensor carpi radialis longus, and pronator teres. Muscles with multiple heads had the muscle branches connected to different insertion and origin points but were controlled by the same input signal for simplicity. As a result of the experiment, the mean dimensionless jerk measure was significantly lower for the realistic arm as compared to the simple arm, both in trained and naïve networks. This suggests that the musculoskeletal arm for any biologically reasonable input generates smoother movements than the simple arm [10]. This experiment demonstrated that increasing the realism of the arm model reduces the arm trajectory jerk and results in velocity profiles closer to biology, which reflect into smoother robot movements. CASE STUDIES There have been a number of real world tests of the neurologically controlled arm. Here are some of the finding that these different researchers have found. The first study we will talk about was conducted by the Applied Physics Laboratory of John Hopkins University. In this study a 52 year old women with tetraplegia is tested through various tasks using an anthropomorphic prosthetic limb [3]. To start the study, the researchers placed two intracortical microelectrode arrays into the left motor cortex of the patient [3]. Over the course of three weeks the patient was trained in the use of the arm, first in the general movement of the arm and then the use of the fingers and joints. After this training, the patient came back and did nine tests out of 19 possible tests a day for 98 days [3]. The tests included activities such as stacking cones and moving block a certain distance. Each test was timed. The patient was graded on completion and the time it took to complete the task. Over the course of the test the results of the patients steadily improved. This case study illustrates how these prosthetics can perform tasks for people who normally would never be able to do them. This could be a huge step in overcoming their disabilities. Another case study has been performed by the University of Pittsburgh in which monkeys have been trained to control a virtual arm to grab a point in space on a computer with their mind [11]. Although this study is not as far along as the previous study, the promising point here is that even monkeys have the ability to operate the device. The University of Pittsburgh expanded on their work with neuroprosthetics when they tested a neuroprosthetic arm on a 52 year old female patient with tetraplegia [12]. To get the prosthetic to work, researchers examined the patient’s brain pattern as they asked her to move her arm and hand in a certain way [12]. They then programed the prosthetic to make those motions and handshapes as it sensed the patient making them. For calibration, the researchers hooked the patient into a computer and asked her to do the same tasks in virtual reality. This study went further than most other studies in the number of ways the hand could be manipulated [12]. Previous studies only had six hand formations to choose from, this study tried increasing this number to ten [12]. The patient was 4 Timothy Bowers, Jeremiah Hoydich successfully able to manipulate all the additional handshapes [12]. Although these advancements in the number of handshapes is a good step forward the functionality of the prosthetic suffered. The ability to hold and move objects was inconsistent through the first round of tests [12]. As the researchers improved the process of calibration the success rate of object manipulation rose [12]. The success rates of this study were surprisingly good. Success rate of tasks by the patient were 70% [12]. Overall, this study illustrates the bright future of the neuroprosthetic arm. This study was able to mimic more hand motions than before which means that these prosthetics are on the path to becoming as fully functional as a regular arm. These case studies show how neurological devices are not just a science fiction fantasy anymore. They are science fact. Fact that deserves further funding and research because of the help that it can bring to potentially millions of people. FROM IMAGINATION TO REALITY Neurologically controlled prosthetics is a stepping- stone into the future of rehabilitation science. It gives people the chance to perform simple tasks that once seemed impossible to them. The history of prosthetics shows the drastic progression of a field that’s sole purpose is to improve the lives of those in need. The history of arm prostheses identifies the increasing development and potential this technology is moving towards to help people. The prosthetics improvement over time has garnered its way to an astounding feat. The brain’s motor cortex and secondary cortices have given people the ability to do numerous tasks in their everyday lives from the simple task of picking up a glass of water to the complex tasks of playing an instrument. The use of the brain to control a machine has been thought of and imagined throughout history and the technology to do so is astounding. The neurologically controlled prosthetic arm has allowed an entirely new group of people to experience a form of independence after a part of their independence has been taken from them. By tapping into the brain’s motion capabilities of the human body people who have lost the control of movement in their arms are now able to provide movement to artificial arms to perform their everyday tasks. A part of their independence is finally being able to be given back to them. The case studies show that this new technology is paving a way to a new future for those who are trying to regain the independence in their everyday lives. Along with this new technology, ethical concerns arise. The positives seem to far outweigh the negatives as the main purpose is to benefit the lives of those in need. The idea of neurologically controlled devices is stepping out of what was only thought possible in the imagination into the reality of our everyday lives. ETHICS There are currently three ethical dilemmas facing the future of the neuroprosthetic arm currently. These three dilemmas are the relatively high cost of the prosthetic, the cybersecurity risk, and the invasiveness of the procedure. Currently, Medicare will only cover advanced prosthetics for leg amputees who they believe have a high level of mobility [4]. And if a patient would want a neuroprosthetic arm that is fully functional the cost for the prosthetic alone can be upwards of $80,000 [4]. To combat the high cost of these prosthetics an advocacy group named the Amputee Coalition has been lobbying for Medicare to cover a broader range of patients. Cybersecurity has also been a growing risk in the health field in general. With increasing computer reliance the threat of hacking medical devices has also risen. Neuroprosthetics are no exception. Their reliance on computers makes them an easy target for hackers [13]. These prosthetics likely don’t have any countermeasures to these hacks as they are not often expected. This can leave the user in a dangerous situation as hacking the prosthetic and disabling it during a dangerous task could be potentially deadly. The last ethical dilemma is the invasiveness of the procedure. For the neuroprosthetic to work, electrodes need to be placed within the patient’s brain. For amputees and patients suffering from tetraplegia this is not a problem, but for stroke victims this can be an issue. Since some stroke victims can regain the use of their arms through physical therapy the invasiveness of the procedure could potentially harm their recovery because of the location of the electrodes on the areas of the brain which control limb movement [4]. There have been improvements to get rid of the invasiveness of the procedure. Researchers at the University of Houston have developed a cap that reads brain activity the same way the invasive procedure does and negates the need for the electrodes to be planted within the brain [4]. REFERENCES [1] T. Yanagisawa. (2011). “Electrocorticographic control of a prosthetic arm in paralyzed patients.” Annals of Neurology. (Online article). http://onlinelibrary.wiley.com/doi/10.1002/ana.22613/ful. [2] A. Jackson. (2012). “Neural interfaces for the brain and spinal cord-restoring motor function.” Nature Reviews Neurology 8. (Online article). http://www.nature.com/nrneurol/journal/v8/n12/full/nrneurol. 2012.219.html. [3] J. Collinger (2013). “High-performance neuroprosthetic control by an individual with tetraplegia.” The Lancet. (Online article). http://www.sciencedirect.com/science/article/pii/S014067361 269. 5 Timothy Bowers, Jeremiah Hoydich [4] J. Jacob. (2015). “Advance Prosthetics Provide More Functional Limbs”. The Journal of the American Medical Association. (Online journal). http://jama.jamanetwork.com/article.aspx?articleID=2319161 [5] K. Norton. (2007). “A Brief History of Prosthetics”. Amputee Coalition. (Online article). http://www.amputeecoalition.org/resources/a-brief-history-of-prosthetics/. [6] K. Warwick. (2003). “The Application of Implant Technology for Cybernetic Systems”. The Journal of the American Medical Association. (Online journal). http://archneur.jamanetwork.com/article.aspx?articleid=7847 43. [7] S. Schwerin. (2013). “The Anatomy of Movement” Brain Connection. (Online article.). http://brainconnection.brainhq.com/2013/03/05/the-anatomyof-movement/ [8] “The Motor Cortex” THE BRAIN FROM TOP TO BOTTOM. (Website.) http://thebrain.mcgill.ca/flash/i/i_06/i_06_cr/i_06_cr_mou/i_ 06_cr_mou.html [9] R. F. ff. Weir. 2004. Standard Handbook of Biomedical Engineering and Design. McGraw-Hill; [2004; 2016]. [10] S. Dura-Bernal, X Zhou, S. A. Neymotin, A. Przekwas, J. T. Francis, W. W. Lytton (2015). “Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm” Frontiers in Neurobotics. (Online article.). http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4658435/ [11] M. Velliste. (2014). “Motor Cortical Correlates of Arm Resting in the Context of a Reaching Task and Implications for Prosthetic Control”. The Journal of Neuroscience. (Online journal). http://motorlab.neurobio.pitt.edu/pub/MotorCorticalCorrelate sofArmResting.pdf. [12] B. Wodlinger. (2015). “Ten-dimensional anthropomorphic arm control in a human brain-machine interface: difficulties, solutions, and limitations”. Journal of Neural Engineering. (Online journal). http://iopscience.iop.org/article/10.1088/17412560/12/1/016011/meta;jsessionid=C5E7D3D74BF5B3C923 45D3F4A3012A6E.c4.iopscience.cld.iop.org. [13] J. Hsu. (2014). “Feds Probe Cybersecurity Dangers in Medical Devices”. IEEE Spectrum. (Online article). http://spectrum.ieee.org/tech-talk/biomedical/devices/fedsprobe-cybersecurity-dangers-in-medical-devices. ACKNOWLEDGEMENTS We would like to thank our co-chair, Ms. Jesse Liu, for her help in guiding us in this assignment. Our writing instructor, Professor Janet Zellman, for providing us with critique on our writing. And the University of Pittsburgh librarians who provide us with many avenues to do research. 6