Improving the Accuracy and Safety of a Robotic System
advertisement
Agysebészeti robot pontosságának és biztonsági funkcióinak fejlesztése Improving the Accuracy and Safety of a Robotic System for Neurosurgery Haidegger, Tamás 2008 Agysebészeti robot pontosságának és biztonsági funkcióinak fejlesztése Improving the Accuracy and Safety of a Robotic System for Neurosurgery Diplomaterv - Diploma Thesis Budapesti Műszaki és Gazdaságtudományi Egyetem – Irányítástechnika és Informatika Tanszék Budapest University of Technology and Economics – Department of Control Engineering and Information Technology Johns Hopkins University – Center for Computer-Integrated Surgical Systems and Technology Haidegger, Tamás Egészségügyimérnöki szak - MS Candidate in Biomedical Engineering haidegger@eestec.hu Konzulensek - Supervisors: Peter Kazanzides Dr. JHU – ERC CISST Benyó, Zoltán Dr. BME - Irányítástechnika és Informatika Tanszék Baltimore, May 27, 2008 SEMMELWEIS EGYETEM BUDAPESTI MŰSZAKI ÉS GAZDASÁGTUDOMÁNYI EGYETEM SZENT ISTVÁN EGYETEM EGÉSZSÉGÜGYI MÉRNÖKKÉPZÉS DIPLOMATERV FELADAT Haidegger Tamás részére Diplomaterv címe: Agysebészeti robot pontosságának és biztonsági funkcióinak fejlesztése A kidolgozandó feladat: A modern egészségügy és orvoslás egyre nagyobb mértékben támaszkodik a technikai eszközök nyújtotta segítségre. A számítógépek és a robotok bevezetése a műtőkbe (Computer-Integrated Surgery) minőségi változást hozott máris a betegellátásban. Az agysebészet egy kiemelt terület, mivel az anatómiai tájék sérülékenysége miatt kifejezetten nagy pontosságú beavatkozást lehetővé tevő eszközök alkalmazása szükséges. A Johns Hopkins University-n üzembe helyezett kísérleti összeállítás lehetővé teszi koponyaalapi sebészeti beavatkozások végrehajtását egy átalakított NeuroMate agysebészeti robot segítségével. A robot végére erősített 6 DOF erő/nyomatékérzékelő és fúrófej megfelelő szabályozással egy nagy pontosságú, nagy megbízhatóságú eszközként szolgál. Nagyon fontos a megfelelő biztonsági funkciók megvalósítása. Igen kritikus a műtétek alatt a robot és a fúró optikai marker alapú követése, a mozgások összehasonlítása az eredeti parancsokkal. A jelöltnek be kell kapcsolódnia a Johns Hopkins ERC központjában folyó kutatásba, együtt kell működnie a fejlesztőcsapattal, és önállóan dolgoznia a robotrendszer biztonságának és pontosságának fejlesztésén. Elvégzendő feladatok: 1. Tanulmányozzon és alkalmazzon különféle regisztrációs algoritmusokat a NeuroMate robot pontosságának növelésére. Tervezzen meg egy megfelelő validációs metódust a komplett agysebészeti rendszer regisztrációjához! 2. Tervezzen meg és implementáljon egy új módszert a műtéti elrendezés intra-operatív mozgáskövetésére, az esetleges nem-akaratlagos elmozdulások kiküszöbölésére. Az algoritmusokat a CISST keretrendszeren belül kell megvalósítani C++ nyelven! 3. Végezzen méréseket in-vitro az új algoritmus eredményességének bemutatására. Mutassa meg a módszer klinikai jelentőségét! Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery A diplomatervet kiadó tanszék: Irányítástechnika és Informatika Tanszék Tanszéki konzulens: Dr. Benyó Zoltán, egyetemi tanár Külső konzulens: Dr. Peter Kazanzides, Associate Research Professor (Johns Hopkins University - Center for Computer-Integrated Surgical Systems and Technology) Záróvizsga-tárgyak: 1. Folyamatszabályozás - VIFO2FSA 2. Klinikai műszeres diagnosztika - BMEVIOB2KMD Beadási határidő: 2008. május 30. Budapest, 2008. január 4. ........................................ ....................................... Dr. Szirmay-Kalos László Dr. Benyó Zoltán Tanszékvezető Szakbizottság képviselője Gesztor Kar: BME-Villamosmérnöki és Informatikai Kar Cím: 1111 Budapest Egry József utca 18. -5- Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery SEMMELWEIS BUDAPESTI UNIVERSITY OF UNIVERSITY TECHNOLOGY AND ECONOMICS SZENT ISTVÁN UNIVERSITY Graduate School of Biomedical Engineering DIPLOMA PROJECT for TAMÁS HAIDEGGER Diploma title: Improving the accuracy and safety of a robotic system for neurosurgery Description: Modern technology is more and more intensively used in medicine and healthcare. The introduction of robotics and automation to the operating room (ComputerIntegrated Surgery) provides better patient care and new, advanced surgical solutions. A prominent field is the area of brain surgery that requires extremely accurate positioning and high precision interventions due to the vulnerability of the anatomical region. The innovative setup at the Johns Hopkins University uses a modified NeuroMate stereotactic robot in cooperative control mode to provide a high-fidelity cutting tool for skull base surgery. It is crucial to introduce advanced safety features to ensure the reliability of the system. Most important is the real time optical tracking of the robot and the tool during the surgery. The candidate of the diploma project has to join the ongoing research project at JHU, get involved with the development and improve the overall safety and accuracy of the neurosurgical robot. Tasks to be solved: 1. Examine and study different registration algorithms to improve the accuracy of the NeuroMate robot. Build up an adequate validation concept for the registration of the neurosurgical system. 2. Design and implement a method for intra-operative tracking that can be used to monitor and compensate any spatial changes within the OR. The algorithms should be embedded to the CISST software framework and libraries using C++ language. 3. Perform simulations, in-vitro and cadaver tests to validate the new algorithm. Show clinical relevance of the research. Prepare the system for clinical trials. -6- Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Supervisors: Dr. Zoltán Benyó, Professor (BME - Dept. of Control Engineering and Information Technology) Dr. Peter Kazanzides, Associate Research Professor (Johns Hopkins University - Center for Computer-Integrated Surgical Systems and Technology) Final exams: 1. Folyamatszabályozás - VIFO2FSA (Process control) 2. Klinikai műszeres diagnosztika - VIOB2KMD (Clinical equipment and diagnostics ) Deadline: May 30, 2008 Budapest, January 5, 2008 ..................................... ...................................... Szirmay-Kalos, Laszlo Dr. Benyo, Zoltan Dr. Head of Dept. Repr. of Edu. Committee Gestor Faculty: BME-Faculty of Electrical Engineering and Informatics Address: 18. Egry József utca, Budapest, H - 1111 -7- Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Nyilatkozat Alulírott Haidegger Tamás, a Budapesti Műszaki és Gazdaságtudományi Egyetem hallgatója kijelentem, hogy ezt a diplomatervet meg nem engedett segítség nélkül, saját magam készítettem, és a diplomatervben csak a megadott forrásokat használtam fel. Minden olyan részt, melyet szó szerint, vagy azonos értelemben, de átfogalmazva más forrásból átvettem, egyértelműen a forrás megadásával megjelöltem. Declaration I, undersigned Tamás Haidegger, student at the Budapest University of Technology and Economics hereby state that this Diploma is my own work wherein I only used the sources listed in the Bibliography. All parts taken from other works, either in citation or rewritten keeping the original contents, were unambiguously marked by a reference to the source. …..….…………… Haidegger, Tamás Baltimore, MD, May 27, 2008 -8- Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Kivonat Az elmúlt húsz évben a robotizált eszközök a gyártásautomatizálás mellett az egészségügyben is egyre komolyabb szerepet kaptak. Az agy- és idegsebészet volt a legelső alkalmazási területük, és mára már több tucatnyi rendszert fejlesztettek a világ számos országában, amelyek különböző módokon igyekeznek megfelelni a szigorú biztonsági és pontossági követelményeknek. Az amerikai Johns Hopkins University kutatólaboratóriumában egy új, agyalapi sebészeti beavatkozások hatékony támogatására szolgáló orvosi robot-rendszer fejlesztésében vettem részt. A kutatás célja, hogy egy NeuroMate robot és StealthStation navigációs rendszer összekapcsolásával pontosabbá tegyük a koponyafúrással járó beavatkozásokat, csökkentsük a kockázatot, valamint a műtéti időt. A robot erő/nyomaték irányítás révén folyamatosan követi a sebész kezének mozgását a fúró és a robot közé illesztett érzékelő segítségével. Az integrált eszköznek három kiemelkedő előnye van. Először is kiváló műtéti vizualizációt tesz lehetővé, képes megjeleníteni a sebészeti eszközt a beteg 3D pre-operatív CT felvételekből készített modelljén. Ezen túlmenően, mivel a fúrófej a robothoz van rögzítve, az egész szerkezet stabil és robosztus, teljesen kiküszöbölve a kézremegést. Végezetül a rendszer legfontosabb jellemzője és egyben igazi újdonsága, hogy lehetővé teszi virtuális határok (virtual fixture) definiálását. Az orvos a műtétet megelőzően a CT felvételeken azonosítja az eltávolítani kívánt koponyacsont-szegmens, majd e köré felépíti a virtuális határokat, amelyek védelmet nyújtanak a sérülékeny anatómiai képleteknek. A robot a beavatkozás során (a regisztrációs eljárásnak köszönhetően) képes ezeket a korlátozásakat a 3D térben értelmezni, lassítani, ha a közelükbe ér és megakadályozni, hogy az orvos a fúróval behatoljon a tiltott területre. Ezek a funkciók együttesen nagy mértékben javíthatják a műtétek pontosságát, és jelentősen megkönnyítik a sebész feladatát. Két félévet töltöttem a robot pontosságának és biztonsági funkcióinak fejlesztésével, és kezdettől fogva aktívan részt vettem a kutatás minden területén. Első feladatként tanulmányoztam a robotot és segítettem a fantom-teszteket, majd a hibák és pontatlanságok okainak azonosításán dolgoztam. Számos területen sikerült eredményeket elérnem, többek között megvalósítottam a robot zárt kinematikai láncon alapuló paraméter-kalibrálását, illetve Kálmán szűrőt terveztem a sebészeti navigációs rendszer mérési zajának csökkentésére. Implementáltam egy új és innovatív algoritmust, amely a műtét folyamán képes a beteg elmozdulását követni, és annak megfelelően kompenzálni a robot mozgását. A robot-rendszer kezdeti tesztjei igen bíztató eredményekkel zárultak, és reméljük, hogy a beavatkozás pontosságának további növelésével egy nap majd klinikai alkalmazásba kerülhet. -9- Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Abstract Medical robotics has only existed for twenty years and yet has already made an impact on the classical practice of interventional medicine. Neurosurgery was the first field of application for robots in surgery, and throughout the past years, dozens of research projects have been devoted to robotics with brain and spine applications, differently addressing the challenges of accuracy and effectiveness. We have developed a cooperatively-controlled robot system at the Johns Hopkins University to assist with skull base surgery. The goal of the project is to improve the safety and quality of brain surgery while significantly reducing the operating time. The robot is run in compliant control mode, which means the surgeon guides the manipulator by applying force on the drilling tool, mounted at the end-effector. Our system has three major advantages. First, it offers superior visualization features for stereotactic surgery, enabling the tool’s position to be tracked on the 3D model of the patient acquired from pre-operative CT scan. Additionally, the drilling tool is mounted on the rigid mechatronic structure of the robot, greatly increasing the stability of the device. Finally, the most important and novel advantage of the application is that the surgeon can define boundaries—called virtual fixture—on the CT scan. The virtual fixtures are created prior to the operation and registered to the robot to serve as a 3D motion guide during the operation. Tool velocities are scaled around the critical areas and the controller prevents tool tip penetration of the boundaries. Together these features greatly increase the safety and the reliability of the procedure, easing the surgeon’s task and reducing the time of operation. I joined the research project for two semesters and become involved in every aspect of the project. In addition to learning about and testing the existing system, my task was to identify the main sources of errors in the setup and find solutions to fix and improve them. I have developed several additional features to the system, such as the extended closed kinematic loop calibration of robot parameters and Kalman filtering of navigation noise. Meanwhile, I have implemented a new and innovative solution to monitor and compensate for patient motion in the operating room, which would otherwise cause serious errors in the procedure. Hands-on surgery offers remarkable advantages, and the preliminary phantom and cadaver tests with our system have promising results for future applications. By further improving its accuracy, we hope to introduce this system into clinical use one day. - 10 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Acknowledgements I am grateful to Dr. Peter Kazanzides for his tireless efforts guiding me through the projects and for his valuable perspectives and ideas that founded the work. It was great honour to work with him on a fascinating robot system that shall influence the future of health care. I would like to acknowledge my laboratory mate Tian Xia, as he introduced me to the robot and we worked well together or many issues. I gratefully acknowledge the contributions of Clint Baird, George Jallo and Iulian Iordachita at the Johns Hopkins University and Johns Hopkins Medical Institute. The StealthStation navigation system was donated by Medtronic Navigation and the NeuroMate robot and force sensor were donated by Integrated Surgical Systems. The eMax surgical drill was obtained on loan from The Anspach Effort Inc. I thank Hani Haider and Andres Barrera at the University of Nebraska for providing the phantom for the robot and navigation system accuracy tests. Further, I would like to thank Tricia Gibo for the thorough and careful proofreading of my materials. The scientific work was supported in part by the NSF EEC 9731748 and the Hungarian National Scientific Research Foundation, Grant No. OTKA T69055. Last but not least, I am thankful for the generous scholarship of the Hungarian-American Enterprise Scholarship Fund that made it possible to spend two semesters in the United States and write my diploma thesis in Baltimore. - 11 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Content Introduction..............................................................................................................................14 Problem statement ................................................................................................................14 Scope and interest of the work ..............................................................................................15 Structure of the thesis ...........................................................................................................16 Notations and symbols..........................................................................................................17 Chapter 1 ..................................................................................................................................18 History and background............................................................................................................18 1.1 Introduction to surgical robotics......................................................................................18 1.2 Ontology .........................................................................................................................21 1.3 Overview of literature......................................................................................................23 1.4 History of robots in neurosurgery....................................................................................24 1.5 Past and present neurosurgical robots..............................................................................25 1.6 System improvement strategies........................................................................................31 1.6.1 Improvement of stereotactic surgery.........................................................................32 1.6.2 Integrating imaging devices.......................................................................................33 1.6.3 Hands-on surgery .....................................................................................................34 1.7 Challenges of skull base surgery.......................................................................................35 Chapter 2 ..................................................................................................................................38 Inroducing the JHU neurosurgical robot system .......................................................................38 2.1 System description...........................................................................................................38 2.2 System components.........................................................................................................39 2.2.1 The NeuroMate robot ..............................................................................................39 2.2.2 StealthStation............................................................................................................41 2.2.3 Other system components ........................................................................................42 2.3 Operational setup ............................................................................................................45 2.3.1 Calibration procedures..............................................................................................47 2.3.2 Registration procedures ............................................................................................49 2.4 Operation in cooperative control mode...........................................................................51 2.5 Results of phantom and cadaver experiments ..................................................................54 Chapter 3 ..................................................................................................................................56 Identification and measurements of the system .........................................................................56 3.1 Identifying the sources of errors ......................................................................................56 3.2 Intrinsic accuracy of the NeuroMate robot ......................................................................58 3.3 Deriving the kinematic model of the NeuroMate robot ...................................................60 3.4 Intrinsic accuracy of the StealthStation ............................................................................62 3.5 Virtual fixture definition ..................................................................................................64 - 12 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Chapter 4 ..................................................................................................................................66 Increasing the accuracy and safety .............................................................................................66 4.1 General approach ............................................................................................................66 4.2 Improving the components’ accuracy ..............................................................................66 4.2.1 Extended robot calibration .......................................................................................66 4.2.2 Improving the StealthStation’s accuracy....................................................................70 4.2.3 Verifying measurements............................................................................................74 4.3 Challenges in the OR environment ..................................................................................74 4.3.1 Compensating for patient motion .............................................................................76 4.3.2 Sensor fusion for accuracy ........................................................................................78 4.4 Verifying the results.........................................................................................................82 Conclusion ................................................................................................................................85 Future work ..............................................................................................................................87 Appendix ..................................................................................................................................88 References.................................................................................................................................89 - 13 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Introduction Surgical robotics is one of the most fascinating interdisciplinary fields of biomedical engineering. While Computer-Integrated Surgery (CIS) has only existed for a few decades, it has already spread world wide, with well over 100,000 operations performed. In the near future, newly developed robotic systems may conquer even the most challenging fields—such as neurosurgery—to provide better patient care and superior medical outcomes. We can anticipate that the future trends of clinical applications are outlined by the current leading research directions. My diploma thesis introduces the major systems and different strategies applied in robotic neurosurgery. It also presents in detail the neurosurgical robot system at the Johns Hopkins University (JHU), the research and experiments I conducted in the past two semesters. In addition to appropriate design, adequate control strategies are required to ensure maximal safety in surgical robotics. This makes any sort of robotic assisted or automated neurosurgery a technologically challenging area for researchers. Throughout my work, I tried to investigate and experiment with different computational and control methods to improve the accuracy, safety and usability of the system. This thesis gives a thorough introduction to the setup and discusses the primary results. The conclusions of current research and innovation will lead forward on the path of further improvement of medical robotic systems. Problem statement Despite the growing popularity of surgical robots, the actual clinical applications are limited to a small number of procedures and interventions. Besides the general purpose, more universal robotic systems (Zeus, da Vinci), there is a clear need for smaller, more specialized and less expensive robotic solutions. By focusing on a certain field of application, special purpose systems can achieve better precision, ergonomics and clinical outcome. Neurosurgery is one of the most demanding areas of CIS, where the complexity of the anatomical regions, the high sensitivity and delicate consistency of the tissues require fine accuracy and a narrow margin for errors. None of the previous neurosurgical robots managed to move on towards real mass production and achieve the success of the well-known da Vinci teleoperated system. We cannot talk about a major financial breakthrough because of certain functional limitations and the higher investment/maintenance costs. New systems tend to offer more significant clinical advantages that may well compensate for their high cost [1]. - 14 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Neurosurgery involves bone drilling in many cases; surgeons have to cut the skull to gain access to the actual surgical area. The minimally invasive (with minimal brain tissue resection) way to get access to the target location can be through the natural orifices of the face—typically the nostril—or through the temporal skull. The procedure is conducted with hand-held drills and can last for several hours. The most critical issues are • • • risk of damage to critical structures limits of human dexterity surgeon fatigue, malpractice. We are addressing all of these problems with the neurosurgical system at JHU. It may be possible to significantly reduce the operating time of skull base drilling with the robot, while increasing the safety of the procedure. Our system is composed of commercially available elements, making it easier to validate as a new device; however, several compatibility and protocol issues first had to be solved. To achieve acceptable application accuracy, both the individual components’ and the entire system’s precision had to be increased. Scope and interest of the work In 2007, the Hungarian American Enterprise Scholarship Fund [2] awarded me an Undergraduate Fellowship to spend my last two semesters before graduation at the Johns Hopkins University (JHU), Baltimore, MD, United States. I got involved in the research at the National Science Foundation (NSF) Engineering Research Center for Computer-Integrated Surgical Systems and Technology (ERC CISST), under Professor Peter Kazanzides, head of the Sensing, Manipulation and Real-Time Systems Laboratory (SMARTS Lab). My diploma project was to improve the accuracy and safety of the NeuroMate robot based system at the ERC CISST. The research took place partially at the JHU Homewood Campus and at the Johns Hopkins Medical Institute (JHMI), East Baltimore. While the research as a whole spans all aspects of development of a new clinical system, I have been focusing on the identification, calibration, registration and control of the robot. Throughout the development, it was a major concern to maintain the modularity of the system, creating generic solutions that may be applied for other installations as well. The combined use of robotic manipulators, surgical navigation systems and virtual fixtures as an image guided system can be adapted to many fields where safety and efficiency are paramount. - 15 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Structure of the thesis The diploma thesis consists of five major parts. Chapter 1 gives an introduction to surgical robotics, reviewing the basic definitions and classification principles of the area. It features an extensive list of past and present neurosurgical systems based on my literature research. The most important systems are presented in detail. Chapter 2 introduces the JHU neurosurgical robot system I have been working on, giving a detailed description of the components, their capabilities and the software environment. Besides testing the existing system and learning about it, my task was from the very beginning to identify the main sources of errors in the setup, and individually find solutions fixing and improving them. Chapter 3 contains the list of errors and sources of problems identified allowing further development of the system. Major sources of errors are inspected individually, and measurements are documented. Chapter 4 contains the theoretical and numerical results of my project, introducing and evaluating the developed solutions for the improvement of robot calibration, control precision and adaptation to the changing operating room (OR) environment. Finally, the conclusion and future directions of the work can be found at the end. The research continues with the aim of producing a clinically validated surgical instrument with serious market potential. - 16 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Notations and symbols Common abbreviations CAS CIS CT DH DRB DOF ERC CISST FRE GUI IGS J JHMI JHU K(d) Loc LS MIS MRI OR q RA RRB RW To From T TCP TRE US VF Computer Assisted Surgery Computer-Integrated Surgery Computer Tomography Denavit-Hartenberg (parameters) Dynamic Reference Base (coordinate frame) Degree(s) of Freedom NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology Fiducial Registration Error Graphical User Interface Image Guided Surgery Jacobian matrix of the robot Johns Hopkins Medical Institute The Johns Hopkins University Scaling matrix for virtual fixture implementation Localizer’s coordinate frame Least Squares Estimation Minimally Invasive Surgery Magnetic Resonance Imaging Operating room Robot joint vector Robotic Assisted (procedure) Robot Rigid Body (coordinate frame) Robot World (coordinate frame) Homogenous transformation matrix between frame “From” and “To” Tool Center Point of the robot (coordinate frame) Target Registration Error Ultrasound Virtual Fixture - 17 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Chapter 1 History and background 1.1 Introduction to surgical robotics Robotic surgery is entering its adulthood due to the continuous development made by research groups all over the world. From the close cooperation of engineers and physicians great medical robotic innovations have emerged. Surgical robotics represents a dynamically growing area, and according to the Medical Robotic Database, MeRoDa [3], there are more than 200 ongoing research projects worldwide. A great number of different robots have been built to perform desired tasks in neurology, orthopaedics, urology, pediatric surgery, gynecology, spine and brain surgery, cardio-vascular operations and general surgery. Laparoscopic interventions—when the surgeon controls the device based on the information provided by an image from an endoscope—are the most widespread forms of application. To improve versatility, robotic systems with two or three arms have been built. Different end effectors can be used on the tip of each arm, depending on the actual application: scissors, knives, graspers, forceps, ultrasound (US) probes, needles, sensors or even a combination of them. Unquestionably, the most well known commercialized surgical robot is the da Vinci (figure 1.1) from Intuitive Surgical Inc. (Sunnyvale, CA, USA) [4]. The robot is capable of performing complex surgical procedures with laparoscopic technique, guided remotely by a skilled surgeon. Figure 1.1 The da Vinci-S second generation general purpose teleoperated surgical manipulator system from Intuitive Surgical. (Photo: Intuitive Surgical) - 18 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery In the past few years, one of the most important applications has become prostatectomy. It paved the way for further development by providing results of improved patient outcome and winning the professional community’s approval while building the public’s trust. Beyond the da Vinci teleoperated system, there are many other robots versatile in size, function and structure. Robotic systems have several potential advantages [5], [6], [7] and [8]: • • • • • • • • • • • • • superior 3D spatial accuracy stabilization of the instruments within the surgical field improvement of manual dexterity, motion scaling tremor filtering real MIS integrated 3D vision system specific design for maximum performance advanced ergonomics high fidelity information integration stable performance invulnerability to environmental hazards patient advantages (shorter recovery) eventually shorter hospitalization We can categorize surgical robots based on their different roles in the operating room (OR) [9]. Passive robots only serve as a tool holding device once directed to the desired position. Semi-active devices perform the operation under direct human control (e.g. in compliant mode). Active devices are under computer control and automatically—or teleoperated—perform certain interventions (e.g. bone machining). Surgical robots can be involved in the procedure with various functionality; and from the control point of view, can be distinguished based on the different level of autonomy [10]. Systems that are able to perform fully automated procedures—such as CT-based biopsy or cutting—are called autonomous, or supervisory controlled. (A human supervisor would always be close to the robot, but does not intervene as long as everything goes according to the surgical plan.) This can be well combined with the classic tools of IGS. When the planning is completed, the doctors have to match the robot’s coordinates with the patient’s anatomical points, mapping the physical space to the robot’s working frame. This process is called registration. Once appropriately registered, the robot can autonomously perform the desired task by following the pre-programmed plan exactly. On the other hand, if the robot is entirely remote-controlled and the surgeon is absolutely in charge of the motion of the robot, we call it a teleoperated system. These complex systems consist of three parts; one or more slave manipulators, a master controller and a vision system - 19 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery providing visual feedback to the user. Based on the gathered visual (and sometimes haptic) information, the surgeon guides the arm by moving the controller and closely watching its effect. This can be realized by a master-slave manipulator system like the da Vinci (figure 1.2). Modifying the teleoperation control paradigm, we can introduce cooperative (also called compliant) control. It means that the surgeon is directly giving the control signals to the machine, while leaving some space for automation. This is called the hands-on technique, as the human is always in contact with the robot. In this case, the robot is the extension of the doctor’s hand equipped with special features and effectors. This technique is often used in the case of micromanipulation operations, such as micro-vascular, urology, or eye and brain operations. Figure 1.2 Master controller and slave manipulators of the first generation da Vinci telesurgical system. Before a system can be used on real patients, several in vitro and in vivo tests have to be performed. Clinical trials mean an important step towards commercialization. In the European Economic Area, the CE marking has to be acquired for a system, proving that all the relevant European Directives have been met. In the United States, the federal Food and Drug Administration (FDA) is responsible for enforcing all safety regulations in this field. Historically, the FDA has been very careful on approving new, technology-based interventional medicine devices, forcing research test projects to meet the very strict requirements. There is still a lot of room for development. Basically every aspect of the existing systems could be optimized and further improved. The SAGES-MIRA Robotic Consensus Group [6] stated in 2006 that the most important unsolved challenges are the lack of haptic feedback, the size of the robotic instruments, limitations in functionality, inflexibility of certain energy devices and the lack of multi-quadrant surgery (allowing more versatility in surgery). Beyond these, the possibility to adapt to soft tissue properties and automatically compensate for their motion is also in the main focus of research. Another promising direction is NOTES (Natural Orifice Translumenal Endoscopic Surgery), which effectively applies and extends the meaning of - 20 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery minimally invasiveness. Despite the obvious limitations of the existing systems, the offered advantages convince more and more hospital managers to invest into this technology, and many surgeons—once skeptics—have become active users of the available systems. 1.2 Ontology Surgical robotics covers many areas and sometimes different terminologies exist in the medical and engineering fields. A brief definition is given to the most important terms to facilitate the reading of the thesis. Robotic surgery is defined by the SAGES-MIRA Robotic Consensus Group [6] as “A surgical procedure or technology that adds a computer-technology-enhanced device to the interaction between the surgeon and the patient during a surgical operation, and assumes some degree of freedom of control heretofore completely reserved for the surgeon. This definition encompasses micromanipulators, remotely controlled endoscopes and console-manipulator devices. The key elements are enhancement of the surgeon’s abilities—by the vision, tissue manipulation, or tissue sensing—and alteration of the traditional direct local contact between surgeon and patient.” Beyond remote telepresence systems—such as the da Vinci—the field incorporates other smart tools and intelligent devices as well. Minimally Invasive Surgery (MIS) originally referred to the laparoscopic procedures (keyhole surgery), where the abdominal cavity is accessed through 3-5 small incisions (0.5 – 3 cm). This procedure was first reported on humans in 1910, performed by Hans Christian Jacobaeus in Sweden. Since then, different methods have been developed to access other parts of the human body as well. Today, it is getting to be a popular alternative to open procedure in many cases, reducing the patient trauma and operation risk. Robot Assisted MIS is often used to characterize the da Vinci type systems, where the robot basically serves as a replacement of the human operator manipulating endoscopic tools. Computer-Integrated Surgery (CIS) is the most commonly used expression to cover the entire field of interventional medical technology from image processing and augmented reality applications to automated tissue ablation. A subfield of it—Computer Aided Surgery—usually means that the digital system involved does not take part in the physical part of the operation, but improves the quality of surgery by better visualization or guidance. Image Guided Surgery (IGS) covers the latter field partially and had existed even before robotic innovation appeared in medicine. The idea of stereotaxis dates back to 1906; however, the first human sub-cortical procedure was performed in 1947 [11]. The technique was originally aimed at improving the performance of brain tumor surgeries, and became popular from the ‘70s due to - 21 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery the appearance of inexpensive computational power and advanced image processing. IGS means the real-time registration (correlation and mapping) of the operative field to a preoperative imaging data set of the patient, thereby defining a reference coordinate system to help with the task performance (stereotactic surgery). This leads to advanced visualization, and can be used to improve free-hand navigation, accurate positioning of equipment, or guidance of robotic devices. Figure 1.3 Figure 1.4 Image guided surgery with stereotactic frame. (Image: www.ivmed.com) Fiducial based frameless IGS registration method. (Image: www.elekta.com) Many different modalities are available for medical imaging, providing different advantages (table 1.1). Usually there are special devices involved, such as fiber optic guides, internal video cameras, flexible or rigid endoscopes or ultrasonographs. There are two common ways to perform the registration [12]. According to the more classical, frame-based method, a stereotactic frame is mounted to the patient’s head prior to the CT or MR imaging and serves as an immovable coordinate system by which any point of the brain can be referenced (figure 1.3). Image modality CT MR US Laser scanner Optical localizer resolution high (3D) high (3D) medium (2.5D/3D) high (2.5D) high (3D) reliability high high noisy image easily disturbed usually noisy discernible features bone, contrast material, (soft tissue) soft tissue, (bone) soft tissue, (bone) surface, (tissue) position information latency not real-time not real-time high (intermittent) high (intermittent) low temporal resolution n/a n/a very low low (improvable) high sampling cost high high rel. high low none patient stress high from radiation high (time, acoustic) low low none Feature Table 1.1 Most frequently used image modalities for surgery [13]. - 22 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery The more recent technique—frameless stereotaxis—involves a hand-held surgical probe, and it does not require the full rigid head-frame (figure 1.4). The probe is tracked by either mechanical, optical, ultrasonic or electromagnetic techniques, while used to touch designated points. The registration between the image space and the tracker coordinates can be achieved through fiducial-, or anatomical landmark-based paired-point registration, surface matching (point-cloud registration), or some kind of hybrid transformation. A recent study shows the superiority of the fiducial-based method, regarding the achievable application accuracy [14]. The surgical navigation system matches the two frames and provides the tool coordinates in image space. In either case, the patient’s head must be fixed relative to the mounted reference frame (Dynamic Reference Base); otherwise the registration loses its validity. Due to the improved fidelity of volume imaging systems, the increase of computing capacity and availability of cheaper 3D digitizers, IGS has become affordable and wide-spread. 1.3 Overview of literature Many books have been published in this area in the past 15 years; the most remarkables are [15], [16], [17] and [18]. The more general field of medical robotics is well covered by journals and periodicals. The most prominent research articles are published in the IEEE Transactions; Transactions on Biomedical Engineering, Transactions on Robotics (former Transactions on Robotics and Automation), Transactions on Mechatronics and in the various thematic journals, such as the Journal of Computer Aided Surgery (Taylor and Francis), Intentional Journal of Medical Robotics and Computer Assisted Surgery (Wiley), Journal of Robotic Surgery (Springer) or the Journal of Medical Devices (ASME). These publications are all available on the internet. Further teaching and information materials are available online. One of the best sources of information is the WebSurg [19] virtual university; launched by Prof. Marescaux and run by the European Institute of TeleSurgery (EITS) in Strasbourg, France. The portal is dedicated to information sharing and knowledge distribution, providing free tutorials, video materials and presentations. A good source is the Robotic Surgery Research Website [20] operated by Prof. Lysaght’s students at the Brown University (RI, USA). General medical robotic news is posted on the MedGadget internet journal [21]. There are many conferences specific to this field. The Minimally Invasive Robotic Association (MIRA), the Society for Medical Innovation and Technology (SMIT), the International Society and Conference Series on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Computer Assisted Orthopaedic Surgery (CAOS), and Medicine meets Virtual Reality (MMVR) have annual conferences dedicated to CIS and medical technology. - 23 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Furthermore, the annual Computer Assisted Radiology and Surgery (CARS), the International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), the IEEE International Conference on Robotics and Automation (ICRA), the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) and the relatively young biannual IEEE / RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) also welcome original publications dealing with the broader topic. The most classic general introductory articles and basic readings of CIS are [7], [22] and [23]. 1.4 History of robots in neurosurgery Robotics is involved in several fields of medicine, and the da Vinci-style general purpose laparoscopic multi-manipulators have performed tens of thousands of operations so far. Most of the emphasis has been put on gastro-intestinal, cardio-vascular and orthopaedic surgery. However there are promising achievements in other areas as well. Neurosurgery was the first field of application for robots in interventional medicine. Throughout the past decades, dozens of research projects have focused on the challenges brain and spine surgery. Robotic-assisted procedures offer remarkable advantages both for the patient and the surgeon. The ability to perform a surgery on a smaller scale with robots makes microsurgery a reality. The use of mechatronic devices can increase the stability of the system; using medical images it can give increased accuracy to navigate and position the surgical tool at the target point. Furthermore, there is the option to introduce advanced digital signal processing to control or record the spatial points-of-interest and motions. This can be useful for surgical simulation and risk-free training. Finally, robotized equipment can add to the ergonomics of the procedures. The main specific advantages of robotic neurosurgery systems based on [1] and [24] are: • • • • • • increased precision high quality control stability and robustness standardization, planning and reproduction of the operation saving in time (after learning the system) use of MIS techniques (e.g. in skull base surgery) In the case of neurosurgery, there is a great need for high precision, and surgeons traditionally use optical lenses and special tools to enhance their personal capabilities. CIS offers various possibilities to improve and augment human dexterity. RA-MIS promises significant results in the case of brain procedures for two main reasons. First, the skull provides a rigid frame, and - 24 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery makes it easier to register real word structures to preoperative scans of the patient. (This is the basis of effective image-guided surgery). Second, the compactness of the head allows for minimal soft tissue motion during the intervention, enabling a more accurate use of pre-operative planning. However, if a large skull opening is made during the procedure, there may be significant tissue motion. Modeling and compensating for this motion is a major field of research, although out of the scope of this thesis. 1.5 Past and present neurosurgical robots In the past decades, several different robotic neurosurgical devices have been created, with a few reaching the market. It was first proven more than twenty years ago that robots can extend human surgeons’ capabilities. Table 1.2 lists the most remarkable research projects, while the more significant systems are introduced in detail. Further descriptions of the major devices can be found in [9], [25], [26] and [27]. The first robot used on a human patient was a Puma 200 (Programmable Universal Machine for Assembly) robot—originally manufactured by Unimation Inc. (Danbury, CT)—manipulating a biopsy cannulae using a Brown-Roberts-Wells stereotactic frame (mounted to the robot’s base). The operation took place in the Memorial Medical Center (Long Beach, CA, USA) in 1985 [28]. The robot’s repeatability was 0.05 mm with an overall accuracy of 2 mm. The pneumatic gripper was used to clasp a brain retractor [29]. In later experiments, the Puma performed complete stereotactic neurosurgical operations based on the CT scan, processing the scanned images, positioning the arm at the target point and manipulating different probes. In the same year, a Cartesian linear robot was used together with a COMPASS stereotactic head frame (Compass International Inc., Rochester, MN, USA) to improve the efficiency of stereotaxis [30] by positioning the patient’s head (the target area) with the robot into the center of the stereotactic arc [5]. At the beginning of the ‘90s, a research group in Lausanne, Switzerland developed the Minerva system [31]. It was designed to have maximum access to the brain (with 5 degrees of freedom (DOF) structure) while the patient is in the CT, allowing the procedures to be performed under real-time imaging (figure 1.5). The robot was capable of performing automated skin incision, cranium drilling and instrument manipulation. It was mounted on a horizontal carrier which moved on rails. A Brown-Roberts-Wells reference frame was attached to the robot gantry and coupled to the motorized CT table by two ball and socket joints arranged in series. In 1993-95, fourteen human patients were operated with the robot at the CHUV Hospital in Switzerland [32]. The project was terminated due to patient security issues. - 25 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 1.5 Figure 1.6 Photo of the Minerva stereotactic robot system [32]. The NeuroMate stereotactic robot performing simulated brain biopsy [33]. The NeuroMate [34] was the first neurosurgical robotic device to get a CE mark in Europe, followed by the US Food and Drug Administration’s (FDA) approval in 1997 for stereotactic neurosurgical procedures (figure 1.6). After having performed over 3000 operations, it was also approved for frameless stereotactic surgery in 1999. It has a CE mark for neuro-endoscopic applications as well. Originally developed at the Grenoble University Hospital and produced by Innovative Medical Machines International (IMMI, Lyon, France), the 5 DOF NeuroMate provides an accurate and trusted assistance for supervised needle positioning and instrument holding for brain biopsy. The system was mostly used with an imaging workstation (VOXIM)— a software for planning and registration of pre-operative images. The optional ActMate module was for visualization of the instrument during the procedure, allowing the user to select a particular configuration of the robot when the same target point could be reached in multiple configurations. Project [ref.] Category Institute, company Main features Alpha robot [35] Active, teleoperated MicroDexterity Systems Inc.; Albuquerque, NM, USA 5 DOF parallel manipulator mounted on the stereotactic frame, CA BWH MRI robot [36] Active, automated Brigham and Women’s Hospital; Harvard Medical School; Boston, MA, USA 5 DOF MRI guided robot for percutaneous procedures, tool navigation and biopsy. CAS-BH5 robot system [37] Active, teleoperated Navy General Hospital of PLA, Beijing University; Beijing, China Facilitates remote planning and transmission of neuronavigation data, monitoring and manipulating Cranio [38] Active, automated RWTH-Aachen / Lehrstuhl für Biomedizinische Technik; Aachen, DE Craniectomy with 6 DOF hexapod robot Active, automated Active, automated University of Karlsruhe; Karlsruhe, Germany Modified 6 DOF Stäubli RX-90 robot for craniofacial bone milling, under optical tracking for safety Accuray Inc.; Sunnyvale, CA, USA Image guided radiotherapy, tumor irradiation, CA Craniofacial Surgery Robot [39] Cyberknife [40] Table 1.2 Major neurosurgical robotic projects and their features. (CA = commercially available) - 26 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Project [ref.] Category Institute, company Main features Evolution 1 [ 41 ] Semi-active, automated Universal Robot Systems; Schwerin, Germany - defunct 6 DOF hexapod robot for pedicle screw placement and adenoma dissection, CA, discontinued FC robot for otoneurosurgery [42] Active, automated Fraunhofer-Institut für Produktionstechnik und Automatisierung (IPA); Stuttgart, DE Force-controlled Stäubli RX-130 robot for automated temporal skull base drilling IGOR [43] Passive/active /semi-active TIMB (Trait. de l’Information et Modélisation en Bio-médecine); Grenoble, France Prototype robot for general image guided neurosurgery (later became NeuroMate) Cooperative control Active, teleoperated Active, automated Active, autom. /teleoperated Active, automated Passive, automated Active, automated Active, teleoperated Active, automated Active, automated Active, teleoperated Active, automated Active, teleoperated Cooperative control Active, automated Passive, automated Active, automated Active, automated Active, teleoperated Active, teleoperated Johns Hopkins University; Baltimore, MD, USA DLR / BrainLAB AG; Feldkirchen, Germany Mazor Surgical Technologies Inc.; Caesarea, Israel University of Erlangen-Nurenberg, FAU Medical School, Erlangen, Germany Lab. of Microengineering, Swiss Federal Inst. of Tech.; Lausanne, Switzerland Beijing University of Aeronautics and Astronautics; Beijing, China Skull base drilling with force based co-operative control with virtual fixtures Light-weight, high payload 7DOF robot for MIS neurosurgery, CA soon FDA approved, light-weight, head mountable robot for needle insertion, CA soon Fully automated spheniodotomy with a Mitsubishi RV-1a 6 DOF robot Real time frameless stereotactic instrument guidance in CT scanner, discontinued PUMA260 robot based system with optical tracking for brain biopsy 6 DOF manipulator for surgical microscope positioning, CA Master-slave system with two 6 DOF arms and a HD 3D vision system for neuro-microsurgery Light-weight, 6 DOF universal-prismatic-spherical parallel mechanism for skull drilling High Intensity Focused US treatment for destruction of subcortical lesions MRI compatible complete multi-manipulator, in clinical trials Instrument guidance, skull-base surgery with a 5+6 DOF parallel robot with force sensor Teleoperated system with three arms and increased micromanipulation capabilities 4 DOF serial robot with remote center of motion for, discontinued JHU project w/ NeuroMate [44] KineMedic [45] MARS robot (SmartAssist) [46] MI Transsphenoidal surgical robot [47] Minerva [48] MIS Stereotactic Robot [49] MKM [50] MM-1 robot [51] Modular Parallel Robot [52] Motorized HIFU system [53] neuroArm [54] NeuRobot [55] NeuroBot [56] Neurobot [57] NeuroMaster [58] NeuroMate [59] NIRS [60] PathFinder [61] Raven [62] RAMS [63] Steady-Hand Robot [64] SurgiScope [65] Tele-Robotic Skull Drill System [66] UPenn robotic setup [114] UTokyo MRI robot [67] Cooperative control Active, manual / automated Active, autom. / teleoperated Active, teleoperated Active, automated Table 1.2 cont. Carl Zeiss AG; Oberkochen, Germany University of Tokyo; Tokyo, Japan Yuan Ze University; Taiwan Imperial College of Science, Tech. and Medicine, London, UK University of Calgary; Canada Nanyang Technological University; Singapore Shinshu University School of Medicine; Matsumoto, Japan Imperial College, London, UK Robotic Institute Beihang University; Beijing, China IMMI / ISS / Schaerer Mayfield NeuroMate Sarl; Lyon, France 6 DOF robot for stereotactic procedures 5 DOF cannulae positioning for biopsy, neuroendoscopy, CA National Neuroscience Institute; Singapore Automated pocket milling of the skull base Prosurgics Ltd. (formerly Armstrong Healthcare Ltd.); High Wycombe, UK 6 DOF manipulator for instrument guidance, CA University of Washington; WA, USA 6 DOF general surgery, automated suction NASA JPL; Pasadena, CA, USA 6 DOF manipulator for eye and brain surgery with motion scaling and tremor filtering, discontinued Johns Hopkins University; Baltimore, MD, USA Intelligent Surgical Instruments & Systems; Grenoble, France Dalhousie University; Halifax, NS, Canada University of Pennsylvania, Philadelphia, PA, USA University of Tokyo; Tokyo, Japan 7 DOF robot with advanced tremor filtering for MIS needle driving Ceiling mounted robotized tool-holder device for surgical navigation, CA Test bed for skull base drilling with Mitsubishi PA-10 6 DOF robot Transoral skull base surgery with a da Vinci robot; first human application Two ultrasonic motors and 6 DOF sterilizable manipulator for needle insertion Major neurosurgical robotic projects and their features. (CA = commercially available) - 27 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery A typical clinical procedure with the NeuroMate follows the steps of classic IGS. First comes the initial data acquisition step (obtaining images in either DSA, CT, MRI or in digitized radiograph format [68]), then the path planning that determines the trajectory from skin entry point to the target location using a specially developed software program. Once the path is defined, the images are transferred directly from the planning workstation to the control computer in the OR. The technology was bought by Integrated Surgical Systems Inc. (Sacramento, CA, USA) in 1997 and later was acquired by Schaerer Mayfield NeuroMate AG (Lyon, France) [51]. The Evolution 1 robotic system (Universal Robot Systems, Schwerin, Germany) had been used for endoscope-assisted transphenoidal pituitary adenoma resections, endoscopic third ventriculostomy and pedicle screw placements in the spine [41]. The 6 DOF Hexapod Robot is based on a parallel actuator configuration, allowing very fast and precise positioning—supporting accurate stereotaxis. The system is no longer in production. The Mechatronics Laboratory at the University of Tokyo, Japan developed an MRIcompatible needle insertion manipulator intended for stereotactic neurosurgery [67]. The smallsized manipulator (491 mm high) was made of polyethylene terephthalate (PET) and ultrasonic motors were used for the actuators. Non-ferromagnetic materials (brass, aluminum, delrin and ceramics) were used to build the rest of the structure, keeping it compatible with a magnetic field of up to 0.5 T. The robot was tested on watermelon phantoms and the positioning error was measured to be 3.3 mm at maximum. The Harvard Medical School in cooperation with the Mechanical Engineering Laboratory, AIST, MITI (Tsukuba, Japan) have built a 5 DOF MRI compatible robot [29]. It is made with non-magnetic ultrasonic motors, para-magnetic materials (titanium, plastic) and a parallel link configuration. The system works together with an intra-operative MRI system to assist MIS catheter direction and navigation. NeuRobot is a popular name in the field, there are at least three different projects under this label (see table 1.2). Most successful is the da Vinci-like telemanipulator system developed at the Japanese Shinshu University [56] and [69]. It consists of four main parts; three fully deployable 6 DOF slave arms, set up in an insertion cylinder measuring 10 mm in diameter, a manipulatorsupporting device, an operation-input device (the master manipulator) and a three-dimensional display monitor (figure 1.7). The 3D endoscope holder has three DOF (rotation, neck swinging, and forward/backward motion). Successful human clinical trials have been reported [70] and also teleoperation experiments were performed on rats from 40 km distance [71]. - 28 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 1.7 Figure 1.8 The three-armed NeuRobot telemanipulator system at the Shinsu University, Japan [56]. The RAMS microsurgical teleoperated arms at JPL, Pasadena, CA. (Photo: NASA) The National Aeronautics and Space Administration (NASA) become interested in surgical robotics at the beginning of the ‘90s, and by 1997 the Jet Propulsion Laboratory (Pasadena, CA, USA) and the MicroDexterity Systems Inc. (Albuquerque, NM, USA) developed their new robot, the RAMS (Robot-Assisted Micro-Surgery) [63]. The 6 DOF RAMS arms are 25 cm long and 2.5 cm wide, having a remarkable 400 cm3 workspace (figure 1.8). They are both equipped with 6 DOF tip-force sensors to provide haptic feedback to the operator. The robot was originally aimed for ophthalmic procedures, especially for laser retina surgery. The master controller is a tendon-driven small arm, also with six joints, and is capable of 1:100 scaling (achieving 10 micron accuracy), tremor filtering (8-14 Hz) and eye tracking. Although it was not intended for market sale, clinical trials were scheduled after successful animal tests [72]. Unfortunately, the project was discontinued due to financial reasons. Other, more compact surgical robots have also been developed. Doctors and scientists at the BioRobotics Laboratory, University of Washington have developed a portable surgical robot that can be installed anywhere with its limited size and 22 kg overall mass [62]. The robot—called the Raven—has two articulated arms, each holding a stainless steel shaft for different surgical tools (figure 1.9-10). It can be easily assembled even by non-engineers, and its communication links have been designed for long distance remote-control. In addition to the possibility of haptic feedback, multiple sensors are mounted on the robot to provide more information to the surgeon and to avoid any critical failure due to the communication delay. Compactness was considered as priority throughout its development; the creators optimized the robot’s dimensions and motion by computer, minimizing the space occupied without compromising its manipulation capabilities. Currently, different ways of automated suction during neurosurgery are investigated with the robot. - 29 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 1.9-10 Figure 1.11 The Raven robot in field and underwater telemanipulation test. (Photo:University of Washington) The M7 robot on board of the NASA Aquarius sea habitat performing the world’s first automated US guided tumor biopsy. (Photo: NASA) Realizing the importance of a light but stiff structure, SRI International (developer of the first prototype of the da Vinci) in Menlo Park, California started to develop the M7 robot in 1998 under a contract from the Telemedicine and Advanced Technology Research Center (TATRC). M7 is a portable and deployable light-weight (15 kg) surgical robot (figure 1.11). The system is able to exert significant forces compared to its size, enabling it to carry out bone drilling tasks as well. The system consists of two anthropomorphic 6 DOF arms (plus gripper) and is equipped with motion scaling (1:10), tremor filtering and haptic feedback. The various effectors used by the robot (e.g. laser tissue welding tool) can be changed very rapidly. The software of the M7 has been updated lately to better suit the requirements of teleoperation and communication via Ethernet cable, with additional improvements to the master optics and stereo video processing. The German Aerospace Center (DLR) has already built several generations of light-weight robotic arms for ground and space application. Their latest 7 DOF surgical robot is called KineMedic, which consists of a single arm weighting only 10 kg and capable of handling a 30 N payload with high accuracy. It has been considered primarily for neurosurgical interventions. Its industrial version can be equipped with a dexterous 4-finger artificial hand and has already won several awards [45]. KUKA Roboter GmbH (Augsburg, Germany) is about to commercialize it. Mechatronic systems have been used since the 1970s to better position the patient’s head for radiation therapy. The Leksell Gamma Knife Perfexion (Leksell, Sweden) has already treated more than 500,000 patients world wide [73]. Using the same basic idea, one of the most successful robotic applications is the CyberKnife (Accuray, Sunnyvale, CA, USA). This stereotactic radiosurgery system integrates IGS with robotic positioning (figure 1.12). The 6 MeV LINAC relatively light-weight photon device is mounted on a KUKA 6 DOF industrial manipulator (KUKA Roboter GmbH, Augsburg, Germany). Its primary use is the irradiation of - 30 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery brain and spine tumors. X-ray cameras are used to track the spatial displacement of the patient and compensate for motion caused e.g. by breathing. The overall accuracy of the system is 0.42 ± 0.4 mm (mean ± standard deviation). Figure 1.12 Figure 1.13 CyberKnife robotic system for radiation therapy. (Photo: Accuray Inc.) The Mehrkoordinaten Manipulator (MKM) robotized neurosurgical microscope. (Photo: Carl Zeiss AG) Surgical microscopes have also profited from the development of technology for neurosurgical tools, with applications including a robotized microscope holder for frameless registration. The Mehrkoordinaten Manipulator (MKM) (Carl Zeiss AG, Oberkochen, Germany) [50] and the SurgiScope (Intelligent Surgical Instruments & Systems, Grenoble, France) [65] systems both use laser beams to determine the location of the focal point of the robotic microscopes (figure 1.13). The pre-operative images of the brain are downloaded to the workstation, where the surgeon can determine the target point and the approach route to the lesion. The microscopes are able to auto-focus on the chosen markers and assist throughout the procedure. It is also possible to superimpose additional information on the image, such as the contours of the lesion. Beyond the presented systems, many other exist, and every year new, interesting concepts are presented at conferences and in research papers. The United States is still considered to be the leader in the field, but there is strong competition from Western Europe and more recently from Asia. 1.6 System improvement strategies Current research projects are trying to increase the utility of the surgical equipment along different strategies. They are mainly focusing on three areas for improvement: • augmenting the overall accuracy and/or efficacy of the classic stereotactic systems • increasing the added-value of the equipment • further enhancing the capabilities of the human surgeon, providing smarter tools. - 31 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery The following ongoing research examples give insight to how these issues are addressed and what are the benefits for future patients. Safety is paramount in all cases, and should determine the way research is conducted. Patient safety is addressed differently in each discussed system. 1.6.1 Improvement of stereotactic surgery The European Union’s most recent initiative—the ROBOCAST project (Robot and Sensors Integration for Computer Assisted Surgery and Therapy, FP7)—aims to augment existing IGS techniques and to find new ways to perform high-precision keyhole neurosurgery [61] (figure 1.14). The modular system to be built will consist of two manipulators and one smaller probe, actively cooperating in a bio-mimetic sensory-motor integrated framework. The PathFinder system (Prosurgics Inc., UK) forms the basis of the larger positioning robots (figure 1.15). The stereotactic 6 DOF PathFinder is already available on the European market. It works with the CT or MRI images of the patient and automatically registers the position of the probe (with at least 1.25 mm accuracy). In general practice, it is capable of aligning the surgical tools within 1 mm of the target. In 2003, human experiments showed that the application accuracy was 0.44 ± 0.02 mm using the robot, in comparison with an error of 0.98 ± 0.02 mm with the stereotactic frame and 1.96 ± 1.6 mm with a standard (frameless) navigation system. The ROBOCAST systems will use optical trackers for patient safety (to monitor and compensate for any change in the patient’s position) and provide visual information of the surgical field. Given an accurate registration, the controller will use the preoperative diagnostic information to plan the path of the intervention. Figure 1.14 Figure 1.15 The ROBOCAST project’s visionary setup [74]. Tool navigation with the PathFinder robot. (Photo: Prosurgics Inc.) Improving the efficiency and precision of stereotactic procedure will lead to more gainful surgery of certain brain tumors and lesions. Deep brain stimulation electrodes could be placed very accurately with this kind of system, resulting in the routine treatment of Parkinson and similar diseases. The trajectory within the brain will be planned on the basis of a risk atlas - 32 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery (identifying critical structures), reproducing a fuzzy representation of statistical anatomical atlases. Construction of the atlas will be based on cognitive learning, with the possibility of simulating predicted consequences of certain surgical maneuvers. Intra-operative imaging devices (e.g. ultrasound) will be used to continuously update and improve the atlas. Applications include effective percutaneous brachytherapy, where radioactive seeds are implanted to kill the cancer cells, and the removal of blood clots based on preoperative images [74]. Currently, ROBOCAST is planned to be used to inject stem cells into the brain to treat Alzheimer’s and other diseases 1.6.2 Integrating imaging devices The other main direction of development is to integrate the robots with advanced imaging devices to increase their utility by allowing intraoperative imaging. This can be very challenging technically, but offers the highest level of added-value to the procedure. Magnetic resonance imaging (MRI) gives a fine resolution picture of soft tissues with an acceptable time rate (see table 1.1), while avoiding patient and surgeon radiation exposure. MR compatible robotics has been in the focus of research interest since the mid ‘90s and is the subject of a recent issue of the IEEE Engineering in Medicine and Biology magazine [75]. Figure 1.16 Figure 1.17 The MR safe 6 DOF arms of the NeuroMate robot. (Photos: University of Calgary) The neuroArm and project leader Dr. Sutherland during the first open demo. NeuroArm [54] is a recent teleoperated anthropomorphic robot from a University of Calgary led consortium (figure 1.16-17). The MRI compatible robot (up to a 1.5 Tesla magnetic field) is designed for stereotaxis and microsurgery. Beyond motion scaling and high definition visual feedback, the neuroArm is able to provide very accurate 3D information of its two 7 DOF arms. It uses three displays to give complete visual coverage of the operating environment, showing in parallel the 3D stereoscopic view of the operation, the MR image of the patient and the control - 33 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery panel. The system has been used on two human patients so far, and with further clinical trials beginning shortly, the robot may hit the market in the next years. 1.6.3 Hands-on surgery Another neurosurgical research project at the Johns Hopkins University is a good example of the cooperative control concept [64]. The hands-on system is called the Steady-Hand Robot is capable of significantly increasing the performance of human surgeons (figure 1.18-19). The robot holds interchangeable tools and the force sensor that has low-threshold silicon strain gauges built in to detect forces. The force measurements are coupled to the control system after tremor filtering and smoothing. This approach has the advantage of simplicity, less expensive implementation and provides greater immediacy for the user. However, the possibility of teleoperation is completely lost. Surgeons may be more willing to accept this form of robotized equipment, as it still supports the classical way of doing the procedure. The robot is based on a Cartesian stage (allowing three orthogonal translational motions) and a Remote Center of Motion (RCM) stage (with two orthogonal rotational DOF) that helps to keep a user-defined point (RCM) of the robot at the same position in space. This allows safe surgery through the abdomen (or other tissue layers), where the incision point limits the motion in space. Figure 1.18-19 The Steady-Hand Robot at the Johns Hopkins University. (Photo: CISST) There are other concepts and approaches that seek to find ways to better serve the clinicians. Any new strategy must be developed using the same strategic principles: the system should pose minimal risk to the patient (compared with classical methods), there shall be major clinical advantages to justify its use and finally, the investment and maintenance costs should be reasonable for medical centers. We chose the latter paradigm for our neurosurgical system at Johns JHU. Our robot is using cooperative control to assist the surgeons, and does not change the general method of the - 34 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery procedure, and has good safety features (keeping the surgeon in control of the robot at all times). These aspects may lead to an easier validation and approval, while still having significant impact on the clinical outcome. 1.7 Challenges of skull base surgery Nearly all neurosurgical procedures require some amount of bone cutting—typically, craniotomy (temporary opening) or craniectomy (when the bone is permanently removed)—to gain access to the brain (figure 1.20). The surgery always begins with the opening of the skull. For example, posterior skull base lesions (such as acoustic neuromas or meningiomas of the cerebellopontine angle) and neurovascular compression syndromes (such as trigeminal neuralgia or hemifacial spasm) can all be approached through a small size opening behind the ipsilateral ear—via a suboccipital approach. This is achieved by first making a 3 cm retrauricular scalp incision and then performing a 1.5 cm retrosigmoid craniectomy. The dura is incised and CSF is drained from the posterior fossa. After relaxation of the cerebellum, a surgical endoscope or any other device can be introduced into the posterior fossa [76]. Another example is transnasal approach; anterior endonasal endoscopic surgery allows physicians to reach the paranasal sinuses or the anterior fossa. Table 1.3 presents the major traditional and MIS approaches in skull base surgery. Figure 1.20 Figure 1.21 Classic pterionale craniotomy procedure, when the skull is opened for further operations. (Photo: Melissa de Wolfe) The complex anatomical structure of the skull base. (Picture: Anatomy Atlases) Minimally invasive treatment of the skull base is already possible with classic laparoscopic and endoscopic tool; however, robotic technology can greatly improve the effectiveness of the procedures. MIS in this field has shown several advantages over classical one, 95% of the patients undergoing this type of operation spend only one night at the emergency room postoperatively and released home within 48 hours. - 35 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Traditional MIS (endoscopic) Bifrontal craniotomy Endonasal (transnasal) Translabyrinthine Retromastoid, suboccipitalis Pterional craniotomy Supeaorbital Table 1.3 Traditional approaches for skull base surgery and their MIS equivalent [76] Different pathologies can be treated with MIS skull base surgery, as listed in table 1.4. Figures 1.22-24 show three different MIS transnasal approaches, together with a particular case. For better understanding, the Skull Base Institute runs an interactive site to present the most common surgical cases and their treatment, with 3D animations presenting the procedures [76]. In certain cases, it is necessary to do further bone cutting inside the skull, at the skull base (e.g. partial removal of the acoustic canal), to gain access to a tumor, chordoma or vascular lesion (including aneurysms, malformations of the veins or arteries and fistulas). During the procedure, physicians typically operate on one of the three regions of the skull (anterior, middle and posterior fossa). The skull base is one of the most complex and vulnerable anatomical areas [77], making it a unique challenge to navigate around while avoiding damage to the cranial nerves, brain tissue and fine blood vessels (figure 1.21). Eight nerves exit the skull base in the region typically affected during these procedures. If the cranial nerves are damaged, permanent or temporary loss of critical neural functions may occur. This is an extremely delicate task requiring high precision and can last for hours. Neurosurgeons who work within these areas of the brain must deal with the specific region carefully, with regard to the size and type of lesion to be removed. The limited workspace also means that the surgical area is often obscured, or covered by tissue. Presently, surgeons use microscopes and perform the operation with extreme caution to avoid costly and fatal mistakes. MIS transnasal approach Type of pathology Pituitary Adenoma Intrasellar Craniopharyingioma CSF leak repair Suprasellar Craniopharyngioma Planum Meningioma Lateral Sphenoid Meningocele Infratemporal Fossa Tumor Osteoma Olfactory Groove Meningioma Pre-pontine Cyst Transsphenoidal app. Transtuberculum / transplanum app. Transpterygoid app. Transethmoidal / Transcribiform app. Transclivial app. Table 1.4 Different kinds of diseases and leisure with their surgical solutions (i.e., skull base drilling) [78]. - 36 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Due to the delicate nature of the interventions, patients may be in the operating room from six to fifteen hours and beyond. This is physically and mentally tiring for the physicians, who must hold the drills and focus for several hours. Fatigue results in further extension of the operating time, increasing both the costs and the chance for human error. Currently used stereotactic navigation systems only provide tool location information and anatomical visualization to support the operation, but do not address many of the problems mentioned. f Figure 1.22 Figure 1.23 Figure 1.24 Schematic drawing of endoscopic endonasal removal (via the nostril) of anterior skull base tumors and an example image showing the delivery of a meningioma [79]. Endos. endonasal removal of the clival or the posterior cranial fossa tumors and pre-operative axial MR view of a large chordoma in the clivus [79]. Endos. endonasal removal of tumors at the cavernous sinus, optic nerve, sella and suprasellar region and a pre-operative MR sagittal view of a recurrent malignant pituitary tumor (T) [79]. - 37 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Chapter 2 Inroducing the JHU neurosurgical robot system 2.1 System description At the Johns Hopkins University I joined the SMARTS team in developing a cooperativelycontrolled robot system to assist with skull base surgery [44]. The system consists of a modified NeuroMate robot, a surgical navigation device—Stealth Station and adequate network and control equipment (figure 2.1-2). The goal is to improve the safety and quality of brain surgery while significantly reducing the operating time. The robotized solution is only used for the removal of the bone tissue, to gain access to the anatomical region affected by a tumor or other disease. Our technical approach is to use a preoperative image—such as CT—to identify the region of the skull base that can be safely drilled. We chose a cooperative control implementation (also called compliant control), in which the surgeon applies forces to move the robot and the robot enforces the safety boundaries. Other example for robots with similar “hand-on” control concept include the Steady-Hand Robot at JHU [44] and the Acrobot developed at Imperial College (London, UK) for total knee replacement [80]. This chapter presents our system and the results of preliminary phantom and cadaver studies. Figure 2.1 Figure 2.2 The NeuroMate robot in action, bone cutting on the skull base in a cadaver experiment at JHMI. The integrated robotic system moved to the R. A. Swirnow Mock Operating Room at JHU. The JHU system has three major advantages. First, it offers the visualization features used in stereotactic surgery; the tool’s position can be followed on the 3D model of the patient, acquired from pre-operative CT scans. Second, the surgical tool is mounted on the rigid robot, thereby improving its stability. The stiffness of the structure basically eliminates the physiological hand - 38 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery tremor. The surgeon still holds the classic drill tool and directs its movement, but he or she can release the tool any time. Sometimes it is necessary to release the drill in order to perform other actions, such as irrigation, suctioning, manipulating the endoscope or adjusting one’s grip on the drill. It is now possible to make these without removing the tool from the operating area. The most important advantage—and the real novelty of the application—is that the surgeon can define virtual boundaries on the CT scan prior to the operation. These are called virtual fixtures (VF), and once registered to the robot, they are enforced in real space, thus preventing the tip of the tool from going beyond the defined safe area. These features together greatly increase the safety and the reliability of the procedure, ease the surgeon’s task and therefore potentially reduce operating time. 2.2 System components The JHU system consists of two major FDA approved hardware elements, the NeuroMate robot and the StealthStation navigation system (figure 2.3). These and other components—such as the force sensor, the drill, or the visualization and control workstations—are introduced in detail below. Figure 2.3 Elements of the integrated neurosurgical system at JHU. 2.2.1 The NeuroMate robot We have developed an integrated system that uses a NeuroMate robot as a new and effective surgical tool by adding a force sensor to the tip of the manipulator (figure 2.4). Our system - 39 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery possesses all the advantages of the original system, while the modifications allow for extended use of the robot. The NeuroMate consists of 5 revolute joints, each mobilized by a separate, high precision servo. The joint values are read by encoders with a resolution of 1/26825 degree due to the high gearing. The Neuromate contains embedded joint controller boards that are integrated into the links of the robot, significantly reducing the required cabling (figure 2.5). Each joint controller board contains a microprocessor and is responsible for controlling up to two axes of the robot, including the power amplification. Our Neuromate contains a newer design of the joint controller boards (provided by Integrated Surgical Systems Inc.) that does not exist in the standard product. The power supplies are placed in the triangle shaped base, eliminating the need for a separate controller rack. The system communicates with the main PC through a Controller Area Network (CAN) bus. The CAN bus is a broadcasting based differential serial bus protocol for connecting electronic control units, allowing each node to send and receive messages in one direction at a time. We are able to communicate with the joint controller board in every 18.2 ms. In every communication cycle, we can read the joint encoders and give new commands to the servos. On the lowest level, the joints are given position commands. The highest linear velocity of the robot is approximately 50 mm /sec. The NeuroMate’s previously reported intrinsic accuracy (i.e., the precision of the individual hardware and software component) is 0.75 mm, with a repeatability of 0.15 mm [33]. In a human stereotactic surgical setup, conducted in 2002, the application accuracy (i.e., the overall precision in performing the desired task) was measured to be 1.95 ± 0.44 mm (mean ± standard deviation) [59]. Figure 2.4 Figure 2.5 The NeuroMate robot used at JHU for skull base drilling. Controller boards located within the links of the robot. - 40 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery 2.2.2 StealthStation Our system uses an FDA-approved navigation system, StealthStation (Medtronic Navigation, Louisville, CO). IGS became feasible with the spread of surgical localizers. These devices are able to detect objects in 3D space, with a limited range of approximately 1.5 – 2 meters and 15 - 20 degrees. The StealthStation uses an infrared LED array to illuminate the target area, whereupon the reflections are monitored by two Polaris cameras, with a base distance of 50 cm. The system can use active or passive markers. The passive markers consist of small balls with highly reflective paint (in the infrared spectrum). The images are segmented and processed by the controller within the StealthStation rack to gain the information about the tools. Each rigid body has four balls mounted in different arrangements. Three would be enough for 3D localization, but redundancy is used to obtain better accuracy. If all markers are visible, the StealthStation computes the base point relevant to the rigid body (based on the defined geometric parameters). The system is only capable of tracking two rigid bodies at a time (one reference frame and one tool), but there is the option to manually switch between different frames and tools. It is possible to access the raw data from the StealthStation through the StealthLink research interface [81]. In every cycle we can read the homogenous transformation matrix from the inner base (camera) coordinate frame to the two tracked rigid bodies, in addition to a geometry error that gives information about the accuracy of segmentation and model fitting. This means that the camera can be moved during the surgery at any time without losing position information, as the navigation system can always provide the position of the tracked rigid bodies relative to each other. Unfortunately, we cannot get direct information about certain sub-optimal conditions, such as a partial coverage of the rigid bodies or when just three markers are in sight. Figure 2.6 Figure 2.7 Screenshot of the StealthStation surgical navigation system’s GUI during registration. The JR3 force sensor and the Anspach eMax 2 drilling tool attached to the robot. - 41 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery The StealthStation rack consists of the controller for the system and a PC for visualization of the surgical tool with respect to the preoperative image. In addition to the axial, coronal and sagittal views of the CT, the 3D reconstruction is also displayed. The computer’s standard inputs serve as the human interface (figure 2.6). The reported intrinsic accuracy of the system is 0.04 – 0.29 mm in different arrangements [82], with an application accuracy of 1.6 ± 0.68 mm [83], though it was shown to be greatly dependent on the lighting conditions. The position of the probes and line of sight for the infrared CCD cameras are critical issues throughout the procedure, as there is heterogeneity in the localization error over the workspace and covering of any of the optical markers can result in significant error. In our setup, we use three different rigid bodies (out of which two can be tracked at the same time). One marker frame is fixed on the robot’s end-effector, below the last joint. Another one is connected to the patient (at a Mayfield skull clamp). These two marker-sets allow us to determine the robot’s relative position to the skull. A third tool, a hand-held pointing probe is used for registration. The information gathered with the latter one is used to register the CT coordinates to the real world (see Section 2.3.2). 2.2.3 Other system components To obtain the cooperative control feature, a force sensor had to be added to the robot (figure 2.7). A 6 DOF force sensor (JR3 Inc., Woodland, CA, USA) measures the forces and torques applied on the end-effector. It is useable up to a maximum of 100 N force in the X and Y directions and 200 N in the Z direction. Its sensitivity is 0.1 N, 0.1 N, 0.2 N respectively and the time resolution is 400 Hz. (We read the sensor output from the vendor’s custom adapter board installed in the PC.) This resolution allows fine manipulation with force control. The tool at the end-effector is an Anspach eMax 2 high-speed, clinical bone drilling surgical instrument (The Anspach Effort Inc., Palm Beach Gardens, FL, USA). This is a real-life surgical device, designed to be a classic hand-held tool, controlled by a foot-pedal (figure 2.8). It has torque compensation, varying cutting burr loading and “soft start” performance for smooth cutting. The drill provides bi-directional rotation and operates at a maximum velocity of 80,000 rpm and comes with interchangeable tool-heads. We have several 3-5 mm, diamond coated milling and drilling heads (figure 2.9). The tool-holder (with reinforcing bracket) is attached to the end of the NeuroMate through the force sensor (figure 2.7). - 42 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 2.8 The Anspach eMax 2 surgical drill’s console, foot pedal and hand-held tool. (Image: The Anspach Effort Inc.) Figure 2.9 Different milling tools for the eMax 2 surgical drill. The system further integrates the 3D Slicer (http://www.slicer.org) software [84] for preoperative planning and intra-operative visualization (figure 2.10). 3D Slicer is an open source, cross-platform application for visualizing and analyzing medical image data, developed by researchers at the Brigham & Women's Hospital (Boston, MA, USA) and around the world. This is used as our planning system because it enables us to create complex virtual fixtures and export them in an open file format (VTK polydata). In the planning phase (prior to the operation) we use the image editing and model creation features of Visualization ToolKit (VTK) to define the VF. We can display the tool in Slicer and arbitrary manipulate the CT scans. The open platform allows implementation of further tools and features. During the procedure, Slicer displays the cutting tool with respect to the virtual fixture and preoperative image (the StealthStation can only display the image, since we have no means to load the virtual fixture). The concept of the VF was originally introduced by Rosenberg in 1993 [85] and successfully applied to robotic surgery at JHU with the Steady-Hand Robot [64]. Implementing a VF involves the superposition of an abstract, pre-defined spatial subspace on the real workspace of a robot. By applying different control rules within the VF, the effectiveness of telepresence and telemanipulation can be greatly improved. When a human is guiding a robot without force feedback, it is hard to follow geometric shapes or surfaces. VF serves as a 3D ruler, allowing the user to move along a certain boundary. Once the preoperative image is registered to the robot coordinate system, the robot can ensure that the mounted cutting tool, remains within the safe zone defined in the preoperative image. Beyond safety, the ergonomic implementation of the VF is also important, to smooth the robot’s motion; in other worlds, the scaled motion should resemble to the natural surgical hand movements. - 43 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 2.10 Screenshot of the Slicer 3D program with loaded phantom CT and defined virtual fixture. Figure 2.11 The Main window of the neurosurgical controller with various functionalities. In our application, the use of virtual fixtures is the most significant feature for improving the surgeon’s performance. (However, for research purposes, it is possible to deactivate it.) It is defined in Slicer 3D as an arbitrary volume, and approximated with a convex hull for further computations. The VF divides the robot’s workspace into three areas: • free space (away from the VF), where the robot is free to move • boundary zone (within the proximity of the VF), where the robot’s speed is reduced • forbidden zone, where the robot is not allowed to penetrate the VF Depending on which region the robot operates in, a different control strategy is chosen, as described later. The central controller of the robot is run on a separate PC workstation containing the RealTime Application Interface (RTAI) for Linux [86]. RTAI is an open-source, real-time extension for the Linux kernel that applies strict timing constraints. The robot control software was written in the C++ language and contains approximately 10,000 lines. It uses the open source ERC CISST Software library package [87]. This is a set of libraries developed at the ERC for robotic applications [88]. The CISST SW facilitated the use of basic linear algebra operations, matrix manipulations and numerical methods. Other libraries are for real-time support, device interfaces, tracking systems, robot control and stereo vision development. Our high-level robot controller consists of a mainTask, controlTask and neuromateTask. The neuromateTask communicates with the robot through the CAN bus to gather the joint feedbacks and establish target joint positions. Joint velocity commands are created on this level. The controlTask implements the supervisory control layer and is responsible for the realization of the cooperative force control and virtual fixture computation during drilling. It also communicates with the StealthStation and reads the force sensor. The neuromateTask and controlTasks require - 44 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery periodic, real-time execution, which is provided by RTAI. On the highest level, the graphical user interface (GUI) is managed by the main thread. The GUI (figure 2.11) was created using the Fast Light Toolkit (FLTK) [89] and allows for the execution of all the major robot functions. 2.3 Operational setup In most integrated systems, the flow of data is rather complex, and the flawless combination of different devices requires fine installation. In our case, each component has its own way to communicate with the other elements, and to ensure the real-time cooperation of the system, the controller must have access to all (figure 2.12). To run the system, a certain procedure has to be followed. This contains several compulsory steps that must be executed prior to every operation, while other calibrations need only be performed when significant changes are made to the setup. These are described in the next section. Figure 2.12 Data flow of the integrated neurosurgery system [44]. The neurosurgical system uses several different coordinate systems, as every device has its own frame (figure 2.13). Homogenous coordinate transformations allow us to compute the position and orientation of an arbitrary point in any of the frames once the intermediate transformations are known. The purpose of the setup-registration is to determine every single transformation for smooth control of the system. To begin, the CT scan of the patient is acquired. We use 0.5 or 2 mm slice thickness. Naturally, this already influences the overall accuracy. The CT scans can be transformed to Slicer coordinates (Right Anterior Superior orientation - RAS), where they can be visualized. - 45 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery The virtual fixture for the procedure is created in Slicer. In the original setup, all control computations were made in the robot’s coordinate frame (Robot World - RW), this means that the VF definitions must be transformed into RW coordinates. This can be achieved through the chain of the following transformation: RW Slicer T= RW DRB DRB T ⋅ Stealth T ⋅ Stealth SlicerT (2.1) RW DRB T is the transformation between the NeuroMate’s base point (RW) and the marker, mounted on the patient, called the Dynamic Reference Base (DRB). This is computed through a registration procedure (see Section 2.3.2). The DRB Stealth T transformation is determined by registering the skull with a fiducial-based frameless method directly supported by the StealthStation software. Finally, the Stealth SlicerT is a fixed known transformation that connects the two different image coordinate conventions followed by the devices. Slicer follows RAS, while the CT scans are in the DICOM standard LPS orientation (Left Posterior Superior). The optical tracker provides information on the position of the mounted fiducials, on the Dynamic Reference Base (DRB) and Robot Rigid Body (RRB) with respect to the Localizer’s coordinate system (Loc). By composition, we can obtain the transformation from the DRB to the RRB: RRB DRB T= RRB Loc Loc T ⋅ DRB T (2.2) The transformation between the Robot TCP and the Robot World (TCP to RW) is obtained by computing the forward kinematics of the robot. We are able to read the high-precision encoders within the robot that give an exact value of the motor positions. Based on the encoder data and the known Denavit-Hartenberg parameters, we can compute the kinematic model. During the operation, the tool’s position is propagated back through the fixed cutterTip transformation (acquired by pivot calibration) and using the robot’s kinematics. This way, the controller can compare the VF and the tool position on the same coordinate system (RW) and determine the adequate control commands. Using the RW frame makes it easy to transform and send motion commands to the servos. On the other hand, this makes it harder to measure against the StealthStation’s readings. In the first experimental setup—described here—the navigation system is not required after registration, provided the RW frame is used consequently for control [90]. - 46 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 2.13 Coordinate frames of the JHU neurosurgery system In general, the use of the robot is better for positioning because of the higher accuracy, but we cannot track the robot’s motion relative to the patient (or to another device) in the OR. Thus, we need the navigation system for the purpose of increasing the safety; for example, to ensure that the virtual fixture computations take into account any motion of the patient relative to the robot (see Section 4.3). 2.3.1 Calibration procedures In our system, calibration refers to the estimation of the parameters of the inserted drill tool. The tooltip can vary in shape, length and size; therefore we conduct pivot calibration to determine the most important features. During pivot calibration, the end of the tool (a spherical shape tip) is guided into the same location in several robot configurations. The symmetry of our drilling tools (a near perfect sphere) makes it ideal for pivoting, resulting in a fast and easy procedure. One drawback of this calibration method is that we can only acquire the 3D position of the tool and not the orientation. We overcome this difficulty by deriving the full homogenous TCP RRB T transformation from another registration procedure (see Section 2.3.2). - 47 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Pivot calibration determines the location of the robot’s tooltip (i.e., the translations from TCP to CutterTip and from RRB to CutterTip in figure 2.13. In the basic case, the transformation between the robot’s base and the end (“pivotPoint”) is cutterTip TCP T ⋅ 50T = T(q)+ R(q)C = pivotPoint (2.3) where T is the transformation matrix’s position displacement, R is the rotation, C is the vector from the TCP to the tip of the drill and q is a vector of the five joint values (figure 2.14). We assume that the spherical tool at the end of the robot can perfectly fit in a cone milled in an aluminum plate; therefore the central point (pivotPoint) does not change. The tip is manually guided to the vicinity of the cone and then led by an automated forcebased method (ball-in-cone strategy [91]) to center the tip inside the cone. This is repeated for different orientations of the tool. We compute a least squares solution for the problem: cutterTip ( R )( -I ) = -T pivotPoint (2.4) where T and R are the (known) translation and rotation parts of the forward kinematic transformation, cutterTip is the (unknown) translation, pivotPoint is the (unknown) position of the pivot point and I is the identity matrix. Figure 2.14. Closed kinematic loop (pivot) calibration of the NeuroMate robot. To quantitatively measure the fit, we use the residual error, defined as: 2 N ∑ εres = X i - X est (2.5) N meas where Xi are the robot tip positions computed with the estimated cutterTip, Xest is the estimated pivotPoint and Nmeas is the number of measurements. Minimally six configurations are required to solve (2.4), however, further data can increase the effectiveness of the estimation. The typical - 48 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery residual error for our pivot calibration is 0.5 – 0.8 mm. Pivot calibration should be done whenever the tool or the end-effector had been changed and the resulting parameters must be stored for further use, both on the control PC and on the StealthStation controller. If the residual error is too high, εres > ThresCalibration the calibration procedure should be repeated. There are other methods for determining the desired tool parameters, based on visual identification of the tool, laser interferometry, laser triangulation touching reference parts using supersonic distance sensors, micro-switches or conduction fields, applying theodolites or simply measuring with calipers. Some provide far more accurate results than pivot calibration. However, our method is very convenient, as it does not require any additional hardware. A possible extension of the pivot calibration is to mount a laser pointer to the tip of the robot, and target it to a distant virtual pivot point. Using visual servoing, the robot can direct the laser to the desired point autonomously, while the distance amplifies the orientation deviations and provides a more accurate calibration [92]. A similar concept to pivot calibration is to use known features of the tool (e.g. a straight edge), and guide it to touch a distinct point with the two end of that edge. Doing it along at least two not co-planar axes we can compute the orientation of the tool. We consider implementing a similar calibration solution for our robot in the future. 2.3.2 Registration procedures Each element of the system uses its own frame, thus requiring the derivation of the appropriate coordinate transformations when integrated. To conduct a surgical procedure, the following steps must be done: • • • • acquire CT scan – create (load) VF in Slicer register CT to StealthStation – load CT data – register using fiducials on skull register Robot to StealthStation – record six positions in space get ready for cooperative control Figure 2.17 introduces the procedure in a flowchart. By downloading and processing the CT scan and the VF, we gain all the required information in the RW frame. One of the most critical and tedious steps is registering the scan loaded to the StealthStation to the real world coordinates, seen by the StealthStation cameras. This is achieved by the classic method of frameless stereotactic IGS: touching fiducial points (at least four) on the skull with a tracked pointer probe (figure 2.15). A special geometry rigid body serves as the pointing device and the - 49 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery registration method provided by StealthStation calculates the result of a paired-point registration. Paired-point registration finds a least squares solution to match two sets of points, each with the same corresponding arrangement [93]. Anatomical points can also be used if they are clearly identifiable both on the skull and on the pre-operative image. The average residual error of the registration is typically 0.5 – 0.9 mm. However, it sometimes takes an extended period of time to acquire a good registration, as the procedure should be repeated if the error is bigger than ThresRegistr. The lack of automated fiducial identification and localization on the StealthStation also reduces the accuracy. Figure 2.15 Figure 2.16 Frameless registration procedure to connect the StealthStation and the physical world coordinates with the help of a hand-held probe. Performing phantom test (foam block cutting) with the robot. The NeuroMate is registered to the localizer (Robot2StealthStation, DRB to RW) by recording at least six robot positions measured by the navigation system (tracking the RRB) and the robot controller (reading the internal encoders). The changes in RRB position (as seen by the StealthStation) are equivalent to the motion of the end-effector (as a result of the joint’s motion); therefore we can compute the RW DRB T transformation. It is important to exercise the robot around all configurations of the surgical procedure to get a valid mapping for the workspace that will be used. (We can get a very low residual error by slightly moving the robot, but it will not be valid for the entire target area.) The typical residual error of the registration is between 0.2 – 0.7 mm. Once registered, the system is ready to work in cooperative control mode (figure 2.16). - 50 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 2.17 Pre-operational setup procedure (calibration and registration) of the JHU neurosurgery system. 2.4 Operation in cooperative control mode In our setup, the NeuroMate robot is able to run in cooperative control mode, where the readings of the force sensor are used to control the manipulator’s motion (hands-on surgery). Depending on the orientation of the force applied by the surgeon, the robot moves in the - 51 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery prescribed direction with a velocity proportional to the force. For convenience, we can choose to implement only translational or rotational motion of the tooltip. The lower lever controller program on the PC (neuromateTask) communicates with the embedded processors in the robot over the CAN bus in every 18.2 ms. The controlTask can run at any cycle speed between 18.2 ms and the period time of the mainTask; we ran it at 20 – 30 ms. The maximum allowed linear velocity of the robot is 25 mm/s. While in compliance mode, the robot uses the following admittance control law: F q = J -1(q) ⋅ K(d) ⋅ G ⋅ w Tw (2.6) where q is the joint vector, J is the Jacobian matrix resolved at the tooltip, K(d) is a diagonal matrix of scale factors, G is a diagonal matrix of admittance gains, Fw, is the measured force and Tw is the vector of the torques. Figure 2.18 The GUI displays the most important tool information during compliant mode. Figure 2.19 The pendant of the robot with the emergency stop button. The GUI serves as the ultimate control panel for the robot. In addition to displaying the basic information of the robot, it offers the possibility of switching between different guidance modes, force/position control, performing calibration/registration, or saving the motion sequences of the robot. These functions are all available via the user interface (figure 2.18). Security is considered a serious issue in our system. There is a stop button on the GUI plus a separate emergency stop button (on the pendant) mounted on the base of the robot to help preventing any accidents (figure 2.19). In the future, further software and hardware elements will be added for maximum patient and surgeon safety. To improve the quality and precision of the operation we use a virtual fixture. As described in the previous section, the boundary of this 3D volume is created in 3D Slicer, and once registered - 52 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery to the robot, it enforces the tool to stay within the predefined area. Within the proximity of any of the planes of the virtual fixture, the robot controller rescales the corresponding component of the motion through the K(d) matrix in (2.6), resulting in a proportional reduction in velocity near the boundary. If the surgeon forcibly pushes the robot towards the VF, the robot still slows down and stops at the surface. The rigidity of the manipulator prevents major overcut, although it is still possible to force the robot’s tip past the VF by approximately 1 mm due to the structural compliance of the mechatronic system. In the current implementation [90], we compute the perpendicular distance of the tool tip to the closest VF plane in each of the Robot World frame coordinate directions ( i = X, Y, Z). di = ( C - P )× N (2.7) Ni where C is the robot’s Cartesian position vector, N is the normal to the VF plane, P is the closest point on the plane and di is the distance to P. The velocity commands sent to the robot are scaled based on di. If the tooltip is within the proximity of the VF (i.e., closer then a preset distance D) the following K(d) scaling factor is used to reduce the robot’s velocity: di (2.8) D While di is positive, the robot is in the free space (allowable region) and D is the boundary K(d) = zone’s width around the VF, where the robot velocities are scaled (Section 2.2.3). A negative value means that it has passed into the forbidden zone. In the latter case, we only allow motion back towards the safe area. This implementation gives a fine solution preventing overcut, providing an unambiguous solution for any geometry of the VF. However, it does not always allow the option of sliding along the VF surface because of the Cartesian-aligned calculations. For some of the experimental tests, we implemented a more ergonomic version of the VF control that is based on the following control rule: V j = V - f j (V × N j )× N j (2.9) where V is the robot’s original velocity in RW, Vj is the new velocity, Nj is the normal vector to the VF plane and fj is the scale factor for the jth plane: f j = 1- d D (2.10) where d is the distance from the VF plane and D is the boundary zone. If the V velocity has any component towards the VF plane (i.e., V × N i < 0 ) and the tip is within the D boundary, the velocity component towards the VF will be rescaled according to (2.9). This method facilitates sliding along the VF plane and eliminates the perpendicular component of the motion in the most extreme case (i.e., on the VF surface). However, complications arise in the computation - 53 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery within the proximity of multiple VF planes that are not perpendicular. We only used this method for foam block cutting with a rectangular VF. Currently, we are working on a new implementation of the VF that is based on a constrained optimization method [94] and [95]. This will allow for the computation of a one-step optimized motion-control, considering all the VF planes, the joint speed limitations and singular configurations of the robot—resulting in a smoother motion control of the NeuroMate. 2.5 Results of phantom and cadaver experiments We performed preliminary phantom and cadaver experiments to measure the efficacy and performance of the system. In the phantom experiments, we defined a box-like virtual fixture corresponding to an interchangeable foam block installed inside a plastic skull, as described in [82]. Prior to the tests, 2 mm slice spacing CT scans were taken of the skull with mounted adhesive fiducials (figure 2.21). We performed the registrations described in Section 2.3.2 and set up the system for mock operation (figure 2.20). Figure 2.20 Figure 2.21 Figure 2.22 Experimental setup for phantom tests. The CT scan of the phantom with the inserted foam block [44]. Foam blocks after cutting with a cubic VF. After cutting 12 foam blocks in cooperative control mode, we used calipers to measure the actual size of the cavity and compared it to the desired shape (size of the virtual fixture). We separated the error into a placement error (the difference in centroid locations between desired and actual cavities) and a dimensional error (the deviation in dimensions), which were 0.6 ± 0.8 mm (mean ± standard deviation) and 0.6 ± 0.3 mm, respectively. The different cavities milled in the foam blocks are shown on figure 2.22. Cadaver tests were performed at the Johns Hopkins Medical Institute to verify the system with a more complex VF and gain insight to the emerging difficulties of a more realistic setup. A - 54 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Hopkins neurosurgery resident performed the cuttings and provided valuable feedback on the system for further development. An appropriate VF was created for the resection of a hypothetical acoustic neuroma via a sub-occipital approach. This is a typical skull base operation that involves the cutting of around 0.2 – 1 cm3 of bone. We used a surgical endoscope camera and lights to aid the surgeon during the procedure (figure 2.23). Clinical accuracy of the experiment was determined by manually aligning the post- and preoperative CT scans. Overlaying the VF on the post-operative images, we saw typical overcuts of about 1 mm, with a maximum overcut of about 2.5 mm (figure 2.24). One interesting problem is to find an adequate 1D measure for the accuracy of the 3D drilling. Undercuts (unremoved bone within the operating area) are usually not a problem, as long as the surgeon can reach the desired anatomical region through the hole. However, overcuts can be dangerous, risking the critical body structures of the patient. An appropriate method to measure the quality of the cut is to create a volumetric model based on both of the pre- and post-operative CT scans, and define the exact amount of missing bone in the area of interest. We plan to use a simplified method to determine the under/overcuts (i.e., the error of the cut) based on a volumetric analysis: cuterror = VF ∩ Vcut (2.11) where VF is the volume of the virtual fixture and Vcut is the actual cut. This way we can calculate for a numeric result of the operation’s accuracy. Phantom and cadaver tests are essentials in the process of development to validate any surgical system for further studies and finally for clinical use. However, many times, the accuracy numbers do not only show the performance of the devices, but also certain disturbing effects of the operating environment. Either way, these experiments give great help to further investigate the sources of errors, and to better understand the needs of the surgeons. Figure 2.23 Figure 2.24 Experimental setup for cadaver tests. Pre- and post-operative CT scans of the cadaver, showing the virtual fixture and the error of the cut [44]. - 55 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Chapter 3 Identification and measurements of the system 3.1 Identifying the sources of errors My first task at the SMARTS lab was to help run the experiments described in Section 2.5 and investigate ways for further improvement. The overall goal of our neurosurgery project is to improve the application accuracy to sub-millimeter scale and eliminate extreme deviations throughout the procedure. In general image guided surgery, 3 – 5 mm accuracy is considered acceptable, whereas 2 mm is more recommended for IG neurosurgery. There are three different types of accuracies [12] that must be addressed differently: • Intrinsic (technical) accuracy (typically 0.1 – 0.6 mm ) • Registration accuracy (typ. 0.2 – 3 mm ) • Application accuracy (typ. 0.6 – 10 mm ) The intrinsic accuracy applies to certain elements of the system, such as the robot and the localizer. It describes the average error of the component in operational use (i.e., in our case the positioning or localization error). Mechanical compliance, loose hardware elements, resolution of the imaging device, inadequate control and noise can all result in low intrinsic accuracy. Further, all registration methods involve some kind of error, as we can only compute a least squares solution for our mathematical fitting problem. The main sources of errors are the markers (different types, forms and materials), displacement of the fiducials and determination of the center of the fiducials. We may investigate the error-dependency on the application environment (registration performed in the laboratory vs. in the OR). Application accuracy refers to the overall targeting error of the integrated system while used in a clinical or clinical-like setup. This factor characterizes most realistically the effectiveness and is used for the validation of a system. The application accuracy is dependent on all other sources of errors, but it is not simply their mathematical sum. To derive the application accuracy of a system, phantom, cadaver and clinical tests have to be performed. - 56 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery In our neurosurgical setup, the main sources of errors were identified in accordance with the previously introduced categories: 1. Individual system components’ technical error a) CT scanner b) NeuroMate robot c) StealthStation navigation system d) virtual fixture definition e) tool bending 2. Registration error a) equipment b) registration methods 3. Changing operating environment a) skull motion during the procedure (relative to the robot) b) reference frame motion during the procedure (relative to the skull) c) physiological changes Inaccuracy types 1 d) and e) can only cause dimensional error, while the others from category 1 and 2 can introduce both dimensional and placement error. Type 3 mainly causes placement error, but the cumulative effect of small placement errors would appear as dimensional. After analyzing the results of previous phantom and cadaver tests, I could determine which areas can be improved most effectively. 1 b) c) d) and 3 a) were chosen for further investigation, as discussed later in Section 4.1. It is unquestionable that 1 a)—the CT scanner— fundamentally determines the precision of the system. However, we do not have the means to change the registration and modeling algorithms within the scanner, nor to interfere with the device in other ways. To achieve the best possible result, we tried to acquire the finest available, 0.5 mm slicing scan of the phantom and the cadaver head. Patient motion in the scanner can further decrease the quality of the image in human application. To reduce the effect of tool bending, 1 e), a rigid metallic bracket was added to the endeffector. Tool bending is not a major safety issue while performing drilling since the bending would only result in undercuts that pose no additional health-risk to the patient. Concerning error group 2, no changes were done to the registration procedures (see Section 2.3.1). Although it can be a major source of error, it was not considered for further modifications. Currently, the residual error is computed based on (2.5) after every registration and the procedure can be repeated if the error is high. In the mean time, the condition of the optical markers is maintained by changing the reflective balls periodically and replacing bent pointing needles. Replacing the entire registration hardware might be too expensive. The results of another major task—investigating and implementing a method for patient motion, 3 a)—are discussed in Chapter 4. Unfortunately, as we use an optical tracker, there are no means to monitor errors originating from case 3 b), when the rigid body is loose on the - 57 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery patient. There are other methods, based on different imaging modalities (some developed at the ERC CISST [96]) that may help in this issue, but they all require additional hardware elements. Clinical solutions include markers screwed directly to the skull or to the teeth to minimize the chance of motion. If the surgeon notices deviation during the application (i.e., there is an offset between the visualization and the device, or the VF is not at the right place), he or she should reregister the system immediately. We only intend to perform bone drilling, so we do not expect any intra-operative tissue shift (case 3 c). However, in other applications, involving the manipulation of the grey material, brain motion and compliance is a major source of error [97]. This is a current and important field of research, and many promising solutions are under development with the promise of compensating for soft tissue movement within 1 mm of error [98]. 3.2 Intrinsic accuracy of the NeuroMate robot The next step was to determine the intrinsic accuracy of the NeuroMate. Generally the absolute positioning accuracy and the repeatability are given for a manipulator to characterize the overall effect of the precision of the encoders, the compliance of the hardware elements, the servos and the rigidity of the structure. The intrinsic accuracy was reported to be 0.75 mm with a repeatability of 0.15 mm [33]. I performed accuracy tests on two metallic phantom plates. The first aluminum board contains thirteen conical divots at different positions and heights. These were created on a Computer Numerical Control (CNC) machine in a known arrangement with an accuracy of 0.0127 mm (previously used for the validation of another project on small animal hypoxia measurement [99]). The accuracy can be measured by locating the holes with the robot tooltip and comparing the positions measured by the robot to the known CNC. The robot was guided in cooperative control mode to the pivot holes until the 5 mm diameter tooltip fit perfectly. The robot’s joint encoders were recorded in this position. To evaluate the test, I computed the Fiducial Registration Error (FRE) and Target Registration Error (TRE) (figure 3.1) using a program created by Dr. Kazanzides. FRE is the residual error (2.5) of the paired-point registration between the given subset of the known and recorded fiducial coordinates [93]. 1 N 2 (3.1) T(pi ) - qi ∑ N i where N is the number of fiducials used during the registration and qi is the position of the ith FRE 2 = fiducial in one space (e.g. the robot), pi is the same in the other (e.g. CNC) space and T is the - 58 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery computed homogenous transformation connecting the two spaces [100]. In the ideal case FRE = 0. TRE is typically used for the characterization of schematic point-based registrations. TRE is the error of locating markers (that were not used for registration) in robot coordinates (figure 3.1). Four independent accuracy measurements were performed; a given subset of the points was used for registration and the rest were used for TRE computation. The average FRE was 0.093 ± 0.041 mm and the TRE was 0.435 ± 0.223 mm. Figure 3.1 Definition of FRE and TRE. The black and white circles represent corresponding point pairs in the two different spaces. FRE is the residual error of the points used to derive the T transformation, while TRE is the mapping error of an independent point [100]. To acquire more data on the intrinsic accuracy of the NeuroMate system, I performed several accuracy tests on another measurement object (phantom) that was developed at the University of Nebraska to support a draft American Society for Testing and Materials (ASTM) standard [101]. The phantom contains 47 CNC milled cone-shaped holes arranged in 3D space (figure 3.2). Their exact positions were obtained previously through a Coordinate Measuring Machine (CMM). I accomplished five complete tests—similar to the previous case—and the collected data was evaluated again based on the paired-point registration method (figure 3.3). Every other measured points were used in the registration, resulting in an average FRE of 0.361 mm; the remaining points were used to compute the TRE. The results showed that the intrinsic accuracy of the robot is 0.335 ± 0.168 mm (TRE). During the same experiment I measured the noise of the encoder readings (quantization noise), which turned out to be an insignificant 0.001 mm in average. - 59 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 3.2 Figure 3.3 The black CNC machined phantom from the University of Nebraska for the accuracy test. The NeuroMate performing the accuracy test. The tests showed that the NeuroMate’s intrinsic accuracy is already fairly good, while there are still opportunities to improve it through parameter calibration. The repeated measurements of FRE gave indirectly information of the repeatability of the robot that showed to be an order lower than the kinematic error; therefore we did not investigated it further. 3.3 Deriving the kinematic model of the NeuroMate robot Exact measurements had to be taken to investigate appropriate solutions to reduce the technical error of the selected components. It was hypothesized that the NeuroMate intrinsic accuracy could be improved by performing a kinematic calibration. This required a kinematic model of the NeuroMate. Figure 3.4 Frame transformations to derive the Denavit-Hartenberg parameters of the robot [102]. - 60 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery First, the Denavit-Hartenberg (DH) parameters of the NeuroMate had to be re-calculated. The kinematic model of the robot had been previously derived and rough calibration measures were also taken, but no appropriate documentation was available. To be able to develop an augmented and accurate calibration for the NeuroMate, I had to re-compute the DH parameters. Figure 3.4 shows the transformation between two links according to the most common DH convention [102], [103] and [104]. For every robot link the DH parameters determine a rotation around the Z axis ( ϑi ), a translation along the Z axis (di), a translation along the X axis (ai) and a final rotation around the X axis ( αi ) that leads from the coordinate frame of the i-1th link to the ith Cϑi Sϑi i i-1T = Rot(z,ϑi ) ⋅ Trans(z,d i ) ⋅ Trans(x,ai ) ⋅ Rot(x,αi ) = 0 0 -Sϑi Cαi Sϑi Sαi Cϑi Cαi -Cϑi Sαi Sαi Cαi 0 0 aiCϑi ai Sϑi di 1 (3.2) where Sϑi and Cϑi stand for sin( ϑi ) and cos( ϑi ), respectively. Figure 3.5 and 3.6 show the frames assigned to the robot in q = [0 0 0 0 0]’ and in an arbitrary joint configuration. The DH parameters of the system are in table 3.1. Figure 3.5-6 Coordinate frames of the NeuroMate in q = [0 0 0 0 0]’ and an arbitrary configuration. The DH parameters d1 and a1 can be arbitrarily chosen. Thus we decided to leave the BASE frame at the origin of the first frame; in other words the BASE equals the Robot World. The origin of the frame was chosen to be outside of the robot’s physical boundaries of the robot to reduce the calculations by one link-length parameter. Joints q1-5 belong to the corresponding ϑ rotation of the previous links: qi = ϑi −1 . The last link does not belong to the robot, it is a noncanonic DH representation of the tool. There is no degree of freedom modeled in the tool; the d6 and a6 parameters give a displacement along the X and Z axes, but an additional 90˚ rotation - 61 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery term had to be introduced to align the last frame with the previous without breaking the DH convention. This formulation of the coordinate frame cuterTip BASE T allows for a single step calibration of the DH parameters. The forward kinematics of the robot can be now determined using the DH parameters. (3.3) and (3.4) show the rotation and translation part of the homogenous transformation cuterTip RW T separately. C12 ⋅ C345 -C12 ⋅ S345 S12 T= T⋅ , Rot = S345 C345 0 (3.3) -S12 ⋅ C345 S12 ⋅ S345 C12 -C12 ⋅ (S34 ⋅ a5 + S3 ⋅ a4 ) + S1 ⋅ a2 + (-C12 ) ⋅ S345 ⋅ a6 + S12 ⋅ d 6 Transl = C34 ⋅ a5 + C3 ⋅ a4 + C345 ⋅ a6 (3.4) S12 ⋅ (S34 ⋅ a5 + S3 ⋅ a4 ) + C1 ⋅ a2 + S12 ⋅ S345 ⋅ a6 + C12 ⋅ d 6 cuterTip RW cuterTip 5 Transl3x1 Rot T = 3x3 1 01x3 5 RW During the production of the NeuroMate robot (in Switzerland), link-lengths are measured and factory values are provided with the system (table 3.2). However, my experiments showed that the given numbers were not exactly valid for this particular robot (possibly due to documentation error); therefore I implemented a calibration method for a better estimation of the DH parameters (see Section 4.2.1). i \ DH 0 1 2 3 4 5 6 ϑi ϑ1 -π/2 +ϑ2 π/2 +ϑ3 π/2 +ϑ4 -π/2 +ϑ5 0 -π/2 Table 3.1 Table 3.2 di d1 0 0 0 0 d6 0 ai a1 a2 0 a4 a5 a6 0 αi -π/2 0 π/2 0 0 0 0 Factory values for link-lengths a2 = 125.350 mm a4 = 355.566 mm a5 = 349.690 mm DH parameters of the NeuroMate together with the surgical tool. Link-length parameters provided by the manufacturer. 3.4 Intrinsic accuracy of the StealthStation I used the same University of Nebraska phantom to test the accuracy of the StealthStation. Measurements were performed both by tracking the Robot Rigid Body (RRB) and the hand-held pointer probe (used otherwise for CT registration). I conducted five experiments with the RRB and three measurements with the pointer probe. In the first case, the results showed that the navigation system had 0.494 mm Fiducial Registration Error and 0.489 ± 0.221 mm Target Registration Error. In the second case, and TRE was 0.513 ± 0.423 mm (FRE: 0.515 mm). To - 62 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery verify the numbers, I mounted the pointer probe on a three axis linear robot (New England Affiliated Technologies, Lawrence, MA, USA)—designed for high precision applications—and recorded the position readings of the localizer while moving the robot along the straight axes (figure 3.7). The measurements supported the previous results; the StealthStation’s distance measurement error was between 0.105 – 0.788 mm. Figure 3.7 Figure 3.8 A Cartesian robot with the pointer probe mounted for the verification of the StealthStation. The noise of the StealthStation (blue) and the average filtered values (averaging over 1 s). The StealthStation’s position information acquired through the StealthLink interface is significantly delayed with respect to the robot motion. Measurements were taken by directing the robot to move in a periodic motion pattern, while recording the encoder values and values of the optical tracker at the same timeframe. The results showed that it is possible get a new position update from the StealthStation in 149 ± 35 ms, but due to the communication network, there is a mean latency of 247 ms compared to the robot encoder readings. Another issue is the significant noise present in the StealthStation’s position information, originating from the imperfection of the markers’ segmentation and spatial identification. I conducted further experiments to identify the noise and prepare an adequate filter. To reduce the noise in the experiment described above, I used the averaging filter previously implemented with the robot controller. It averages the position information over 1 s. The smoothening effect can be seen in figure 3.8, where the blue dots show the raw position readings and the red dots show the filtered ones. StealthLink data was recorded over 450 seconds to measure the noise. The average standard deviation of the measurements, i.e., the noise value in the present setup of the localizer is 0.2602, 0.2266, 0.2452 mm in X, Y, Z directions respectively, with approximately normal Gaussian distribution. - 63 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery 3.5 Virtual fixture definition Virtual fixtures are defined around the specific region of the skull base for safety and increased accuracy. We use a simplified VF model by creating a six-sided convex hull based on the CT scan, with one or two sides left open to enable cutter entry (figure 3.9 and 3.10). In some of the clinical cases (such as the sub-occipital approach for acoustic neuroma resection) and for the phantom tests, this method is precise enough. In the future, we plan to use a more complex representation that may serve for other clinical procedures as well. Figure 3.9 Figure 3.10 A virtual fixture defined in Slicer for acoustic neuroma resection procedure [44]. A virtual fixture on the pre-operative CT scan (A) and on the post-operative 3D model of the patient (B) [44]. From the control point of view, the boundary zone surrounding the VF is important, as the relative motion of the robot must be scaled while the tool is close and moving towards the VF. To determine the size of boundary, I simulated the robot’s motion in the proximity of the VF in MATLAB. To choose an adequate width for the boundary, we calculate with the maximum speed of the robot (perpendicular to the VF) and the lowest execution rate of the ControlTask. The software should be able to stop the moving robot and scale its motion under these extreme assumptions. The maximum linear velocity at the end-effector is approximately 50 mm/s and the largest sampling time is around 100 ms. Figure 3.11 shows the simulated distance of the tool (di) and the linear velocity of the tool vs. time. The simulated motion profile showed that even in the most extreme case, a 5 mm wide boundary zone is enough to prevent the tool from entering the VF. - 64 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Motion profile in the VF 5 4 4 Distance (mm) Distance (mm) Motion profile in the VF 5 3 2 Position in VF VF Boundary 1 0 0 50 100 150 200 250 Time (ms) Speed profile in the proximity of the VF 300 350 Position in VF VF Boundary 1 0 100 200 300 400 500 600 Time (ms) Speed profile in the proximity of the VF 0 100 200 700 800 700 800 20 40 Speed (mm/s) Speed (mm/s) 2 0 50 30 20 10 0 3 0 50 Figure 3.11 Figure 3.12 100 150 200 Time (ms) 250 300 350 15 10 5 0 300 400 500 Time (ms) 600 Motion profile in the proximity of the VF with the most extreme case. The scaling of the robot’s motion heading towards the VF in a realistic scenario. Moreover, in an actual surgical configuration of the robot, the maximum linear velocity is always under 25 mm/s, typically a mere 1 – 1.5 mm/s; besides, we use 20 - 30 ms cycle time (and minimum of 18.2 ms is achievable). In a more realistic scenario, the motion profile would be a smooth curve safely keeping the robot out of the VF, as seen in figure 3.12. As the tool gradually approaches the VF plane with decreasing speed, the operator has enough time to notice the proximity of the VF. In all of our tests, we used the safe 5 mm boundary width. - 65 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Chapter 4 Increasing the accuracy and safety 4.1 General approach The results of the experiments presented in the previous chapter motivate our investigation for methods to improve the overall accuracy of the neurosurgical system. Achieving an acceptably low error margin is a crucial step in the research towards the development of a clinically deployable system. The application accuracy depends on the precision of the individual components and the integration of the system, as discussed earlier in detail. Although the error margin derived from our preliminary phantom and cadaver experiments is sufficiently low for certain domains, the experiments described in Chapter 3 elucidated the need for further improvement of the system. My task was both to focus on the hardware instruments and on the software architecture; particularly looking at three areas for improvement: 1) improving the accuracy of the NeuroMate 2) increasing the precision of the StealthStation 3) compensating for motions in the OR. 4.2 Improving the components’ accuracy 4.2.1 Extended robot calibration To improve the precision of the NeuroMate manipulator, I extended the already implemented pivot calibration to a single-loop closed kinematic chain model [105] to identify the joint offsets within the same procedure. We anticipated that the major source of inaccuracy of the robot’s positioning is due to the offset of the potentiometers in the links. The great advantage of pivot calibration is that simply by collecting more data in different configurations with the same setup, further estimations can be made. Similar to the original computation (2.3), we acquired the least squares solution of the following equation ∂(T + RC) ∂(T + RC) T + RC + dq + dC = pivotPoint ∂q ∂C - 66 - (4.1) Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery where q is the (known) joint position vector, dq is the (unknown) joint offset vector, T and R are the (known) translation and rotation parts of the forward kinematic transformation of the NeuroMate, C is the (known) nominal cutterTip offset in RW coordinates and the (unknown) ∂(T + RC) ∂(T + RC) pivotPoint. The final estimation of the tool tip is C+dC. The and terms ∂q ∂C give the forward kinematics dependency on the small changes of q and the cutterTip coordinates, respectively. To ensure quick convergence the estimation is based on a rough estimate of C, provided by the original implementation of the pivot calibration (2.3). I computed the partial derivatives using the MATLAB Symbolic Toolbox and then manually applied the first order Taylor series expansion to the sine and cosine terms. Thus I gained the locally linearized form of the partial derivatives and I assumed that the higher-order terms were insignificant. The LS estimation ran iteratively in the following form: dq ∂(T + RC) = −T − RC dC (4.2) ( R )( − I ) ∂ q pivotPoint ∂(T + RC) where I stands for the identity matrix and = R , as the translational part does not ∂C depend on the cutterTip parameters. The estimated dq and dC parameters are added to the original measurements of q and C, while the pivotPoint is recalculated in every cycle. With this method we can theoretically calibrate for joints q2 – q5. It is not feasible to estimate dq1 because its value represents a rotation of the entire Robot World coordinate system. Difficulties arose with dq5 because in the presented formulation the dCy parameter is redundant with respect to dq5. We solved the issue by changing the optimization formulation from C x Cy Cz to [ a ϑ ψ ] , where Cx a cos(ϑ ) C = cutterTip = a sin(ϑ ) y Cz ψ (4.3) a = C y /sin( ϑ ) , ϑ = atan(C y /C x ) , ψ = C z (4.4) In the above formulation ϑ = dq5 . By changing the equation, the partial derivative had also been changed to R11cos( ϑ ) + R12 sin( ϑ ) a( - R11sin( ϑ ) + R12 cos( ϑ )) R13 ∂RC = R21cos( ϑ ) + R22sin( ϑ ) a( - R21sin( ϑ ) + R22 cos( ϑ )) R23 ∂a∂ϑ∂d R31cos( ϑ ) + R32sin( ϑ ) a( - R31sin( ϑ ) + R32 cos( ϑ )) R33 - 67 - (4.5) Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery A relatively large number of robot configurations (12 – 20) were collected during the pivoting and used for the calibration. Finding a LS solution to a bigger dataset that covers the wider range of the joint values ensures that the calculated solutions are valid for the entire joint space of the robot. Initial experiments showed that the calibration reduced the residual error by half; however, the results were inconsistent, suggesting that there are other major sources of errors in the system. To get better results, I further extended the closed-looped kinematic calibration, including other Denavit-Hartenberg (DH) parameters of the robot. In the case of the NeuroMate, the relevant DH parameters are the three link-lengths (see Section 3.3). The new solution included the link-lengths and used the factory values as a initial estimate (table 3.2): dq ∂ (T + RC ) ∂RC ∂T dC = −T − RC ( − I ) dL ∂q ∂C ∂L pivotPoint S1 ∂(T) = 0 ∂L C1 -C12 S3 C3 S12 S3 -C12 S34 C34 S12 S34 a2 125.350 + dL2 a = L = 355.566 + dL 4 4 a5 349.690 + dL5 (4.7) (4.6) (4.8) This method failed to produce acceptable results, as the one-step LS estimation could not solve ∂ (T + RC ) for the cross-dependencies; i.e. the contributions of the separately computed ⋅ dq ∂q ∂T and ⋅ dL terms to the final solution were competing during the iteration cycles. We decided ∂L to move on and use a new formulation. I described the tool as the 6th link of the robot and acquired the corresponding DH parameters (see Section 3.3). I set up the new equation for the same closed-loop kinematic parameter optimization, again with first-order Taylor series expansion to linearize the system: ∂(Text ) ∂(Text ) dq = −Text + pivotPoint ∂q ∂L dL where Text refers to the extended (4.9) cuterTip RW T homogenous transformation of the robot and the tool. As we did not introduce a new joint by adding the 6th link of the tool, the rotation matrix does not change in the kinematic model. (4.9) should lead to the same results than: - 68 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery dq ∂(T + RC) ∂T dL = −T − RC ∂L ( − I ) ∂q pivotPoint (4.10) Equation (4.9) was used for estimating dq2 – dq5 joint offsets, a2, a4, a5 link-lengths, a6, d6 tool parameters and the pivotPoint (X, Y, Z). Given a preliminary estimation from the original pivot calibration and solving (4.10) iteratively, I could acquire the joint offsets and the final estimation of the pivot calibration results as well. Figure 4.1 Convergence of the estimated parameters during the extended calibration procedure. The method converged fairly quickly, reaching the final solution (where dq and dL are zero) in 8 – 10 iterations (figure 4.1). We could directly measure the performance of the calibration by comparing the residual error for the original data set with that of the new parameters. On average, the residual error decreased from 0.367 to 0.278, by 24%. Parameter dq1 dq2 dq3 dq4 dq5 a2 a4 a5 a6 (cutterTip (X)) d6 (cutterTip (Z)) Original values After calibration Impl. increment -0.2* -1.3* -0.1* -0.277* 0* 125.350** 355.566** 349.690** 109.1374 *** -154.6592 *** 1.024 -0.433 -0.22 1.126 125.501 355.6698 349.0183 108.5874 -154.4022 0 0.08 -0.333 0 1.126 0.151 0.1038 -0.6717 depends on tool depends on tool * : From carpenter level calibration (in degrees) **: Factory values (in mm) ***: From initial pivot calibration, example values (in mm) Table 4.1 Results from closed kinematic loop calibration for robot parameters. - 69 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Based on the calibration with five independent data sets, containing altogether more than 60 configurations, the calibration values derived are summarized in table 4.1. “Original values” show the results of the prior carpenter level based calibration, the third column displays the averaged results derived from the different datasets and the last column contains the implemented values. As explained before, due to the nature of the calibration, I was not able to obtain a better measurement for the first joint. Moreover, dq2 and dq4 were bouncing too much (supposedly due to some unmodeled sources of error), therefore a manual trial-and-error based optimization was performed to acquire their final values. The joint offsets were set to the new values in the lower level of the controller and the link-lengths were adjusted in the forward kinematic solution function. After all, the robot will provide better positioning accuracy and it is possible to perform a single pivot calibration upon tool change with the increased precision. 4.2.2 Improving the StealthStation’s accuracy In the experiments described in Section 3.4, the average absolute noise of the localizer was found to be 0.203 ± 0.131 mm with normal distribution. (In a previous setup we measured significantly higher values.) To reduce the effect of the noise, we wanted to develop an appropriate filter. A moving average filter had been implemented previously (figure 3.8), but it is only works well when the robot is not moving. The filter provides the average value of the position readings from StealthLink collected during 1 s. It is useful in improving the quality of certain registration procedures involving the StealthStation, but incurs too much latency to be used during the operation. 4.2.2.1 Linear regression The next idea was to develop a linear regression, capable of approximating first order linear motions of the form y = mx + b + e (4.11) where y is the dependent variable, m is the slope of the curve, b is the intercept and e is the added noise. Linear regression performs a straight line fit (with m and b parameters) given the x dataset and can compensate for noisy inputs. In every motion cycle, the regression line gives a good match to a set of data points given in a moving window, using the least squares criterion to select the best fit line. I implemented linear regression based filtering for the StealthStation readings. This method uses two separate 2D linear regressions in the XZ and YZ planes, then finds the 3D solution by projecting the lines on each other. The linear regression keeps cumulating the - 70 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery following sums during the computations: sxy = ∑ ( x - xˆ )( y - yˆ ) , sxx = ∑ ( x - xˆ ) , s yy = ∑ ( y - yˆ ) 2 2 (4.12) where x̂ and ŷ stands for the average values through the recorded timeframe, consisting of N samples. The parameters of the linear regression can now be computed: sxx , Intercept = b = yˆ - mxˆ (4.13) sxy In our application, we used a timeframe of 1 s that contains approximately 7 position Slope = m = measurements and the StealthStation provides a reading in roughly every 150 ms. The effectiveness of the filter greatly depends on the number of elements used for the regression (the errors are reduced to 1/Nth); however, using more elements for the computations contributes to a larger latency of the filter. To avoid the need for cumulatively calculating average values, I implemented an iterative version that instead requires the continuous calculation of the following sums: ∑x = ∑ x , ∑ y = ∑ y , ∑ xy = ∑ x ⋅ y , The regression parameters are derived as Slope = m = N ⋅ ∑ xy - ∑ x ⋅ ∑ y N ⋅ ∑ xx - ∑ x ⋅ ∑ x Intercept = b = , ∑ xx = ∑ x ⋅ x , ∑ xx ⋅ ∑ y - ∑ x ⋅ ∑ xy N ⋅ ∑ xx - ∑ x ⋅ ∑ x (4.14) (4.15) For verification of the results, the standard deviation and slope error were calculated in every cycle: stdDev = S yy - b ⋅ S xy N -2 , Error = stdDev S xx (4.16) The major limitation of the linear regression is that it can only fit straight line motions and it fails when there is no motion. As a result, additional latency and overshoot can be expected when used. 0.15 0.1 0.05 Z 0 -0.05 -0.1 -0.15 -0.2 9 8.5 8 7.5 7 X Figure 4.2 Figure 4.3 -700 -750 -800 -850 -900 Y The linear filter implemented for noise filtering in a simulation test. Block diagram of a Kalman Filter showing all the variables involved. (Figure: Courtesy of Paul Crowley) - 71 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery 4.2.2.2 Introduction to Kalman Filters We moved towards the more promising solution with a Kalman Filter, which recursively estimates the state of a dynamic system from a series of noisy measurements. It was originally developed by Rudolf Kalman in 1960 and has been widely used in applications ranging from economics to aeronautics [106]. The requirement of the method is that the noise model of the system must be Gaussian. Measurements described in Section 3.4 proved that the StealthStation’s noise has a normal distribution. The Kalman Filter recursively gives optimal estimation of the x state vector based on the covariance matrix of the mean square errors of the measurements [107]. To acquire the next estimation, only the previous time step and the current measurement is required. Figure 4.4 shows the block diagram of Kalman filtering. The Kalman Filter can estimate for the discrete linear system xk = Axk -1 + Buk + wk -1 (4.17) zk = Hxk + vk (4.18) with x state vector, u input and z output (that we can measure). w is the process noise with Q covariance and v is the measurement noise with P covariance, both with normal distribution: wk ∼ N(0,Qk ) , vk ∼ N(0,Rk ) (4.19) A (transition matrix, sometimes F or Phi), B (observation matrix) and H (measurement matrix) follow the classical system equation notation. Initial estimations for xˆ k k -1 and Pk k -1 are gained from a priori information. Figure 4.4 Basic work frame of an optimal Kalman Filter with simplified notation [108]. - 72 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery The Kalman Filter has two distinct phases: prediction and update [108]. The prediction phase uses the state estimate from the previous iteration step to produce an estimate of the state at the current timestep xˆk k -1 = Ak xˆk -1 k -1 + Bk uk -1 (4.20) The prediction estimate covariance matrix is also projected ahead Pk k -1 = Ak Pk -1 k -1 Ak T + Qk -1 (4.21) In the update phase, or correction phase, the optimal Kalman gain (K or b) is computed, based on the measurement information at the current timestep Sk = H k Pk k -1 H k T + Rk , K k = Pk k -1 H k T Sk -1 (4.22) where Sk is the innovation (or residual) covariance. The state estimation is updated by the actual measurement y k = zk − H k xˆk k -1 , xˆk k = xˆk k -1 + K k y k (4.23) where y k is the innovation (or measurement) residual. Finally, the Kalman Filter updates the error covariance matrix. As long as the noise is Gaussian, this method provides an optimal LS estimation. One of the major limitations of the Kalman Filter is that it is only applicable for linear systems. In the Extended Kalman Filter (EKF), the state transition and observation models may be any differentiable functions, but optimal estimation can no longer be provided. The more recent Unscented Kalman Filter (UKF) [109] was developed in 1997 and uses a deterministic sampling technique known as the unscented transform to compensate for this issue. 4.2.2.3 Filtering the system In the first approach, a simple position estimate was acquired through Kalman filtering. With the help of the filter, the noise was reduced standard deviation (X, Y, Z) to 0.06, 0.06, 0.09 mm. Based on the measured noise data, I could create a good set of filter parameters to achieve an effective solution. However, introducing a Kalman Filter added further delay to the position estimate when the robot was moving (figure 4.5). The maximum delay of the filter was between 280 – 380 ms. It is known that estimating the velocities together with the positions—using the same Kalman Filter—can compensate for the latency [110]. I implemented another version to reduce the latency, applying linear projection of the last known position with the estimated velocity. - 73 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 4.5 Figure 4.6 Sample of a recorded robot motion tracking. The filtering introduces significant latency. Shows the linear filter implemented for noise filtering in a simulation test. The new filter provided acceptable results; reducing the noise to 0.06, 0.08, 0.09 mm in X, Y, Z, respectively (figure 4.6). The latency cannot be characterized with a single number, the parameters that determine the delay in the velocity-based projection of the position have been set to follow the fast motion of the robot promptly, but this causes bigger latency when the motion consists of short and sudden movements. In the first case, the latency was 20-40 ms, and it could go up to 220-240 ms. 4.2.3 Verifying measurements To verify the results of the new system improvements, the optimized numbers were used to update the original pivot calibration datasets. I repeated the basic pivot calibration calculations according to (2.4), and found that the residual error’s average decrease was over 24% with the optimized parameters. Furthermore, I ran accuracy tests on the University of Nebraska phantom. These tests resulted in an average improvement of the Fiducial Registration Error by a mere 3% (from 0.390 to 0.378). Most probable explanation is that as the robot’s measured accuracy was already pretty good, while the measurement noise and the cutterTip’s primary (position-dependent) estimation played a more significant role in forming the numeric results. The improved accuracy of the individual components was therefore tested. Note that only complete application accuracy tests can illuminate the real effect of these changes. Unfortunately due to the limit in time and some delays in the new implementation of the virtual fixture control, we could not yet perform comparison phantom tests yet. 4.3 Challenges in the OR environment Introducing any automated device to the operating room (OR) poses additional safety risks, especially if it is not capable of adjusting to the changing environment of the OR. There are - 74 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery several people in the room, constantly in motion among the numerous medical devices surrounding the patient. Image guided surgery is based on the principle that the real-world setup does not change over time and its registration to the still image space thus remains valid. However, unintended changes in its position are prone to happen. The main sources of patient motion during surgery include: • • • • bumping into the operating table leaning against the patient loose setup equipment failure IGS is sensitive to changes, i.e., when the body is moved relative to the marker that tracks its motion. In a more serious case, the entire system can fail to appropriately operate during the remainder of the procedure. The probability of such malfunction can be reduced by using head (skull or teeth) mountable rigid bodies and a redundant number of markers, although these methods may be more invasive or disturbing for the surgeon during the procedure. The vast majority of the currently approved systems uses a Dynamic Reference Base (DRB) and provides the position of the patient-mounted rigid body in respect with that. This way the effect of the camera motion can be excluded. A much more common issue is patient movement (together with the corresponding frame) relative to other equipment in the OR. In classic IGS, this does not present a major problem since the navigation system tracks the rigid base frame and the hand-help probe relative to one another. However, in our setup, the fixed transformation between the robot and the patient is actively used in every control cycle to compute the cutterTip’s position with respect to the VF. If the transformation becomes invalid, placement errors would result during cutting. Figure 4.7 presents this case on the transformation block diagram of our system; RW DRB T becomes invalid as the patient moves relative to the robot. Turquoise colored VF spheres signify the frames where the virtual fixture is still valid, while a red sphere indicates that it is no longer valid (in RW). As the control computations are made in the Robot World, the displacement of the virtual fixture may result in a serious error in the cut or can entirely ruin the procedure. From the control point of view, this motion can also be considered as an error, referred to as patient error. All in all, to adapt to these changes in the operating environment, automated correction is needed and the surgeon should be notified about the event. Many technological solutions exist to handle these situations (some used exclusively in industry). In the next section, a general solution developed for this problem is presented along with results from the first experimental tests. - 75 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Figure 4.7 Accidental displacement of the patient breaks the transformation circle acquired by precise pre-operative registration. 4.3.1 Compensating for patient motion RW In a scenario described above, where the DRB T is erroneous (the patient was moved relative to the robot after the registration procedure), an update is needed to recover the application accuracy of the system. As this may happen in the middle of the surgery, it is absolutely critical to implement a stable and reliable method for registration recovery. To compensate for the unintended motion, the new frame transformation between RW and DRB must be collected intra-operatively. The first step is to monitor and measure the patient movement. There are several possible solutions to acquire this 3D transformation. Robotic setups can involve accelerometers and gyroscopes to detect for sudden translational and rotational changes in the position of the subject. These devices can be small enough to mount on the patient; however there is still need for electric coupling and the resolution may not be sufficient for our application. We also wanted to avoid the involvement of new hardware equipment both for logistical and financial reasons. CCD cameras provide a cheap solution for monitoring the operation site [96], yet again, the resolution may only be enough to conclude whether or not any motion shifts occurred. Furthermore, the inexpensive camera setups require sophisticated image processing algorithms, which was not in the scope of the project. It may be possible to attach a third trackable rigid body to the robot base to directly compute the RW DRB T transformation. Although by solely relying on the StealthStation’s localization, further estimation issues arise, as we are running the controller much faster than the StealthStation - 76 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery updates. We would have to use only an estimation of the tooltip for creating the control signals. Furthermore, the StealthStation can only track two rigid bodies at a time; therefore we may loose the ability to track the end of the robot. Using a more versatile surgical navigator system would only partially solve the issue, as there is still the problem of keeping all the rigid bodies in the limited measurement volume and in the line of sight of the navigation system. The patient’s head and the base of the robot can easily be 2-3 feet away from one another. Furthermore, the transformation between the BASE coordinate system and the DRB would have to be derived through a new registration procedure. If the rigid body was permanently mounted to the robot base, it would be necessary to calibrate the transformation between that rigid body and the RW coordinate system, probably by temporarily attaching another rigid body to the robot tool and moving it to a few points around the workspace. The solution is feasible, but rather cumbersome. A similar, fixed reference frame-based patient tracking method was investigated in [111] for frameless stereotactic neurosurgery, with an average localization error of 4.8 ± 3.5 mm. Figure 4.8 Re-registration of coordinate frames based on a prior RRB2TCP transformation. We chose to keep the current hardware setup and take advantage of the already implemented registration procedures to develop a more convenient solution (figure 4.8). By extending the existing transformation control scheme, we want to compute for the full RRB TCP T transformation as well during the robot’s registration to the StealthStation (Robot2StealthStation, see Section 2.3.2). In the future, we may be able to use the fixed RW DRB RRB TCP T transformation to compensate for T (patient motion) during an operation, as described in the next section. - 77 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery 4.3.2 Sensor fusion for accuracy 4.3.2.1 Coordinate transformations As previously mentioned, we were left with the option of using the StealthStation and two trackable rigid bodies (one on the robot tool and one on the patient) to compensate for patient motion. As described in Section 3.4, the optical tracker, which introduces additional noise, is less accurate and slower than the robot. Because of these drawbacks, we decided not to use it in the low level control in the initial experimental setup (described in Section 2.3). However, from the clinical point of view, the StealthStation provides a significant advantage by tracking the relative positions of the rigid bodies within the operating site. Based on the previous results, we decided to replace the prior control scheme, which only used the robot’s encoders to determine the position of the tool tip, with the method based on StealthStation measurements. In our initial implementation, we performed the virtual fixture computations in the Robot World coordinate system. This process is advantageous because it does not require line-of-sight between the StealthStation localizer and the robot during the procedure. But, because this method transforms the virtual fixture to RW coordinates at the start of the procedure, it cannot detect subsequent deviations from the original transformation (DRB to RW). This can be problematic if the patient’s head is moved or the operating table is bumped during the surgery. This change in the operating environment results in the displacement of the VF position, losing its effectiveness. Figure 4.9 General solution for motion compensation, by using the position information from both the robot and the localizer. We had to find an optimal solution to combine the available devices. On one hand, the robot offers a highly accurate structure for position measurement, but is not capable of tracking - 78 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery absolute motions. Alternatively, there exists a less accurate and noisy navigation system that provides absolute position measurements. It is possible to perform virtual fixture computations both in the DRB coordinate system—using the position of the tool tip as measured by the StealthStation—and in the robot base coordinate system (RW). The difficulty arose in finding a method of sensor fusion that most effectively merged the information derived from the two devices, as figure 4.9 shows. Figure 4.10 shows in detail the general problem we want to solve: finding the optimal estimation of PDRB (DRB position) given the sensory information PDRB and PRW from the StealthStation and the NeuroMate, respectively. The Sensor Fusion block should also provide an optimal solution for the RW DRB T transformation, making the computation of the VF and the robot control accurate. Figure 4.10 Schematic to describe the use of input measurements to create control outputs. Virtual fixture computations should a this PDRB value together with the force sensor reading RW transformed to the same coordinate system (through the rotation component of DRB T ) to create the appropriate control command. The joint velocity command would be transformed back to RW, through the inverse of the same rotation, and applied to the servos. Fortunately, we note that because we are only transforming forces and velocities between the RW and DRB coordinate systems, the controller is not affected by translation of the patient with respect to the robot (although PDRB might be). 4.3.2.2 Variations for sensor fusion Here we discuss three different ways to achieve sensor fusion (figure 4.11). The first option is to use the discrete implementation, where we check regularly for changes in the OR environment - 79 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery using the StealthStation. This technique offers two possible approaches that may lead to an acceptable solution. If measurement noise is high in the optical localization and patient error is rare (the body barely moves relative to the robot), the best strategy might be to perform a good registration, control the robot with that registration, and re-register the robot if the localizer notices any patient movement. During the re-registration procedure, we can apply accurate noise filters to provide good results. On the other hand, if patient error consists more of continuous drift and random motions relative to the robot (with a mean value of error higher than that of the StealthStation), we may be better off directing the robot based entirely on the optical tracker’s information. In this case, Kalman filtering of the localizer data becomes crucial, as it directly affects the overall accuracy of the system. Figure 4.11 The three main approaches to use sensor fusion to get a more accurate estimation for the tool position. To determine which one is the superior solution (providing higher application accuracy over time), we will have to collect relevant noise and error data from real surgical procedures. This may cause some technical difficulties, but we are currently working on a solution to address these issues. The only available clinical data [112] suggest that the first strategy is better in most cases. The other solution is to apply continuous time updates, using the available encoder and localizer information in every motion control cycle. We need an advanced non-linear filter that is capable of mapping the nonlinear transformation between the inputs and generating the required filtered position and transformation output. A finely tuned Unscented Kalman Filter (UKF) should be suitable for this task. Future research plans include the implementation of a UKF. - 80 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery 4.3.2.3 Implementation of the new concept We decided to conduct experiments with the first strategy, using solely the robot encoder feedback for control computations, but checking periodically for patient motion through the localizer’s information. I implemented a new version of the control that uses an architecture (figure 4.9) in which the VF computations are in DRB coordinates, rather than RW coordinates. By choosing the control mode through the StealthStation, the ability to monitor the relative position of the two frames (RRB and DRB) remains, and it is therefore possible to detect and compensate for unintentional motions of the patient with respect to the robot. Once the desired velocities are computed in the DRB frame, they must be transformed back to the RW frame in order to generate the joint velocity commands for the NeuroMate. The fixed, but unknown transformation between the TCP and the RRB is computed during the Robot2StealthStation registration using the transformations RRB TCP RRB Loc Loc RW T ⋅ DRB T ⋅ DRB RWT ⋅ TCPT . T= (4.24) The TCP to RRB transformation is stored after registration and can be used at any time to update the RW to DRB transformation. In the new implementation, the controller compares the tooltip positions computed through the robot kinematics and the localizer readings every 0.5 s. Compensation for patient motion is realized by applying the following transformation: DRB RW T= DRB Loc Loc TCP T ⋅ RRB T ⋅ RRB TCPT ⋅ RW T . (4.25) For better performance of the re-registration, I implemented an averaging filter for the Loc DRB T StealthStation readings. However, the averaging of a transformation matrix’s rotation part is not trivial [113]. One solution is to transform the rotation matrix to Euler angles, apply the averaging, and then transform the Euler angles back to a rotation matrix. I used the Rot(Z,φ ) ⋅ Rot(Y,ϑ ) ⋅ Rot(Z,ψ ) Euler angle convention (yaw-pitch-yaw sequence)—most common in robotics. My filter uses an adjustable window size to acquire filtered data with the desired accuracy. The forward transformation is the following: Cφ ⋅ Cϑ ⋅ Cψ - Sφ ⋅ Sψ Rot2Euler = Sφ ⋅ Cϑ ⋅ Cψ + Cφ ⋅ Sψ -Sϑ ⋅ Cψ -Cφ ⋅ Cϑ ⋅ Sψ - Sφ ⋅ Cψ -Sφ ⋅ Cϑ ⋅ Sψ + Cφ ⋅ Cψ Sϑ ⋅ Sψ Cφ ⋅ Sϑ Sφ ⋅ Sϑ Cϑ (4.26) These inverse calculations are less straight-forward. First of all, we have to check for singular configurations. This occurs when Sϑ = 0; ϑ = 0 ,π (4.27) - 81 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery In this case only ϕ +ψ can be determined; therefore there are infinite number of solutions. We choose one that is close to the previous values of the angles, and supposedly more realistic. Further, two solutions are possible; if ϑ = 0; Cϑ = 1 , then the Euler angles are S ⋅ C + Cφ ⋅ Sψ atan φ ψ C ⋅C - S ⋅S φ ψ φ ψ φ = φold + 2 ϑ = 0, where S ⋅ C + Cφ ⋅ Sψ − φ + ψ old ) atan φ ψ C ⋅ C - S ⋅ S ( old φ ψ φ ψ ψ = ψ old + 2 Sφ ⋅ Cψ + Cφ ⋅ Sψ = Sφ+ψ ; Cφ ⋅ Cψ - Sφ ⋅ Sψ = Cφ+ψ and if ϑ = π ; ϑ = π, − (φold + ψ old ) (4.28) (4.29) (4.30) Cϑ = −1 , then -S ⋅ C + Cφ ⋅ Sψ atan φ ψ S ⋅S + C ⋅C φ ψ φ ψ φ = φold − 2 -S ⋅ C + Cφ ⋅ Sψ atan φ ψ S ⋅S + C ⋅C φ ψ φ ψ ψ = ψ old + 2 Otherwise, for non-singular configurations − (φold + ψ old ) (4.31) − (φold + ψ old ) . Sφ ⋅ Sϑ Cφ ⋅ Sϑ Cφ ( Cφ ⋅ Sϑ ) + Sφ (Sφ ⋅ Sϑ ) ϑ = atan C ϑ -Cφ ( Cφ ⋅ Sϑ ) + Sφ ( Sφ ⋅ Sϑ ) ψ = atan -Sφ ( -Cφ ⋅ Cϑ ⋅ Sψ - Sφ ⋅ Cψ ) + Cφ ( -Sφ ⋅ Cϑ ⋅ Sψ (4.32) φ = atan (4.33) (4.34) + Cφ ⋅ Cψ ) (4.35) It is imperative that in the case of major motion of the patient the surgeons get notified so they are aware of these changes and can be mindful towards verifying the effectiveness of the new registration. I designed an additional display widget that calculates the actual effectiveness of the registration and shows the accuracy of the procedure. It notifies the surgeon if re-registration is needed and allows one to manually select re-registration from the GUI menu. 4.4 Verifying the results To test the new control concept developed, I intentionally changed the relative position of the skull with respect to the robot during a simulated procedure. The DRB was mounted on a high- - 82 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery precision 1 DOF rotary stage capable of moving with an accuracy of 0.001 degree (figure 4.12). RW Having registered the robot to the StealthStation, the program saved both the DRB T and the RRB TCP T transformations. Then, I rotated the DRB around its approximate Z axis by exactly -90 degrees (as it was rigidly attached to the rotary stage.) The program could detect and compensate for the “patient motion” based on the previously stored RRB TCP T transformation (4.24). I made further rotations of -90 deg, +45 deg, and +45 deg (figure 4.13). From the recorded Loc DRB T ⋅ DRB RWT transformation matrices, it was possible to derive the accuracy of the updates: RRB TCP NEW T ⋅ RRB -1 TCP OLD Rot T 0.992 0 0 0 0 = -0.990 0 0.997 0 In another experiment, I was manually updating the ±0.1 (4.36) RRB TCP T transformation, while the DRB was not moved. The acquired new matrix reduced the effect of the noise almost by one order: RRB TCP NEW T -1 ⋅ RRB TCPTOLD Figure 4.12 Figure 4.13 1.0000 0.0006 - 0.0004 - 0.0270 - 0.0006 1.0000 - 0.0005 0.0401 = 0.0004 0.0005 1.0000 - 0.0710 0 0 0 1.0000 (4.37) High precision 1 DOF rotary stage. The DRB mounted on the rotary stage for the verification of the transformation filter. The program was able to detect and compensate for the resulting misalignment under static conditions, when the robot was stopped during the time of the averaging filtering of the Loc DRB T transformation. - 83 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Achieving a precise dynamic re-calibration of the transformations is more difficult, as we must compensate for the time difference between measurements of the transformations in equation (4.25). Specifically, the first two transformations on the right side of (4.25) are measured by the StealthStation and therefore have a delay of up to 380 ms with respect to the last transformation, measured by the NeuroMate. We anticipate that the advantages and effectiveness of the averaging filter provide justification for pausing the operation for approximately 1 s to apply the filter. However, if this occurs too often, it may annoy the surgeon and in that case a different filter should be used, such as the Kalman Filter. For better estimation, we could also take advantage of times where the robot is not moving. Also, there may be frequent times during the procedure where motion is insignificant. Further experiments are to be conducted to determine the accuracy of the compensation for typical, arbitrary changes in the surgical environment. - 84 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Conclusion Robotic neurosurgery has already demonstrated its utility in the case of several applications from biopsy to skull base drilling. Surgical robots have affected the current practice of neurosurgery through many FDA approved devices. Better understanding of the procedure of brain and spine surgery and the human anatomy will help to develop integrated technologies that suit the surgeons need. In the first part of the thesis, the existing neurosurgical installations were presented, and three directions of development were analyzed: the improvement of accuracy in stereotactic procedures, the close integration with imaging devices and the use of the hands-on surgery concept. These have potential to greatly improve the overall quality of computer-integrated neurosurgery. The second part of the thesis described the neurosurgical system at the Johns Hopkins University. It successfully integrates a NeuroMate robot (FDA approved for frame-based and frameless stereotactic surgery), a StealthStation surgical navigation system (FDA approved) and 3D Slicer open source medical navigation and visualization software. The robot is driven in cooperative control mode, where the control signals are created using the readings from the force sensor mounted at the end-effector of the robot. Our system has major advantages that may lead to a significant improvement in the quality of skull base surgery. Beyond offering advanced visualization through 3D Slicer, the systems improves the surgical tool’s stability (essentially eliminating freehand tremor), as it is mounted on the NeuroMate robot. Though the surgeon still holds the conventional drill tool and directs its movement, he or she can release the tool at any time to take a rest, or to attend to another task, while the robot is maintaining the tool’s position. To increase the safety and the reliability of the procedure we apply the concept of virtual fixtures that has already proved its effectiveness in several applications. These boundaries are defined on the pre-operative CT scan by the surgeon to cover and protect the critical anatomical structures. Registered to the robot, the virtual fixture is used to prevent the tip of the tool from going beyond the defined safe area in any direction. The VF allows surgeons to operate safely and move easily within millimeters of vessels and nerves, thus reducing lengthy operating times and associated fatigue, while keeping the necessary high level of precision. In the third part of the thesis, I have thoroughly documented my research efforts to determine the accuracy of the individual system components, model the noise and identify the major source of system errors based on the extensive phantom and cadaver tests of the preliminary system. - 85 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery I examined several methods to improve the overall accuracy, effectiveness and safety of our system. Based on the findings, I extended the previously used pivot calibration method to a closed kinematic chain calibration of the robot’s Denavit-Hartenberg parameters and the cutter tool’s dimensions. Furthermore, I implemented a Kalman Filter to reduce the noise of the optical tracker, as described in the last chapter. My contribution involved the development and testing of a new technique capable of providing improved safety through patient monitoring and automated transformation updating. All methods were tested and found to significantly improve the application accuracy of the system. Throughout the research, it was a priority to use a universal approach and develop general methods that may be used in other medical robotic systems. Partial results have been published in a paper at the 14th Nordic-Baltic Conference on Biomedical Engineering and Medical Physics [27] and another paper has been accepted for the 2nd IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob). We are continuing the research, as the application accuracy of our system is promising for future development towards clinical applications. Through advanced technological solutions, we can improve the quality of future healthcare and justify the higher investment costs of robotic interventions. Systems currently under development will soon deliver great clinical advantages and improved safety features providing advanced procedures that benefit both the patient and the surgeon. We hope that our neurosurgical system will find its way to the market and will be integrated well in the future of health care. - 86 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Future work I described in this thesis our ongoing work to implement and test the most suitable techniques for an integrated skull base surgery system. Presently, hands-on surgery is a promising direction of Computer-Integrated Surgery and our robotic installation may one day contribute to the future of innovative interventional medicine. There are many possible ways to improve the present system. The overall error of can be further reduced (both placement and dimensional) by introducing better algorithms for calibration and registration. We plan to model the bending of the surgical tool via contact forces and compensate for it during the procedure. Moreover, a new implementation of the virtual fixtures is under development, incorporating robot control features and spatial constraints during the optimization as well. Additional phantom and cadaver experiments will be performed to obtain statistical significance for the results. A natural extension is adding the option to automatically update the Robot World to Dynamic Reference Base transformation while the robot is moving. This requires prediction of the tooltip to compensate for the latency introduced by the StealthStation and the Kalman Filter. The filter could also be used to estimate the velocity of the tooltip for this projection. An Unscented Kalman Filter could be used to estimate the position of the tool very accurately and compensate for patient movement in one step. It is necessary to attend more clinical operations, and learn about the environment (general setup, typical noise, changes in the OR). As the concept of cooperative and constrained control is platform-independent, our tools could be applied to other devices suitable for a wider range of surgical procedures. The path of future development is the integration of surgical navigation, telemedicine, nanotechnology, micromachines and microelectrical systems in a common framework supported by powerful computing. Our work takes a bold step forward in the progress towards this goal. - 87 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery Appendix Supplementary CD-ROM A supplementary CD-ROM is attached to the diploma thesis providing the source files, simulation results, videos, pictures and additional materials related to my work. However, to create the executable files and run the programs, the entire CISST Software framework must be installed. (It is open source and freely available [87].) The following list of folder intends to give guidance to the CD: \diploma_haidegger ..\additional_materials ..\pictures ..\references ..\further_references ..\support ..\Adobe_Reader ..\IrfanView ..\VLC_media_player ..\documentation ..\source ..\neurosurgery_robot_control ..\Kalman_filter ..\DH_calibration ..\patient_motion Folder \documentation contains my diploma thesis in .pdf format and the presentation created for the defense. Under \additional_materials the electronically available references can be found along with further publications related to surgical robotics. A selection of additional pictures is also available under \pictures. The \support folder contains the necessary programs to view the pictures, videos and read the documentation. - 88 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery References The references are in order of appearance, following the IEEE standard format. A vast majority of them are available on the Supplement CD, or on the internet. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] B. Plinkert, P. K. Plinkert, “Robotics in skull base surgery“. Computer Assisted Radiology and Surgery; Vol.1230, p.138-142, 2001 HAESF - Hungarian American Enterprise Scholarship Fund; www.haesf.org Medical Robotic Database: MeRoDa: http://meroda.uni-hd.de/ (accessed: May 2008) G. H. Ballantyne, F. Moll, “The da Vinci telerobotic surgical system: the virtual operative field and telepresence surgery”. Surgical Clinics of North America; Vol.83, p.1293–1304, 2003 N. Nathoo, T. Pesek, G. H. Barnett, “Robotics and neurosurgery”. Surgical Clinics of North America; Vol.83, p.1339-1350, 2003 D. M. Herron, M. Marohn, the SAGES-MIRA Robotic Surgery Consensus Group, “A consensus document on robotic surgery”. Surgical Endoscopy; Vol.22, p.313-325, 2008 R. Taylor, D. Stoianovici, “Medical Robotics in Computer-Integrated Surgery”. IEEE Transaction on Robotics and Automation; Vol.19, Issue:5, p.765-781, 2003 J. Speich, J. Rosen, “Medical Robotics”. In Encyclopedia of Biomaterials and Biomedical Engineering, Marcel Dekker, 2004 R. A. Faust (editor), “Robotics in Surgery: History, Current and Future Applications”. Nova Science Publishers, New York, USA, 2007 N. Nathoo, M. C. Cavusoglu, M. A. Vogelbaum, G. H. Barnett, Gene, “In Touch with Robotics: Neurosurgery for the Future” Journal of Neurosurgery; Vol.56, Issue:3, p.421-433, 2005 P. J. Kelly, “Stereotactic Surgery: What is Past is Prologue”. Neurosurgery; Vol.46, Issue:1, p.1626., 2000 P. Grunert, K. Darabi, J. Espinosa, R. Filippi, ”Computer-aided navigation in neurosurgery”. Neurosurgery Review; Vol.26, p.73-99, 2003 P. J. Stolka, D. Henrich, “Using Maps for Local Sensors for Volume-Removing Tools”. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Diego, CA, USA, 2007 P. A. Woerdeman, P. W. A. Willems, H. J. Noordmans, C. A. F. Tulleken, J. W. B. van der Sprenkel, “Application accuracy in frameless image-guided neurosurgery: a comparison study of three patient-to-image registration methods.” Journal of Neurosurgery; Vol.106, Issue:6, p.10121016, 2007 R. H. Taylor, S. Lavallée, G. C. Burdea, R. Mösges (editors), “Computer-Integrated Surgery”. The MIT Press, Cambridge, MA, USA, 1995 G. H Ballantyne, J. Marescaux, P. C. Giulianotti (editors), “The Primer of Robotic and Telerobotic Surgery”. Lippincott Williams & Wilkins, 2004 C.S. Karas, M. N. Baig, “Robotic Neurosurgery”. In Medical Robotics, edited by V. Bozovic, I-Tech Education and Publishing, 2008 R. M. Satava, “Emerging Technologies in Surgery ”. Springer, New York, USA, 2007 WebSurg – World Electronic Book of Surgery, http://www.websurg.com/ (accessed: May 2008) Robotic Surgery Research Website at the Brown University; http://biomed.brown.edu/ /Courses/ BI108/BI108_2005_Groups/04/ (accessed: May 2008) MedGadget – Internet Journal of Emerging Medical Technologies, http://medgadget.com/ (accessed: May 2008) R. Howe, Y. Matsuoka, “Robotics for Surgery”. Annual Review of Biomedical Engineering, 1999 E. Dombre, “Introduction to Surgical Robotics”. Proceedings of the 2nd Summer European University, Montpellier, France, 2005 - 89 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] C. S. Karas, “Neurosurgical robotics: a review of brain and spine applications”. Journal of Robotic Surgery, Vol.1, No.1, p.39–43, 2007 P. Pott , H. Scharf, M. Schwarz, “Today's State of the Art of surgical Robotics”. Journal of Computer Aided Surgery; Vol.10, Issue:2, p.101-132, 2005 N. Nathoo, M.C. Cavusoglu, M.A. Vogelbaum, G.H. Barnett, “In Touch with Robotics: Neurosurgery for the Future”. Journal of Neurosurgery, Vol.56, p.421-433, 2005 T. Haidegger, L. Kovacs, G Fordos, Z. Benyo, P. Kazanzides. “Future Trends in Robotic Neurosurgery”. Proceedings of 14th Nordic-Baltic Conference on Biomedical Engineering and Medical Physics, Riga, Latvia, 2008 Y.S. Kwoh, J. Hou, E.A. Jonckheere, S. Hayati, “A Robot with Improved Absolute Positioning Accuracy for CT Guided Stereotactic Brain Surgery”; IEEE Transactions on Biomedical Engineering, Vol.35, No.2, 1988 D. F. F. Louw, T. Fielding, P. B. McBeth, D. Gregoris, P. Newhook, G. R. Sutherland, “Surgical Robotics: a Review and Neurosurgical Prototype Development”. Neurosurgery; Vol.54, No.3, 2004 B. A. Kall, P. J. Kelly, S. Goerss, G. Frieder, “Methodology and clinical experience with computed tomography and a computer-resident stereotactic atlas”. Neurosurgery; Vol.17, Issue:3, p.400-407, 1985 C. W. Burckhardt, P. Flury, D. Glauser, “Stereotactic Brain Surgery”. IEEE Engineering in Medicine and Biology magazine; Vol.14, Issue:3, p.314-317, 1995 C. Nguyen, K. Cleary, “State of the Art in Surgical Robotics: Clinical Applications and Technology Challenges”. Computer Aided Surgery, Special Issue of Medical Robotics Vol.6, Issue:6, p.312–328, 2002 T. R. K. Varma, P. Eldridge, “Use of the NeuroMate stereotactic robot in a frameless mode for functional neurosurgery”. International Journal of Medical Robotics and Computer Assisted Surgery, Vol.2, Issue:2, p.107–113, 2003 A. L. Benabid, P. Cinquin, S. Lavalle, J.F. Le Bas, J. Demongeot, J. de Rougemont, “Computerdriven Robot for Stereotactic Surgery Connected to CT Scan and Magnetic Resonance Imaging Technological Design and Preliminary Results”, Journal of Stereotactic Functional Neurosurgery; Vol.50, p.153-154, 1987 P. B. McBeth, D. F. Louw, P. R. Rizun, G. R. Sutherland, “Robotics in Neurosurgery”. The American Journal of Surgery; Vol.188, Issue:4, Suppl.1, p.38-75, 2004 K. Chinzei, K. Miller, “An MRI Guided Surgical Robot”. Proceedings of the Australian Conference on Robotics and Automation, Sydney, Australia, 2001 T. Zengmin, L. Wangsheng, W. Tianmiao, M. Belin, Z. Quanjun, Z. Guolai, “Application of a Robotic Telemanipulation System in Stereotactic Surgery”. Stereotactic and Functional Neurosurgery; Vol.86, p.54–61, 2008 A. Popovica, M. Engelhardt, T. Wu, F. Portheine, K. Schmieder, K. Radermacher, “CRANIO– computer-assisted planning for navigated and robot-assisted surgery on the skull”. Proceedings of the 17th International Congress and Exhibition on Computer Assisted Radiology and Surgery; International Congress Series; Vol.1256, p.1269–1275, London, UK, 2003 D. Engel, J. Raczkowsky, H. Worn, “A Safe Robot System for Craniofacial Surgery”. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Seoul, South Korea, 2001 A. Muacevic, B. Wowra, “Cyberknife Radiosurgery”. European Neurological Disease; Issue:7, 2007 M. Zimmermann, R. Krishnan, A. Raabe, V. Seifert, “Robot-assisted Navigated Neuroendoscopy; Neurosurgery; Vol.51, Issue:6, p.1446-1452, 2002 P. A. Federspil, U. W. Geisthoff, D. Henrich, P. K. Plinkert, “Development of the First ForceControlled Robot for Otoneurosurgery”. Laryngoscope; Vol.113, p.465–471, 2003 S. Lavallee, P. Cinquin, “IGOR: Image Guided Operating Robot”. Proceedings of 5th IEEE International Conference on Advanced Robotics (ICAR); Vol.1, p.876-881, San Francisco, CA, USA, 1991 P. Kazanzides, T. Xia, C. Baird, G. Jallo, K. Hayes, N. Nakijima, “A Cooperatively-controlled Image Guided Robot System for Skull Base Surgery”. Proceedings of the 16th Medicine Meets Virtual Reality (MMVR), Long Beach, CA, USA, 2008 T. Ortmaier, “KineMedic: Robot Assisted Placement of Pedicle Screws”. Proceedings of the 2nd Summer European University, Montpellier, France, 2005 - 90 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] R. Shamir, M. Freiman, L. Joskowicz, M. Shoham, E. Zehavi, Y. Shoshan, “Robot-Assisted Image-Guided Targeting for Minimally Invasive Neurosurgery: Planning, Registration, and Invitro Experiment”. Proceedings of International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI); Lecture Notes in Computer Science; Vol.3750; p.131-138, Palm Springs, CA, USA, 2005 K. Bumm, J. Wurm, J. Rachinger, T. Dannenmann, C. Bohr, R. Fahlbusch, H. Iro, C. Nimsky, “An automated robotic approach with redundant navigation for minimal invasive extended transsphenoidal skull base surgery”. Minimally Invasive Neurosurgery; Vol.48, Issue:3, p.159-64, 2005 N. Villotte, D. Glauser, P. Flury, C. W. Burckhardt, “Conception of Stereotactic Instruments for the Neurosurgical Robot Minerva”. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Nice, France, 1992 M. D. Chen, T. Wang, Q. X. Zhang, Y. Zhang, Z. M. Tian, “A Robotics System for Stereotactic Neurosurgery and Its Clinical Application”. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Vol.2, p.995-1000, Leuven, Belgium, 1998 M. Nakamura, N. Tamaki, S. Tamura, H. Zamashita, Y. Hara, K. Ehara, “Image-guided microsurgery with the Mehrkoordinaten Manipulator system for cerebral arteriovenous malformations”. Journal of Clinical Neuroscience; Vol.7, Issue:1, p.10-13., 2000 A. Morita, S. Sora, M. Mitsuishi, S. Warisawa, K. Suruman, D. Asai, J. Arata, S. Baba, H. Takahashi, “Microsurgical robotic system for the deep surgical field: development of a prototype and feasibility studies in animaland cadaveric models”. Journal of Neurosurgery; Vol.103, p.320327, 2005 T. C. Tsai, “Development of a Parallel Surgical Robot with Automatic Bone Drilling Carriage for Stereotactic Neurosurgery”. Proceedings of IEEE International Conference on Systems, Man and Cybernetics, the Hague, the Netherlands, 2004 B. L. Davies, S. Chauhan, Mi. J. S. Lowe, “A Robotic Approach to HIFU Based Neurosurgery”. Proceedings of the First International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI); Lecture Notes in Computer Science; Vol.1496; p.386-396, Cambridge, MA, USA, 1998 G.R. Sutherland, I. Latour, A. D. Greer, T. Fielding, G. Feil, P. Newhook, “An Image-guided MR Compatible Surgical Robot”. Neurosurgery; Vol.62, Issue:2, p.286-293, 2008 D. Handini, T. M. Yeong, L. V. Hong, “System Integration of NeuroBot – A Skull-Base Surgical Robotic System”. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Vol.1, p.43-48, New Orleans, LA, USA, 2004 K. Hongo, S. Kobayashi, Y. Kakizawa, J. Koyama, T. Goto, H. Okudera, K. Kan, M. Fujie, H. Iseki, K. Takakura, “NEUROBOT: Telecontrolled Micropanipulator System for Minimally Invasive Micro-neurosurgery—Preliminary Results”. Neurosurgery; Vol.51, No.4, p.985-988, 2002 B. Davies, S. Starkie, S. J. Harris, E. Agterhuis, V . Paul, L. M. Auer, “Neurobot: a special-purpose robot for Neurosurgery”. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA); San Francisco, CA, 2000 J. Liu, Y. Zhang, Z. Li, “The Application Accuracy of NeuroMaster: a Robot System for Stereotactic Neurosurgery”. Proceedings of the 2nd IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications(MESA); Beijing, China, 2006 Q. H. Li, L. Zamorano, A. Pandya, R. Perez, J. Gond, F. Diaz, “The Application Accuracy of the NeuroMate Robot— A Quantitative Comparison with Frameless and Frame-Based Surgical Localization Systems”. Computer Aided Surgery; Vol.7, Issue:2, p.90–98, 2002 F. Chan, I. Kassim, C. Lo, C. L. Ho, B. T. Ang, I. Ng, “Image Guided Robotic Neurosurgery – In-vivo computer guided craniectomy”. Proceeding of ACCAS - 3rd Asian Computer Assisted Surgery Conference; Singapore, 2007 M. S. Eljamel, “Validation of the PathFinder neurosurgical robot using a phantom”. International Journal of Medical Robotics and Computer Assisted Surgery; Vol.3, Issue:4, 2007 M. J. H Lum, J. Rosen, H. King, D. C. Friedman, G. Donlin, G. Sankaranarayanan, B. Harnett, L. Huffman, C. Doarn, T. Broderick, B. Hannaford, “Telesurgery Via Unmanned Aerial Vehicle”. Proceedings of the 15th Medicine Meets Virtual Reality (MMVR), Long Beach, CA, USA, 2007 - 91 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] H. Das. T. Ohm, C. Boswell, R. Steele, G. Rodriguez, “Robot Assisted Microsurgery Development at JPL”. Proceedings of the Workshop on Medical Application of Virtual Reality; Chicago, IL, USA, 2001 R. Taylor, A Barnes, R. Kumar, P. Gupta, Z. X. Wang, P. Jensen, L. Whitcomb, E. de Juan, D. Stoianovici, L. Kavoussi, “A Steady-Hand Robotic System for Microsurgical Augmentation”. International Journal of Robotic Research; Vol.18, No.12, p.1201-1210, 1999 D. V. Amin, L. D. Lunsford, “Volumetric Resection Using the SurgiScope®: A Quantitative Accuracy Analysis of Robot-Assisted Resection”. Journal of Stereotactic Functional Neurosurgery; Vol.82, p.250–253, 2004 W. Shen, J. Gu, Y. Shen, “Using Tele-robotic Skull Drill for Neurosurgical Applications”. Proc. of the IEEE International Conference on Mechatronics and Automation; Luoyang, China, 2006 K. Masamune, E. Kobayashi, Y. Masutani, M. Suzuki, T. Dohi, H. Iseki, and K. Takakura, "Development of an MRI-compatible needle insertion manipulator for stereotactic neurosurgery". Journal of Image Guided Surgery; Vol.1, p.242-248, 1995 CEDIT Group, “The NeuroMate Neurosurgical Stereotactic Robot”. CEDIT Recommendations, 2001 K. Hongo, T. Goto, Y. Kakizawa, J. Koyama, T. Kawaib, K. Kanb, Y. Tanaka, S. Kobayashi, “Micromanipulator system (NeuRobot): clinical application in neurosurgery”. International Congress Series; Vol.1256, p.509– 513, 2003 T. Goto, K. Hongo, Y. Kakizawa, H. Muraoka, Y. Miyairi, Y. Tanaka, S. Kobayashi, “Clinical application of robotic telemanipulation system in neurosurgery”. Journal of Neurosurgery, Vol.99, p.1082-1084, 2003 K. Hongo , T. Goto, T. Miyahara, Y. Kakizawa, J. Koyama and Y. Tanaka Telecontrolled micromanipulator system (NeuRobot) for minimally invasive neurosurgery”. In “Medical Technologies in Neurosurgery” C. Nimsky, R. Fahlbusch (editors), Springer, Vienna, Austria, 2006 P. D. Le Roux, H. Das, S. Esquenazi, P. J. Kelly, “Robot-assisted microsurgery: a feasibility study in the rat”. Neurosurgery; Vol.48, Issue:3, p.584-589, 2001 E. Shaw, C. Scott, L. Souhami, R. Dinapoli, R. Kline, J. Loeffler, N. Farnan, “Single dose radiosurgical treatment of recurrent previously irradiated primary brain tumors and brain metastases: final report of RTOG protocol 90-05”. International Journal of Radiation Oncology, Biology and Physics; Vol.47, No.2, p.291-298, 2000 ROBOCAST Project (EU - FP7): http://www.robocast.eu/ IEEE Engineering in Medicine and Biology magazine, Special Issue on MR Compatible Robotics; Vol.27, Issue:3, May/June, 2008 Skull Base Institute, Los Angeles, CA, USA; www.skullbaseinstitue.com (accessed: May 2008) Skull Base Anatomy site, Wayne State University, School of Medicine; http://www.med.wayne.edu/diagRadiology/Anatomy_Modules/axialpages/(accessed: May 2008) Karl Storz Endoscopy; Transnasal Skull Base Surgery product catalog, 2007 H. D. Jho, R. L. Carrau, M. L. McLaughlin, S. C. Somaza, “Endoscopic transsphenoidal resection of a large chordoma in the posterior fossa: A case report”. Acta Neurochirurgica; Vol.139, p.343348, 1997 M. Jakopec, F. R. y Baena, S. J. Harris, P. Gomes, J. Cobb, B. L. Davies, “The Hands-On Orthopaedic Robot Acrobot: Early Clinical Trials of Total Knee Replacement Surgery”. IEEE Transactions on Robotics and Automation; Vol.19, No.5, 2003 StealthLink Manual; Medronic, draft version, 2005 M. Kaus, R. Steinmeier, T. Sporer, O. Ganslandt, R. Fahlbusch, “Technical accuracy of a neuronavigation system measured with a high-precision mechanical micromanipulator”. Neurosurgery; Vol.41, Issue:6, p.1431-1437, 1997 M.T. Stechison, “A Digitized Biopsy Needle for Frameless Stereotactic Biopsies with the StealthStation”. Neurosurgery, Vol.46, Issue:1, p.239-41, 2000 S. Pieper, M. Halle, R. Kikinis, “3D Slicer”. Proceeding of IEEE International Symposium Biomedical Imaging: Nano to Macro; Vol.1, p.632-635, Arlington, VA, USA, 2004 L. B. Rosenberg, “Virtual fixtures: Perceptual tools for telerobotic manipulation”. Proceedings of the IEEE Annual International Symposium on Virtual Reality, Vol.1, p.76–82, Seattle, WA, USA, 1993 - 92 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] K. Yaghmour, “The Real-Time Application Interface”. Proceedings of the Linux Symposium, Ottawa, Canada, 2001 ERC CISST Software package for medical robot control; www.cisst.org/cisst (accessed: May 2008) A. Kapoor, A. Deguet, P. Kazanzides, “Software components and frameworks for medical robot control”. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Vol.1, p.3813–3818, Orlando, FL, USA, 2006 FLTK Documentation; http://www.fltk.org/documentation.php (accessed: May 2008) M. Matinfar, C. Baird, A. Batouli, R. Clatterbuck, P. Kazanzides, “Robot-Assisted Skull Base Surgery”. Proceedings of IEEE Intentional Conference on Intelligent Robots and Systems (IROS), San Diego, CA, USA, 2007 P. Kazanzides, J. Zuhars, B. Mittelstadt, R.H. Taylor, “Force Sensing and Control for a Surgical Robot”. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Nice, France, 1992 C. S. Gatla, R. Lumia, J. Wood, G. Starr, “An Automated Method to Calibrate Industrial Robots Using a Virtual Closed Kinematic Chain”. IEEE Transactions on Robotics; Vol.23, No.6, 2007 S. Umeyama, “Least squares estimation of transformation parameters between two point patterns”. IEEE Transactions on Pattern Analysis and Machine Intelligence; Vol.13, p.376-380, 1991 C. Sim, W. S. Ng, M. Y. Teo, Y. C. Loh, T. T. Yeo, “Image guided manipulator compliant surgical planning methodology for robotic skull-base surgery”. Proceedings of the Intentional Workshop on Medical Imaging and Augmented Reality, Vol.1, p.26–29, Hong Kong, China, 2001 M. Li, M. Ishii, R.H. Taylor, “Spatial motion constraints using virtual fixtures generated by anatomy”. IEEE Transactions on Robotics, Vol.23, Issue:1, p.4–19, 2007 C. Kut, C. Chen, Y. Le, R. Susil, J. Wong, R. Taylor, “Improving the utility of in-room video camera systems for continuous surveillance of patient motion during radiation treatment”. American College of Medical Physics Annual Meeting (ACMP), 2008 C. R. Maurer, D. L. G. Hill, R. J. Maciunas, J. A. Barwise, J. M. Fitzpatrick, M. Y. Wang, “Measurement of Intraoperative Brain Surface Deformation Under a Craniotomy”. Proceedings of the First International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI); Lecture Notes in Computer Science; Vol.1496; p.51-62, Cambridge, MA, USA, 1998 H. Sun, H. E. Lunn, H. Farid, XXX, “Stereopsis-Guided Brain Shift Compensation”. IEEE Transactions on Medical Imaging; Vol.24, No.8, 2005 P. Kazanzides, J. Chang, I. Iordachita, J. Li, C.C. Ling, G. Fichtinger, “Development of an imageguided robot for small animal research”. Journal of Computer Aided Surgery; Vol.12, No.6, p.357365, 2007 J. V. Hajnal, D. J. Hawkes, D. L. G. Hill (editors), “Medical Image Registration”. CRC Press, USA, 2001 J. M. Bach, O.A. Barrera, P. Kazanzides, H. Haider, “Evaluation of the draft ASTM CAOS standard”. Proceedings of the 7th Annual Computer Assisted Orthopaedic Surgery (CAOS) Meeting, 2007 J. J. Craig, “Introduction to Robotics, Mechanics and Control”. Addison-Wesley, Reading, MA, USA, 1986 M. W. Spong, S. Hutchinson, M. Vidyasagar, “Robot Modeling and Control“. John Wiley and Sons Inc., 2006 B. Lantos, “Robot control”. (Robotok irányítása, in Hungarian), Akadémia Kiadó, Budapest, Hungary, 2001 D. J. Bennett, J. M. Hollerbach, “Autonomous calibration of single-loop closed kinematic chainsformed by manipulators with passive endpoint constraints.” IEEE Transactions on Robotics and Automation, Vol.7, Issue:5, p.597-606, 1991 R. E. Kalman, “A new approach to linear filtering and prediction problems”. Journal of Basic Engineering Vol.82, Issue:1, p.35-45, 1960 R. M. du Plessis, “Poor man’s explanation of Kalman Filtering”. Technical Report, North American Rockwell Electronics Group, Anaheim, CA, USA, 1967 G. Welsh, G. Bishop, “An Introduction to the Kalman Filter”. Technical Report TR95-041, University of North Carolina at Chapel Hill, 1995 - 93 - Haidegger: Improving the Accuracy and Safety of a Robotic System for Neurosurgery [109] M. H. Moghari, P. Abolmaesumi, “Point-based Rigid-body Registration Using an Unscented Kalman Filter”. IEEE Transactions on Medical Imaging; Vol.26, Issue:12, p.1708 – 1728, 2007 [110] Tyler Sheffield, “EE 525 -Kalman Filter Final Project”. Technical Report, Brigham Young University, 2005 [111] J. R. Martin, R. K. Erickson, D. N. Levin, C. A. Pelizzari, R. L. Macdonald, G. J. Dohrmann, “Frameless stereotaxy with real-time tracking of patient head movement and retrospective patientimage registration”. Journal of Neurosurgery; Vol.85, Issue:2, p.287-292, 1996 [112] B. Westermann. R. Hauser, “Online Head Motion Tracking Applied to the Patient Registration Problem”. Computer Aided Surgery; Vol.5, Issue:3, p.137-147, 2000 [113] M. Moakher, “Means and Averaging in the Group of Rotations”. SIAM Journal on Matrix Analysis and Applications; Vol.24, Issue:1, p.1-16, 2002 [114] B. W. O’Malley Jr., G. S. Weinstein, “Robotic Skull Base Surgery; Preclinical Investigations to Human Clinical Application”. Archives of Otolaryngology—Head and Neck Surgery; Vol.133, Issue:12, p.1215-1219, 2007 Legal disclosure The figures and images were either used with permission and copied form the given source or the author owns their copyright. The charts and drawings were created within the frames of the diploma project by the author. Haidegger, Tamas 2008 - 94 -
