Multi-Link Robotic Endoscope with Haptic FeedbMAACFIsU
dMASSACHUSETI'
-OF TECHNOLOG y
by
.I
Jiahui Liang
.[JUL 3 0 2014
Submitted to the Department of Mechanical Engineering
in partial fulfillment of the requirements for the degree of
LIBRARIES
Bachelor of Science in Mechanical Engineering
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June 2014
@ Massachusetts Institute of Technology 2014. All rights reserved.
Author
redacted
.Signature
Department of Mechar al E ineering
May 9, 2014
Signature redacted
Certified by............................................
Ian W. Hunter
Hatsopoulos Professor of Mechanical Engineering
Thesis Supervisor
Signature redacted
A ccepted by .........................................................
Annette Hosoi
Professor of Mechanical Engineering
Undergraduate Officer
2
Multi-Link Robotic Endoscope with Haptic Feedback
by
Jiahui Liang
Submitted to the Department of Mechanical Engineering
on May 9, 2014, in partial fulfillment of the
requirements for the degree of
Bachelor of Science in Mechanical Engineering
Abstract
This thesis outlines the design of a multi-link continuum robotic endoscope and the
design of a single-axis haptic controller for navigation. The multi-link design presented in this thesis is modular such that the length of the endoscope can be easily
modified for different applications. The final prototype contains 7 endoscope units,
has a mass of 157 g, is 0.91 m long and 15 mm in diameter. Each endoscope unit
contributes 2 degrees of freedom to the robot; altogether, the current design has 14
degrees of freedom and is capable of navigating through convoluted paths while minimizing physical contact with the surrounding walls. The single-axis haptic controller
designed to drive the robot consists of a user input interface and a motor feedback
system. The haptic controller has good command following ability and it is able to
provide force feedback of up to 4.6 N with a 5 V input to the user during operation.
Thesis Supervisor: Ian W. Hunter
Title: Hatsopoulos Professor of Mechanical Engineering
3
4
Acknowledgments
This past four years at MIT has been the best time of my life. Here, I have
grown from a naive teenager to a responsible engineer and I have met many friends
whom I will never forget. It has taken me a long time to get to where I am today and
I have learned a lot from this process. I am very thankful to the people who have
helped me throughout my MIT experience and I would like to express my gratitude
towards them.
First, I would like to thank Professor Ian W. Hunter for providing me the
opportunity to join the MIT BioInstrumentation Lab and the opportunity to work
on the robotic endoscope project. Thank you so much for your support and guidance.
I would like to thank my mentor Ellen Yi Chen for her mentorship, support,
and guidance throughout my thesis and research. Ellen has taught me how to become
a better engineer and researcher. I have learned many invaluable lessons from her,
both professionally and personally.
I would like to thank the members of the BioInstrumentation Lab. Ashley,
Ashin, Cathy, Geehoon, John, Kate, Mike, and Nick, it was a great pleasure getting
to know you all.
I would like to thank Rui Jin for his help and support throughout the writing
of this thesis. I will truly miss the time we spent baking together. I would also like
to thank Qian Long. Thank you for always being there for me.
I would like to thank the people whom I have shared many happy memories
with. Bryan, Emily, Kelly, Pon, Sharon, and Stephen, thank you for your friendship
and support.
Lastly, I would like to thank my Mom, Dad, and Brother for their love and
encouragement.
Thank you for believing in me. I would not have gotten this far
without your support.
5
6
Contents
Contents
7
List of Figures
9
1
Introduction
11
2
Background
15
2.1
Commercial Endoscope . . . . . . . . . . . . . . . . . . . . . . . . . .
15
2.2
Snake Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
2.2.1
Design Architecture. . . . . . . . . . . . . . . . . . . . . . . .
17
2.2.2
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
2.3
3
Haptic Technology
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
2.3.1
Design of Haptic Devices
. . . . . . . . . . . . . . . . . . . .
19
2.3.2
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
Multi-Link Endoscope Design
21
3.1
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
3.1.1
Actuation Module . . . . . . . . . . . . . . . . . . . . . . . . .
21
3.1.2
Continuum Bending Section . . . . . . . . . . . . . . . . . . .
23
3.1.3
Modeling of a Single Endoscope Unit . . . . . . . . . . .. . . .
24
Electrical Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
3.2.1
Circuit Board Design . . . . . . . . . . . . . . . . . . . . . . .
25
3.2.2
Control Strategy
. . . . . . . . . . . . . . . . . . . . . . . . .
28
3.2.3
Communication . . . . . . . . . . . . . . . . . . . . . . . . . .
29
3.2
Mechanical Design
7
4
3.3
Final Assembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
3.4
13 mm Diameter Endoscope Design .....
31
3.5
Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
3.5.1
Endoscope Dynamics . . . . . . . . . . . . . . . . . . . . . . .
31
3.5.2
Multi-Link Endoscope Performance . . . . . . . . . . . . . . .
33
Haptic Controller Design
4.1
5
...................
37
Mechanical Design .......
............................
4.1.1
Design Overview
4.1.2
Modeling of the Mechanical Component ....
38
........................
. 38
............
4.2
Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3
Overview of System Apparatus
4.4
Control Strategy
4.5
39
. 41
. . . . . . . . . . . . . . . . . . . . .
42
. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44
Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
4.5.1
Command Following . . . . . . . . . . . . . . . . . . . . . . .
45
4.5.2
Force Output with Controller On . . . . . . . . . . . . . . . .
46
4.5.3
Response to Obstacles . . . . . . . . . . . . . . . . . . . . . .
47
Conclusions and Recommendations
49
5.1
G oals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
5.2
Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
50
Bibliography
53
8
1-1
Disposable Robotic Endoscope . . .
2-1
Image Formation in Rigid Endoscope
. . . . . . . .
16
2-2
Commercial Endoscope . . . . . . .
. . . . . . . .
17
2-3
Unified Snake Robot
. . . . . . . .
18
2-4
Haptic Feedback Diagram
. . . . . . . .
19
3-1
Motor Module . . . . . . .
. . . . . . . .
22
3-2
Actuation Diagram.....
. . . . . . . .
23
3-3
.
List of Figures
Bending Module . . . . . .
. . . . . . . .
24
3-4 Force Diagram of Endoscope
. . . . . . . .
25
. . . . . . .
. . . . . . . . . . . . .
26
.
.
.
.
12
Motherboard
3-6
Daughterboard
. . . . . .
. . . . . . . . . . . . .
27
3-7
Endoscope Control Strategy . . . . . . . . .
. . . . . . . . . . . . .
28
3-8
Communication Architecture of the Multi-Link Endoscope . . . . . .
29
3-9
Final Assembly of the Multi-Link Endoscope
. . . . . . . . . . . . .
30
. . . . . . . . . . . . .
31
. . .
. . . . . . . . . . . . .
32
3-12 Normal Force at Endoscope Tip . . . . . .
. . . . . . . . . . . . .
33
3-13 Results From Uncoiling Experiment.....
. . . . . . . . . . . . .
35
3-14 Results From Follow-the-Leader Experiment
. . . . . . . . . . . . .
36
.
.
.
3-5
.
. . . . . .
.
3-11 Closed-Loop Response of Endoscope
.
3-10 13 mm Endoscope Unit Design
Haptic Actuator . . . . . . . . . . . . . . .
39
4-2
Optical Encoder . . . . . . . . . . . . . . .
39
.
.
4-1
9
Moment Diagram of Motor ........
. . . . . . . . . . . . . . . . .
40
4-4
Apparatus of Haptic System . . . . .
. . . . . . . . . . . . . . . . .
42
4-5
Block Diagram of Haptic System
. .
. . . . . . . . . . . . . . . . .
43
4-6
Haptic User Interface . . . . . . . . .
. . . . . . . . . . . . . . . . .
44
4-7 Command Following of Haptic System
. . . . . . . . . . . . . . . . .
45
4-8
Force output of Motor . . . . . . . .
. . . . . . . . . . . . . . . . .
46
4-9
Obstacle Experiment I . . . . . . . .
. . . . . . . . . . . . . . . . .
47
4-10 Obstacle Experiment II . . . . . . . .
. . . . . . . . . . . . . . . . .
48
.
.
.
.
.
.
4-3
10
Chapter 1
Introduction
Flexible robotic endoscopy is an important field of research in medical instrumentation. This technology can be applied to colonoscopies as well as minimally invasive
surgeries (MIS) and natural orifice transluminal surgeries (NOTES). It has the potential to help surgeons navigate to sites of interest and position tools during surgical
procedures while reducing risk of perforations and abrasions.
Most commercial endoscopes commonly used for procedures are hand-driven with
long Bowden cables connecting the manipulator handles to the bending tip of the
device. These cables are difficult to control due to variable friction and backlash.
In addition, Bowden cables take up space in the non-bending sections of the endoscope, making it challenging to increase the number of bending axes. Hand-driven
endoscopes can be improved by placing an actuation unit with small motors near
the bending tip of the device [1] [2] [3] [4]. The BioInstrumentation Laboratory at MIT
has developed a motorized endoscope using this principle. The motorized endoscope
shows a significant reduction in cable friction and backlash, making control much
easier [1][2][3][4].
Another approach to improving the endoscope is to repeat several bending units to
make a hyper-redundant robot similar to a snake robot [6] [7] [8] [9]. This type of robot
is good at navigating narrow, convoluted paths [8]. It can also generate its own
11
Turning
Modules
Motor Module
Endoscope Body
10 mm
Tip Module
Figure 1-1: Robotic endoscope developed at the MIT BioInstrumentation Lab. Figure
reproduced from [5].
propulsion as it traverses through the colon or makes its way to a targeted location
during minimally invasive surgeries. A hyper-redundant endoscope is also capable
of performing motions outside of simple bending such as direct translation along a
surface, making procedures easier to complete for the surgeon [1]. As the number
of bending units along the endoscope increases, it becomes increasingly difficult to
control the device, thus it is important to have a proper path planning and ontrol
interface to automate motions for the user. It is also important for the user to get an
instant force response from the endoscope to prevent perforation.
This thesis proposes using haptic sensing to provide this instant response. Haptic
sensing is a promising technology that creates a virtual sense of touch for a user that
is distant from the actual touch environment. The incorporation of haptic feedback
into medical instruments restores the hand-eye coordination of a surgeon performing
a remote procedure by providing immediate feedback.
12
This thesis outlines the design of a modular continuum robotic endoscope and the
design of a single-axis haptic controller. This robotic endoscope is composed of multiple independent bending units. Each unit is 15 mm in diameter and contains its own
actuation module, continuum bending section, and electronics for direction sensing
and driving. Four short monofilament cables are enclosed in each unit to help the
assembly achieve a full 360 deg rotation, thus eliminating the need for long Bowden
cables. The single-axis haptic controller includes a 1:19 geared, back-drivable motor
and two optical encoders for feedback. A commercial microcontroller board is used
for data processing and to direct feedback for the haptic controller. The performance
of both devices are tested and measured and the results are discussed in this thesis.
The goals this thesis focuses on are:
" Create a multi-link robotic endoscope similar to a snake robot.
* Test coordinated motions on the multi-link endoscope.
* Incorporate haptic feedback into interaction with the endoscope.
" Achieve good synchronization between each component in the haptic interface.
* Quantify force output of the haptic actuator.
13
14
Chapter 2
Background
In order to design a better tool for endoscopy, it is important to understand the architecture of existing tools. This chapter discusses the design of commercial endoscopes,
describes snake robots and introduces basic haptic technology and its application.
2.1
Commercial Endoscope
Endoscopes are developed specifically for visual diagnostics and therapy. Most commercial endoscopes can be used to illuminate and examine hard-to-access areas inside
the human body to facilitate diagnosis of hidden diseases [10]. The design architecture of most commercial endoscopes consists of a control handle and a long tube with
a built-in imaging system [10][11].
There are two common types of commercial endoscopes: rigid endoscopes and flexible endoscopes.
A rigid endoscope consists of a long metal tube attached to an
observation channel [10]. Inside the metal tube, a series of prisms and lenses is used
for image formation, transport, and magnification. The metal tube also contains a
working channel for tool insertion, allowing surgeons to work in real-time [10]. Rigid
endoscopes offer the best image quality and resolution [10]; however, it is difficult to
use such devices to reach many areas inside the human body.
15
Figure 2-1: Diagram illustrating the image formation in a rigid endoscope. Figure
reproduced from [10].
In contrast to rigid endoscopes, flexible endoscopes are more agile. A flexible endoscope consists of a long flexible tube with a short bending segment located at the
tip [10]. Long Bowden cables are used to connect the control handles to the bending
segment, allowing the surgeon to control the directional movement of the endoscopes
distal end [10][11]. Flexible endoscopes utilize digital cameras or fiber optic bundles
for imaging, thus eliminating the need for a rigid tubing. The working channel of
a flexible endoscope is usually integrated [10]. A flexible endoscope allows for tool
insertion, camera wire connection, as well as Bowden cable passage for controlling
the bending tip.
16
Figure 2-2: Olympus GIF-H180 conventional endoscope.
[11].
2.2
Figure reproduced from
Snake Robot
Despite lacking rigid skeletons and extremities, snakes are capable of changing the
shape of their body to perform a wide range of locomotion such as crawling, burrowing, climbing, and swimming [6]. Because of the snakes elongated form and lack of
legs, it has a compact cross-section to length ratio which allows it to move into tiny
holes and navigate through complex and convoluted paths [6].
2.2.1
Design Architecture
The design architecture of a snake is a simple structure repeated in a chain. Snakeinspired robots are constructed by chaining together multiple independent links to
create a hyper-redundant structure [6][7][8][9], resulting in many degrees of freedom.
Each link in the chain is typically equipped with an actuation unit and it can move
independently. These links can be a rigid structure or a continuum structure or a
combination of both depending on the final application[6] [9].
17
The redundant design of the snake robot makes the device resistant to failure [6] [9].
When one of the links fails or is destroyed, the robot can continue to operate using
the other links. Moreover, the simplicity of the snake robot design leads to a compact
structure because there is not a fragile appendage that can be broken easily [6].
Figure 2-3: The Unified Snake Robot developed at Carnegie Mellon University. Figure
reproduced from [9].
2.2.2
Applications
The locomotive flexibility of snake robots gives them an advantage to navigate in
many diverse environments, including uneven ground, slopes,
channels, and pipes [9].
This feature makes the robot most useful in tasks such as urban searches, rescue
missions, and industrial inspections [6] [9].
The range of tasks the robot can accomplish depends on the size of the robot, which is
usually dictated by the embedded actuation unit. With the recent innovation in the
manufacturing of smaller and more powerful actuators, it is possible to make snake
robots smaller, making it practical to apply them to the field of medical instrumentation.
18
2.3
Haptic Technology
Haptics commonly refers to the form of interaction that only involves touching. Haptic
technology takes advantage of the sense of touch by providing tactile feedback [12],
including forces and vibrations to users in order to help them better realize distant
or virtual environments, thereby enhancing their ability to control remote machines
or devices.
2.3.1
Design of Haptic Devices
Although haptic technology has undergone many generations of design architecture
change, the basic theory of operation remains similar. When an external factor such
as a user input or a sudden change in the environment of the object being controlled
is detected by the system, an electrical stimulus is generated. This electrical stimulus
will then trigger the actuator to generate a force back to the user. This force feedback
path is what most haptic designs rely on.
Sensor 1
Signalo
Microcontroller
I Sensor :N kH
Rgltr
HAtct
Auao
Figure 2-4: A simplified diagram for typical haptic feedback.
There is a wide range of actuators that can be used in building a haptic system,
including electromagnetic motors, electro-active polymers, piezoelectric and electrostatic actuators [13][14][15]. These actuators all have different working frequencies
and force output limit [14], so it is critical to select the actuator that will complement
the system being developed.
19
In addition to the traditional method of incorporating an actuator for force feedback,
recent innovation in haptic technology enables a new technique called reverse electrovibration, also known as virtual touch, which allows the users to perceive the textures
and contours of remote objects through manipulating the electrical field around the
fingers of the user [15]. This electrical field is created by injecting a weak current on
the user and the tactile feedback the user senses depends on the waveform, frequency,
and amplitude of the input signal[15]. Besides reverse electrovibration, the latest
haptic designs also include pressure sensing. The feedback experienced by the user is
dependent on the amount of pressure he or she exerts on the sensor surface [16].
2.3.2
Applications
The application of haptic technology in commercial products is at a very matured
stage, from personal computers, electronic displays, video game controllers, to mobile devices, the incorporation of haptics into these devices has greatly enhanced the
user experience and interaction with the products. However, there are more possible
applications of this technology beyond consumer goods.
The applications of haptics in medicine are still being investigated by researchers
across different institutes. From virtual reality medical simulation to design of haptic
aided medical devices, haptic technology has shown potential to help increase clinical
proficiency while decreasing medical errors and cost.
20
Chapter 3
Multi-Link Endoscope Design
This chapter outlines the design of a multi-link continuum robotic endoscope comprising multiple bending units connected in series. The mechanical components, electronics, and communication protocol of each endoscope unit are designed to be modular
so that the length of the entire endoscope can be easily modified by adding or removing extra units. Experiments are performed on a single endoscope unit as well as on
the multi-link design and the results are included in this chapter.
3.1
Mechanical Design
Each endoscope unit is 15 mm in diameter and contains its own actuation module
and continuum bending section. Four short monofilament cables are enclosed in each
unit to drive the system, thus eliminating the need for long Bowden cables.
3.1.1
Actuation Module
The actuation module for each endoscope unit in the multi-link continuum design
is modified from the motorized system developed at the MIT BioInstrumentation
Laboratory for a modular disposable robotic endoscope [1][2][3].
21
The new actuation module has a modified rotary-to-linear transmission mechanism
which reduces assembly time and moves the tool passage closer to the center of the
unit. Several critical features of the module such as height and location of the motor
bearings are also modified to house a pair of 6 mm diameter gearmotors with a ratio
of 1:136.
These motors are different from the ones used in the original design in
that they have taller gear boxes and built-in shafts, so it is critical to customize the
actuation module to accommodate these changes. The final actuation module is 35
mm in length and 15 mm in diameter.
Figure 3-1: CAD model (left) and 3D printed prototype (right) of the 15 mm actuation
module. The 3D printed prototype is produced using stereolithography.
The main components inside the actuation module are two miniature gearmotors
and four monofilament cables. Each gearmotor has two cables wound in opposite
directions about its shaft [2] [3]. When one of the cables is pulled by the motor, the
other cable is released, causing the endoscope tip to bend [2] [3]. Each of the cardinal
directions can be bent up to 180 deg and two motors acting in tandem allows the
endoscope to achieve a full 360 in all directions [2][3].
22
Monofilament
Control Cable
Turning
Module
5
~C
m
X Motor
Tansmission
Converter
Gearbox
4?
V
Figure 3-2: Components of the motor actuation system developed at MIT Biolnstrumentation Lab (top) and the actuation module for each endoscope unit in the
multi-link design (bottom). The actuation unit includes a built-in rotary-to-linear
transmission mechanism that converts the rotary motion of the geared motors into
linear motion in the two opposing cables.
3.1.2
Continuum Bending Section
The earlier versions of robotic endoscope developed at the MIT BioInstrumentation
Laboratory utilize turning modules that can freely rotate. This unrestricted rotation
leads to undesired dynamics in the system, making it difficult to predict.
The endoscope unit in the multi-link design incorporates a twisted-joint mechanism in
the continuum bending section. Each turning module has a pair of small connectors
added to its top and bottom surfaces. The twisted-joint mechanism increases the
stiffness of the continuum bending section, so a higher-geared motor, such as the one
discussed for the actuation module, is needed to drive the system. An extension spring
with low stiffness is inserted through the 5 mm hole at the center of the continuum
bending section to serve as a channel for tools and wires.
23
Figure 3-3: CAD model (left) and 3D printed prototype (right) of the 15 mm continuum bending section. The edges of each turning module are shaved off to achieve a
greater bending angle. In the final assembly, the turning modules are held together
by M1 screws. The 3D printed prototype is produced using stereolithography.
3.1.3
Modeling of a Single Endoscope Unit
The architecture of the endoscope unit in the multi-link design is similar to the disposable robotic endoscope developed at the MIT BioInstrumentation Laboratory. Figure
3-4 shows a force diagram that has been created to model the disposable robotic endoscope.
-
L8
Dm + D,
Lh
V - KeN
Rr
KO - 13
-
The full dynamics of the model is described by the following equation [2],
(3.1)
where Fxt is the force exerted by the endoscope tip, N is the gear ratio, Ke and
Kt are motor constants, r is the radius of the motor shaft, R is the resistance of the
motor, 0 is the angle of the motor shaft, K# is the spring force FK, Bf is the damping
force FB, and J is the inertial force Fj [2].
24
V011P
DM
I
Fext
II
N
Figure 3-4: Force diagram of the disposable robotic endoscope developed at MIT.
Figure reproduced from [2].
The endoscope unit in the multi-link design incorporates motors with higher gearing
and different turning module architecture. The new motors used in the actuation
module are not backdrivable, therefore the system does not have any spring force
or damping force. The twist-joint mechanism implemented for the bending section
restricts rotation. The dynamic equation for the new model can be modified from
Equation 3.1 [2],
1
V -KeO/N
Rr
Fext = - (N)Ke-
0
3.2
-cK
J4.
(3.2)
Electrical Design
Each endoscope unit contains its own electronics for direction sensing, driving, and
communication with the motherboard. The electrical design can be broken down into
3 main categories: printed circuit board design, control strategy, and communication
method.
3.2.1
Circuit Board Design
The framework of the electronics architecture for the multi-link robotic endoscope is
adapted from what has been used in the disposable robotic endoscope project at the
25
MIT BioInstrumentation Laboratory, but the layout of the printed circuit boards is
different from the original and new circuit elements are added to the board design.
The electronics consist of a motherboard located outside of the robot and 7 daughterboards, each of which is located on the actuation module of the respective endoscope
unit. The motherboard contains a STM32F1-series microcontroller [20] to communicate with all the daughterboards via serial peripheral interface (SPI) and a FTDI
chip [21] for data transmission with the computer. The motherboard also supplies
power to all the electronics.
E
E
36 mm
Figure 3-5: Layout of the motherboard (left) and the actual populated board (right).
The motherboard is a four-layer board with circuit elements only on the top layer.
26
Each daughterboard contains a STM32F1-series microcontroller [20] to communicate
with the motherboard, two small H-bridge motor drivers [22] that can output up to
5 V, and an Invense MPU-6000 MEMS gyroscope and accelerometer [23].
Back
Front
I
E
E
10 mm
Figure 3-6: Layout of the daughterboard (left) and the actual populated board (right).
The daughterboard is a compact four-layer board with circuit elements on both the
top and bottom layers.
27
3.2.2
Control Strategy
A closed-loop controller is implemented to control the desired bending angle of the
endoscope unit in both directions. This controller takes in the error signal between
the desired angle and actual angle of the endoscope as input, feeds it through a
proportional-integral-derivative controller and performs a dead band compensation
on the result. The controller then converts the compensated value into a PWM
signal that is sent to the H-bridge motor driver.
dDead
band Comp.
D(s)
Plant P(s)
Figure 3-7: Control overview of each endoscope unit.
The transfer function for the controller can be derived from the above simplified
diagram,
0e(s) = {[Od(s) - Oe(s)][Kp + I(s) + D(s)] + C(s)}P(s).
(3.3)
Rearranging the above equation, we can obtain the transfer function for Oe(s),
-
[Kp + I(s) + D(s)]P(s)Od(s)
1 + [(Kp + I(s) + D(s)]P(s)
P(s)C(s)
1 + [(Kp + I(s) + D(s)]P(s)
(3.4)
(
e(s)
The proportional, integral, and derivative gains used in the controller are 65, 10, and 0
respectively. The value for the dead band compensation is experimentally determined
to be 0.8 V.
28
3.2.3
Communication
The motherboard communicates with the daughterboards in the chain through SPI
with all daughterboards sharing the same data bus. When the motherboard sends
commands down the SPI line, it always starts with a unique ID number that represents each daughterboard. This ID number is a cue for the daughterboard to report
their integrated angle data and carry out different actions. Since the angle of each
endoscope unit is determined relative to its position with the previous unit. The
motherboard also passes along the angle data from the previous unit in the chain to
the current unit. The overall communication architecture is shown in Figure 3-8.
Daughterboards
Camera M ,, 5
4
Motherboard
Command
Daughterboard
Response
Figure 3-8: Overview of the communication architecture for the multi-link continuum
robotic endoscope. Figure modified from [1].
29
3.3
Final Assembly
The multi-link design discussed in this chapter is modular. Each endoscope unit
is independent and multiple units can be easily put together to resemble a hyperredundant robot as shown in Figure 3-9. The final assembly includes 7 endoscope
units, has a mass of 157 g, is approximately 0.91 m long and 15 mm in diameter.
The actuation modules and turning modules in the continuum bending section are
produced using stereolithography and the assembly is held together by M1 screws.
Wires for power and communication go through an extension spring inserted through
the 5 mm hole at the center of the continuum bending section and the actuation
module.
--
-
Motherboard
Connector
Distal Tip
50 mm
Segment
Connector
Motor Module Front
Motor Module Back
X Motor
Y Motor
Tool Channel
Transmission
Gearbox
'1-- mm
Daughterboard
Control Cables
Turning Modules
Figure 3-9: Final assembly of the multi-link continuum robotic endoscope. Figure
modified from [1].
30
3.4
13 mm Diameter Endoscope Design
The final diameter of the multi-link design is 15 mm without the outer sheath, which
is bigger than many conventional endoscopes. To improve the usability of the robot,
a 13 mm diameter model is proposed. The CAD model and the printed circuit board
design for the 13 mm diameter endoscope unit is shown in Figure 3-10.
013mm
(7x27)mm2
Figure 3-10: The design of an endoscope unit a diameter of 13 mm and the corresponding PCB layout of the daughterboard.
3.5
Experimental Results
The closed-loop response of a single endoscope unit and its normal force at tip are
measured. The performance of the multi-link endoscope design is tested using paths
generated for planned motions and the results are compared with simulation data.
3.5.1
Endoscope Dynamics
Closed loop dynamics of a single endoscope unit are investigated. Each endoscope
unit needs to be able to reject external disturbance in order to achieve accurate control of the system. In this experiment, the endoscope is commanded to positive 90
deg, then to negative 90 deg. Then, external disturbance is introduced to the system
31
when the endoscope has reached its final destination.
-
X Angle
-
Y Angle
Y Input
X Input
120
80
M,
7FW
Co
40
0
-40
-80
I IffI
-
So
-120
0
5
y
10
15
20
25
30
Time (s)
-X
Voltage -Y
Voltage
4
2
-4
0
I
i
5
10
I
15
I
I
20
25
30
Time (s)
Figure 3-11: The closed-loop response to a command in a single bending axis, and
disturbance rejection capabilities (top). The corresponding voltage plot is also shown
(bottom).
32
The normal tip force exerted by the endoscope is measured by holding the first turning
module in the continuum bending section fixed. The endoscope is then commanded
to turn to different angles and the forces against the tip at these angles are measured.
Results from this experiment are included in Figure 3-12.
0.05
0L.
0.04
0.03
~
. 0.02
I
-u
0.01
0
0
-
z
0
80
40
120
160
Angle (deg)
Figure 3-12: Normal tip force as a function of bending angle.
3.5.2
Multi-Link Endoscope Performance
The final multi-link assembly has 14 degrees of freedom, 2 from each endoscope unit,
so it is capable of performing motions outside of simple bending. For the multi-link
design, two coordinated paths, uncoiling and follow-the-leader, are generated using
incremental waypoints and the experimental results are compared with simulation
data. In each experiment, the endoscope is tested on a metal table covered with a
thin teflon sheet to reduce friction. White circular obstacles are placed on the metal
table to create the path. Due to the mass of the conjoining endoscope units, the
bending speed of the robot is much slower than a single unit. Bending speed varies
depending on the location of the endoscope unit along the chain and ranges from 35
deg/s to 200 deg/s.
33
The uncoiling motion can be used for inserting the endoscope into the human body
and obtaining lateral scans of the colon walls [1]. The experimental data from the
uncoiling motion are shown in Figure 3-13. The desired angles are generally achieved
by the endoscope units but sometimes the error can be as large as 30 to 40 deg. This
is due to external friction and the varying load at the tip of each endoscope which
prevent the robot from achieving the exact position. Despite deviating from the desired angle at the beginning, the robot eventually catches up and achieves the desired
configuration. Another set of motions that moves each unit back and forth for 10 deg
about the center is added to straighten the robot.
The follow-the-leader motion can be used for navigating convoluted paths such as
the human colon [1]. Due to redundancy of the multi-link design, the robot is able
to avoid colon walls during procedure. The experimental data from follow-the-leader
motion in a phantom colon are shown in Figure 3-14. The phantom colon used in
the setup is constructed using white obstacles. During the experiment, the robot is
inserted by hand at a constant speed. The large errors shown in the graph is due to
the fact the all endoscope units not within the path have their controller turned off.
Overall, the robot does a good job of replicating the desired shape without making
much physical contact with the surrounding wall.
34
-Unit
1
-Unit
2 -Unit
4 -Unit
3 -Unit
6
5 -Unit
Unit 7
200
150
L
100
C'
50
C'
0
To
wz~
-----
-50
-100
-150
-200
15
10
5
0
Time (s)
Simulation of Path Plan
N
1000f
1000;
1000
10001
800
800
800
8001
600'
600
600
600
400,
4001
400!
4001
2001
200k
-
200
x (mm)
200
0
x (mm)
20 0
0
200 0 200
x (mm)
j
x (mm)
Experiment
Figure 3-13: Real time data from the uncoiling experiment (top), simulation data
(middle), and snapshot from video recording (bottom). Figure modified from [1].
35
-Uniti
-Unit
100
2 -Unit
3 -Unit
4 -Unit
5 -Unit
6
Unit 7
75
_cF--
50
25
0
C
-25
110
_ _ A- N F1- -W-e -
-50
-75
-100
0
5
10
15
20
25
Time (s)
Simulation of Path Plan
500
500
400,
400
Transcending
300
200,
100
300
Ascending
Cecum
Descending
Sig
10
0
-100-
0
Rectumf
-200
0
x (mm)
200
200
100
400
400
300
300,
200
200
100
100
-200
0
x (mm)
20 0
0
-100
-200
0
200
-100
x (mm)
20 0
x (mm)
Experiment
Figure 3-14: Real time data from the follow-the-leader experiment (top), simulation
data (middle), and snapshot from video recording (bottom). Figure modified from
[1].
36
Chapter 4
Haptic Controller Design
The current endoscope prototype shows potential for reducing stress and damage to
colon during surgeries, but these benefits will be difficult to realize without a specialized feedback control platform at surgeons end. Unlike open surgeries, procedures
involving the use of long tools, such as an endoscope, deprive surgeons of their depth
perception and sense of touch. In order to restore hand-eye coordination during surgeries, a controller with built-in haptic feedback is proposed.
This chapter outlines the design of a single-axis haptic controller that supplements
the multi-link robotic endoscope discussed earlier. The performance of the haptic
controller is tested with one endoscope unit, and the results are included in this section.
The system allows the user to control the turn angle of an endoscope unit by turning
an associated but mechanically uncoupled control handle. An optical encoder and
associated electronics sense the user input and command the endoscope motion. The
resulting endoscope turn angle is fed back and controls the rotation of a haptic feedback motor. An optical encoder and associated electronics keep the rotation of the
haptic feedback motor in sync with the turn angle of the endoscope. The motor is
loosely coupled to the control handle to provide the user with haptic feedback. For
example, if the user input is greater than the current turn angle of the endoscope,
37
the haptic feedback motor provides torque opposing the user input.
4.1
Mechanical Design
The mechanical components of the haptic controller include a Pololu 1:19 geared,
back-drivable motor [24] and two Avago HEDS 3-channel optical encoders [25] with
500 counts per revolution (CPR) for feedback. The finished device is mounted on
a 6 mm thick acrylic structure and elevated above the ground using pre-cut MK
components.
4.1.1
Design Overview
The mechanical design of the haptic controller can be broken down into two parts:
the user driving system and the motor feedback system. Customized mounting plates
are designed and fabricated to hold the two systems together.The encoder for reading
user input is attached to the shaft of the control handle under the top mounting plate,
and the encoder for reading motor position is attached to the shaft of the motor on
the bottom mounting plate. A soft coupling made of 3 mm-thick heat shrink tubing
is used to hold the control shaft and the motor shaft together. Due to the low stiffness of the thick-walled heat shrink, pins are inserted through both shafts and the
coupling to reinforce the connection.
A backdrivable, geared motor as shown in Figure 4-1 is chosen for the haptic feedback
motor because of its relatively low rotational speed and high torque output. However, the additional gearing causes some undesired dynamics in the system. When
the user turns the control handle, a higher initial torque is required to overcome the
static friction of the gearbox. The same problem occurs when the user reverses the
direction of the control handle.
The performance of the haptic controller depends on how well the encoders are aligned
about the control shaft and the motor shaft. When adjusting the height of the encoder
38
disk, it is important to keep the disk at approximately the center of the optical device.
To achieve this precise alignment, a special tool made from a 6 mm thick acrylic sheet
is used.
Figure 4-1: Pololu Geared motor used in the haptic device. Figure reproduced from
[17].
Figure 4-2: Avago encoder used in the haptic device. Figure reproduced from [18].
4.1.2
Modeling of the Mechanical Component
In order to better implement control strategies on the haptic device, it is important
to understand the dynamics of the mechanical component. Figure 4-3 illustrates the
moment diagram of the system.
Using basic motor dynamics, the torque outputted by the motor at its shaft can be
calculated,
e= KeOI
= V -iIR
39
(4.1)
Gout
TM,GM
R
V
e
Figure 4-3: Moment diagram of the haptic component.
Rearrange the variables to get,
V - KeM/N
(4.2)
R
Motor torque is proportional to the input current with constant Kt,
V
-
KeOM/N)
R
R
.
Tm = NKti = Kt(
(4.3)
By balancing all the torque contributions, the dynamic equation of the system can
be derived,
Z T = TM + B(OM - Oout) + K(OM *
_
V
JOut + Bbout + KO0 ut = NKt( V
Oout) - J Out,
(4.4)
-KeOM/N)
R
+ BOM + KOM.
(4.5)
Assume that the input voltage is 0 V. By rearranging the dynamic equation, the
transfer function of the system is obtained,
OOt (s)
(B - KtKc)s + K
OM(s)
Js2 +Bs+K
At steady state, the motor angle Om will be the same as the user angle 0 out.
40
(4.6)
4.2
Electronics
The implementation of haptic feedback adds multiple new elements to the overall
system. The current motherboard design is not able to provide enough input/output
ports to support communication between these new elements, so a commercial microcontroller board, the STM32F3 Discovery Board [26], is used to replace the custom
motherboard for data processing and to direct feedback of the haptic controller. Since
this commercial board does not have built-in serial data communication capability, a
FTDI board [21] is added to the system to support data transmission with the computer. In addition, a full-bridge motor driver [27] that can take in PWM signal from
the microcontroller board and output up to 36 V is used to run the haptic feedback
motor.
41
4.3
Overview of System Apparatus
The final setup of the system includes the haptic controller, a mounting board that
includes all the electronics, and two endoscope units are shown in Figure 4-4. The
setup is placed on a metal table covered with a thin teflon sheet to reduce friction between the endoscope units and the contact surface. A block diagram that summarizes
the interaction between each components is shown in Figure 4-5.
H-Bridge
Motor Driver
Serial Data Transmission
9I
Endoscope
IP
A
Figure 4-4: Apparatus for haptic experiments.
42
Mechanical
User
Electronics
Input
Serial Data
Input Shaft
FTDI Board
Encoderl
Computer
( interface Program)
Rx/Tx
Encoder
1
Signal
..
3
.. :STM32F
End16osc11%ope
Encoder 2
Signal
1PWM
Encoder 2
Motor
. --
--
Voltage
Motor Driver
icommand
Figure 4-5: An overview of the different components in the haptic apparatus.
43
4.4
Control Strategy
A closed-loop controller with multiple inputs and outputs is generated to direct force
feedback to the user. The controller takes in angle input from the user and sends the
command to the selected endoscope unit; at the same time, the controller takes in the
endoscope angle as input and sends it to the haptic motor. The three positions: user
angle, endoscope angle, and motor angle are constantly matched with each other.
An overview of the haptic interface is shown in Figure 4-6. The dynamics of the
endoscope and the haptic controller are described in the earlier sections.
I1
Control Handle
Desired Endoscope
Angle
Encoderl
Endoscope
Controller
Endoscope
Dynamics
Soft Coupling
EncoderMotor
Encoder
2IMeasured
Hpi
Angle
Haptic Motor
Control Motor
Controller
Measured
_ Desired Haptic
Motor Angle
Endoscope Angle
Figure 4-6: Haptic user interface controller.
44
4.5
Experimental Results
The command-following ability of the haptic device is tested and the closed-loop force
output of the haptic motor is measured. The performance of the controller under the
presence of obstacles is also investigated.
4.5.1
Command Following
With the haptic controller, the user can command the endoscope to turn to any angle
between negative 180 deg and positive 180 deg. When the input angle is outside of
this range, the endoscope will hold its desired position at the limit, which will then
cause the motor to provide immediate feedback, warning the user that the robot can
not be bent beyond this point.
Overall, the command-following ability of the haptic controller is adequate, as shown
in Figure 4-7. The errors between the user input angle, the endoscope angle, and the
motor angle are generally less than 5 deg. The time delay between the user input
and the endoscope response is approximately 0.15 s. The time delay between the
endoscope input and the haptic motor response is approximately 0.2 s.
-Endoscope
-User
-Motor
80
60
40
20
Ll/P
0
C-20
4-40
-60
-80
0
10
5
15
Time (s)
Figure 4-7: Command following ability of the haptic system.
45
4.5.2
Force Output with Controller On
The force outputted by the haptic motor is measured at different angle with the
closed-loop controller turn on and the results are shown in Figure 4-8. The values
plotted on the x-axis is the angle difference between the user input angle and the
motor angle.
-4-2V -U-4V
-SV
5
4
U
L
2
0
"
1
I
0
10
20
I
I
I
I
30
40
50
60
70
80
Angle (deg)
Figure 4-8: Motor force output as a function of angle at 2V, 4V, and 5V.
46
4.5.3
Response to Obstacles
Under the presence of obstacles, the angle position of the endoscope becomes constant.
Since the haptic motor constantly matches its position with the endoscope, when the
endoscope gets stuck, the haptic motor seeks to remain at that position. When user
tries to turn the control handle beyond the locking position, force feedback from the
motor is directed to the user.
Figure 4-9 shows the experimental data under the
presence of obstacle. The large angle difference between the user input and motor
position is due to the dynamics of the soft coupling. Because of the low stiffness of
the coupling, user is able to turn the control handle beyond the limit. Once the user
releases the handle, the driving shaft returns to a position that matches the haptic
motor.
-Endoscope
-User
-Motor
120
I Control Handle
100
tX
-o
is Released
80
60
40
====
20
0
0
5
10
15
Time (s)
Figure 4-9: Real time data of the endoscope under the presence of obstacle. The
control handle is later released by the user.
A similar experiment in which the obstacle is removed from the path of the endoscope
is performed. Upon being released from the obstacle, the endoscope continues the
command inputted by the user.
47
-User
-Endoscope
-Motor
80
60
40
20
GJ
Obstacle is
0
====--giRemoved
-20
-40
-60
I~Il
-80
0
5
10
15
Time (sec)
Figure 4-10: Real time data of the endoscope under the presence of obstacle. The
obstacle is later removed from the path.
48
Chapter 5
Conclusions and Recommendations
The multi-link robotic endoscope design presented in this thesis is modular such that
the length of the endoscope can be easily modified for different applications. The final
prototype contains 7 endoscope units, has a mass of 157 g, is 0.91 m long and 15 mm
in diameter. Each endoscope unit contributes 2 degrees of freedom to the robot, and
altogether, the current design has 14 degrees of freedom and is capable of navigating
through convoluted paths while minimizing physical contact with the surrounding
wall. Experiments performed on a single endoscope unit show that the robot has
good command-following ability and it is able to reject external disturbances. The
coordinated-motion experiments performed on the multi-link prototype shows that
with proper planning, it is possible to automate many functions using this design
architecture to simplify surgical procedure in the near future.
The single-axis haptic controller designed to drive the robot consists of a user-driving
system and a motor-feedback system connected together using a soft coupling. A
closed-loop controller with multiple inputs and multiple outputs is implemented in
the haptic interface in order to achieve desired force feedback. Experimental results
show that the haptic controller has good command-following ability and is able to
provide force feedback of up to 4.6 N with a 5 V input to the user during operation.
However, due to the high gearing ratio of the motor used in the controller, the system
exhibits some undesired dynamics that affects the performance of the haptic feedback.
49
5.1
Goals
The original goals of this thesis and the results are listed below.
" Create a multi-link robotic endoscope similar to a snake robot. The finished
prototype consists of independent motorized endoscope units and the length
can be easily modified by adding or removing the units.
" Test coordinated motions on the multi-link endoscope. The multi-link endoscope is able to maintain desired angles generated for the coordinated motions
and the final configuration closely matches the desired shape.
* Incorporate haptic feedback into interaction with the endoscope. The haptic
controller is able detect obstacles and provide feedback to the user.
" Achieve good synchronization between each component in the haptic interface.
The time delay between the user input angle, endoscope angle, and haptic motor
angle is less than 0.2 s.
" Quantify force output of the haptic actuator. The motor is able to provide a
force feedback of up to 4.6 N with a 5 V input.
5.2
Future Work
Future work for this project includes:
" Investigate additional coordinated motions. Automated functions will make
procedures easier to complete for the surgeon and reduce the amount of time
needed per procedure.
* Reduce the endoscope diameter to 13 mm. The current prototype has a diameter of 15 mm without the outer sheath. This is larger than many commercial
endoscopes and not practical for actual surgical procedures.
50
* Use a motor with lower gear ratio in the haptic controller. This is to reduce the
undesired dynamics arising from the static friction in the motor gearbox.
* Expand the haptic controller to drive both axes of the endoscope.
51
52
Bibliography
[1] Y. Chen, J. Liang, and I. W. Hunter, "Modular continuum robotic endoscope
design and path planning," in Proceedings of the 2014 IEEE InternationalConference on Robotics and Automation, [in press].
[2] Y. Chen, S. Tanaka, and I. W. Hunter, "Disposable endoscope tip actuation design
and robotic platform," in Proceedings of the 32nd Annual InternationalConference of the IEEE Engineering in Medicine and Biology Society, pp. 2279-2282,
2010.
[3] Y. Chen, J. M. Oliveira, and I. W. Hunter, "Sensor architecture for a two-actuator
robotic endoscope tip," in Proceedings of the 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 8340-8343,
2011.
[4] Y. Chen, J. M. Oliveira, and I. W. Hunter, "Two-axis bend sensor design, kinematics and control for a continuum robotic endoscope," in Proceedings of the 2013
IEEE InternationalConference on Robotics and Automation, pp. 696-702, 2013.
[5] MIT BioInstrumentation Laboratory, www.bioinstrumentation.mit.edu
[6] J. K. Hopkins, B. W. Spranklin, and S. K. Gupta, "A survey of snakeinspired
robot designs," Bioinspirationand Biommemetics, vol. 4, p. 021001, 2009.
[7] K.-W. Kwok, K. H. Tsoi, V. Vitiello, J. Clarke, G. C. T. Chow, W. Luk, and
G.-Z. Yang, "Dimensionality reduction in controlling articulated snake robot for
53
endoscopy under dynamic active constraints," IEEE Transactions on Robotics,
vol. 29, no. 1, pp. 15-31, 2013.
[8] A. M. Andruska and K. S. Peterson, "Control of a snake-like robot in an elastically
deformable channel," IEEE/A SME Transactions on Mechatronics, vol. 13, no. 2,
pp. 219-227, 2008.
[9] C. Wright, A. Buchan, B. Brown, J. Geist, M. Schwerin, D. Rollinson, M. Tesch,
and H. Choset, "Design and architecture of the unified modular snake robot," in
Proceedings of the 2012 IEEE InternationalConference on Robotics and Automation, pp. 4347-4354, 2012.
[10] Richard Wolf GmbH, www.richard-wolf.com
[11] Olympus Corportation, www.olympusamerica.com
[12] The International Society for Haptics, www.isfh.org
[13] I. Poupyrev, S. Maruyama, and J. Rekimoto, "Ambient touch: designing tactile
interfaces for handheld devices," In Proceedings of the 15th annual ACM symposium on User interface software and technology, pp. 51-60, 2002
[14] A. Mazzone, R. Zhang, and A. Kunz, "Novel actuators for haptic displays based
on electroactive polymers," In Proceedings of the ACM symposium on Virtual
reality software and technology, pp. 196-204, 2003
[15] P. Laitinen, J. Mawnpaa, and N. Center, "Enabling mobile haptic design: piezoelectric actuator technology properties in hand held devices," in Proceedings of
IEEE Intl Workshop Haptic Audio Visual Environments and their Applications,
pp. 40-43, 2006
[16] 0. Bau and I. Poupyrev, "REVEL: tactile feedback technology for augmented
reality," ACM Trans, vol. 31, no. 4, Article 89, 2012.
[17] N. Marquardt, M. A. Nacenta, J. E. Young, S. Carpendale, S. Greenberg, and
E. Sharlin, "The haptic tabletop puck: tactile feedback for interactive tabletops,"
54
In Proceedings of the ACM InternationalConference on Interactive Tabletops and
Surfaces, pp. 85-92, 2009.
[18] Pololu Robotics & Electronics. www.pololu.com
[19] Avago Technologies. www.avagotech.com
[20] STM32F103 Microcontroller Datasheet, www.st.com/web/en/resource/technical/
document/datasheet/CDOO161566.pdf
[21] FT232R Datasheet, www.ftdichip.com/Support/Documents/DataSheets/ICs/DSFT232R.pdf
[22] SI9986 H-Bridge Motor Driver, www.vishay.com/docs/70007/si9986.pdf
[23] MPU-6000 Datasheet, www.invensense.com/mems/gyro/documents/PS-MPU6000A-00v3.4.pdf
[24] Pololu
1:19
Geared
Motor
Datasheet,
www.robotshop.com/media/files/
pdf/datasheet-1442.pdf
[25] Avago HEDS-9100#AOO
500 CPR Optical Encoder Module Datasheet,
www.avagotech.com/docs/AV02-1867EN
[26] STM32F3 Discovery Board, www.st.com/st-web-ui/static/active/jp/resource/
technical/document/data-brief/DM00063389.pdf
[27] SN754411 Datasheet, pdf.datasheetcatalog.com/datasheets/320/500427-DS.pdf
55