Surg Endosc
DOI 10.1007/s00464-008-9894-z
Validated robotic laparoscopic surgical training
in a virtual-reality environment
Dimitrios Katsavelis Æ Ka-Chun Siu Æ Bernadette Brown-Clerk Æ
Irene H. Lee Æ Yong Kwon Lee Æ Dmitry Oleynikov Æ Nick Stergiou
Received: 21 April 2007 / Accepted: 25 February 2008
Ó Springer Science+Business Media, LLC 2008
Abstract
Background A robotic virtual-reality (VR) simulator has
been developed to improve robot-assisted training for
laparoscopic surgery and to enhance surgical performance
in laparoscopic skills. The simulated VR training environment provides an effective approach to evaluate and
improve surgical performance. This study presents our
findings of the VR training environment for robotic
laparoscopy.
Methods Eight volunteers performed two inanimate tasks
in both the VR and the actual training environment. The
tasks were bimanual carrying (BC) and needle passing
(NP). For the BC task, the volunteers simultaneously
transferred two plastic pieces in opposite directions five
times consecutively. The same volunteers passed a surgical
needle through six pairs of holes in the NP task. Both tasks
require significant bimanual coordination that mimics
actual laparoscopic skills. Data analysis included time to
task completion, speed and distance traveled of the
instrument tip, as well as range of motion of the subject’s
wrist and elbow of the right arm. Electromyography of the
right wrist flexor and extensor were also analyzed. Paired
t-tests and Pearson’s r were used to explore the differences
and correlations between the two environments.
A1
A2
A3
A4
A5
D. Katsavelis K.-C. Siu B. Brown-Clerk I. H. Lee N. Stergiou (&)
HPER Biomechanics Lab, University of Nebraska at Omaha,
Omaha, NE 68182-0216, USA
e-mail: nstergiou@mail.unomaha.edu
A6
A7
A8
A9
D. Katsavelis K.-C. Siu I. H. Lee Y. K. Lee D. Oleynikov N. Stergiou
Department of Surgery, University of Nebraska Medical Center,
Omaha, NE, USA
Results There were no significant differences between the
actual and the simulated VR environment with respect to
the BC task, while there were significant differences in
almost all dependent parameters for the NP task. Moderate
to high correlations for most dependent parameters were
revealed for both tasks.
Conclusions Our data shows that the VR environment
adequately simulated the BC task. The significant differences found for the NP task may be attributed to an
oversimplification in the VR environment. However, they
do point to the need for improvements in the complexity of
our VR simulation. Further research work is needed to
develop effective and reliable VR environments for robotic
laparoscopic training.
Keywords da Vinci surgical system Electromyography Kinematics Electrogoniometry Training
It is well established that the reduced invasiveness of laparoscopic surgery results in superior patient outcomes as
measured by a less painful recovery and an earlier return to
a healthy status as compared with traditional open surgery
[1]. However, laparoscopic surgery has also presented
some disadvantages such as poor visualization, difficultness in wrist manipulation, and prolonged abnormal
standing posture. Thus, robot-assisted laparoscopy has
been developed to address these problems [2]. Recent
studies comparing robot-assisted with conventional laparoscopy have shown improved surgical dexterity [3] and
decreased training time for surgical residents [4–6]. However, despite the dramatic increase in the use of robotassisted surgery [7, 8], such technology has not been
widely adopted into the educational curriculum of current
123
Surg Endosc
training programs in general surgery [9]. A survey on
residents’ preference, in a comparative study between the
robotic and conventional techniques, showed that they
would pursue a fellowship in robotic surgical training if it
was available to them, while experienced surgeons
appeared to be reluctant [10]. The relatively high cost of
robot-assisted surgery, the absence of well-constructed
training routines, and the lack of expert technical proficiency may be some of the reasons for the severely limited
robotic surgical educational programs.
Recently, an increasing number of studies have evaluate
the surgeons’ performance in either inanimate [6, 11, 12] or
animate [13, 14] models by using primarily qualitative
measurements. Even though these studies lack direct
applicability to clinical settings [13], their convenience and
low cost make inanimate models essential during the initial
stages of robotic training programs. Virtual reality (VR)—a
form of inanimate model training—is considered an
attractive, inexpensive, and user-friendly mean of motor
learning [15, 16]. VR environments have been widely used
to mimic general and specific surgical tasks, or even entire
surgical procedures [16, 17]. However, there is not enough
evidence to support whether these environments can actually mimic a surgeon’s performance in terms of kinematic
and physiological parameters.
Therefore, the purpose of the present study was to
evaluate a proposed VR environment for robot-assisted
surgery. We compared the VR environment with an actual
training environment using two inanimate tasks that
required significant bimanual coordination. These inanimate tasks have been used by others [12] in an effort to
develop a training module for robotic laparoscopy. We
hypothesized that kinematic and physiological measurements acquired while performing the inanimate tasks in the
VR environment will not differ from those of the actual
environment.
Tasks
The following two inanimate robotic surgical tasks were
performed in this study:
1.
2.
Bimanual carrying (BC), a pick-and-place task: picking up six 15 9 2 mm rubber pieces from a 30-mm
metal cap with the right and left instruments, respectively, and carrying them to the opposite caps
simultaneously (Fig. 1).
Needle passing (NP), a ‘translational’ task: passing a
26-mm surgical needle through six pairs of holes made
on the surface of a latex tube (Fig. 2).
Our robotic VR simulator was constructed using the
simulation software Webots (Cyberbotics Ltd., Lausanne,
Switzerland) based on the specification provided by the
dVSS company, Intuitive Surgical, Inc. The da Vinci
instruments and training task platform were modeled as 3D
objects using SolidWorks (SolidWorks Corp., Concord,
MA, USA; Fig. 3). The simulation was overlaid on the
screen inside the surgeon console of the dVSS, and driven
by the kinematic data from the dVSS robot through
Methods
Subjects
Eight volunteers including six medical students and two
medical research fellows (four men and four women) with
basic surgical knowledge and with no prior experience on
the da Vinci surgical system (dVSS; Intuitive Surgical,
Inc., Sunnyvale, CA, USA) were recruited to participate in
this study. The mean (±standard deviation, SD) age of the
participants was 28.8 (±6.2) years. All participants were
right-handed. Informed consent was obtained from each
subject prior to participation and was in accordance with
the Institutional Review Board of the University of
Nebraska Medical Center.
123
Fig. 1 The bimanual carrying (BC) task in (a) the actual and (b) the
virtual environment
Surg Endosc
Fig. 3 The VR simulator (left) was constructed using the simulation
software Webots (Cyberbotics Ltd., Lausanne, Switzerland). The da
Vinci instruments and training task platform were modeled as 3D objects
using SolidWorks (SolidWorks Corp., Concord, MA, USA). This
simulation was driven by the kinematic data from the robotic operating
console through LabVIEW (National Instruments, Austin, TX, USA)
Measurements
Fig. 2 The needle passing (NP) task in (a) the actual and (b) the
virtual environment
LabVIEW (National Instruments, Austin, TX, USA). All
subjects were instructed to sit at the dVSS console while
performing the training task in both actual and virtual
environments.
Experimental design
All the participants were asked to perform both the BC and
NP tasks in the actual and VR environment in one visit. At
the beginning of the test, the participants received a verbal
explanation about the use of the dVSS and testing procedures from the investigators. The participants familiarized
themselves with the system, but not with the tasks, for
5 min. During this familiarization period, the participants
were allowed to ask questions and receive further verbal
explanation and suggestions from the investigators. After
the familiarization, each participant performed five trials of
each task for each environment. The order of the presentation of the tasks and the environments was randomized.
Temporal and spatial variables with respect to the position
and angular movement of the surgical instruments were
measured for all tested conditions. The variables were
acquired from the force transducers built into the system.
They were extracted at a frequency of 11 Hz by the dVSS
application programmer’s interface. These data sets were
then processed using MATLAB 6.5 (The MathWorks Inc.,
MA, USA) to obtain linear kinematics with respect to the
movement of the surgical instrument tips.
Physiological measurements directly from the participants were acquired with electrogoniometers (Biometrics,
Gwent, UK) that were placed at the wrist and elbow joints
of the right arm and were used to obtain the subjects’
flexion and extension range of motion while performing the
two tasks in the two environments. The muscular activation
of two muscles was also monitored from the participants’
right forearm. These muscles were the flexor carpi radialis
(FCR) and the extensor digitorum (ED), both of which are
superficial and can be monitored by a Delsys surface
electromyography (EMG) system (Delsys Inc., Boston,
MA, USA). Although many other types of movements
(e.g., flexion and extension of the thumb, index and middle
fingers, forearm pronation and supination) and thus many
other muscles are involved, it has been suggested [18, 19]
that the contribution of these two muscles in the BC and
NP tasks are considerably higher than all others. Consequently, measurement of the EMG activities performed by
these muscles was important for the purpose of this study.
123
Surg Endosc
Fig. 4 EMG analysis of the
FCR: (a) graphical
representation of the raw EMG
signal, (b) rectification of the
EMG signal, (c) root mean
square of the normalized EMG
signal, and (d) frequency
spectrum analysis by using fast
Fourier transformation
Surface electrodes were placed over the bellies of these
muscles, as described by Basmajian and Deluca [20]. The
EMG data were collected and extracted at 1,000 Hz
through a LabVIEW-based data acquisition board. These
data sets were processed using MATLAB to obtain normalized EMG outputs (Fig. 4).
Data analysis
For each trial, task completion time (T) was calculated.
Total traveling distance (D) and speed (S) with respect to
the robot surgical instrument tips were calculated for each
trial from the linear kinematics. In addition, wrist flexion/
extension range of motion (WROM) and elbow flexion/
extension range of motion (EROM) were calculated as the
relative angle between the subject’s forearm and hand, and
between upper arm and forearm, respectively.
The EMG signals were then analyzed according to
Narazaki et al. [18] and Judkins et al. [19]. The muscular
activation volume (EMGv) is the integration of the normalized EMG output of the entire task, whereas the
muscular activation rate (EMGr) is the average value of
this task and it can be calculated by dividing EMGv by
completion time. The median frequency (EMGfm) is the
frequency response of each muscle.
123
Statistical analysis
The mean values for the dependent variables of T, D, S,
EMGv, EMGr, EMGfm, WROM, and EROM were compared
between the actual and VR environments for both the BC
and NP tasks with dependent t-tests (p = 0.05) using SPSS
(version 12.0, SPSS Inc, IL, USA). Pearson’s correlation r
was also used to identify correlations of the dependent
variables between the actual and the VR condition.
Results
Significant differences were found for T (p = 0.003), D
(p \ 0.001), WROM (p = 0.002), and EROM (p = 0.002)
for the NP task, with the VR environment always producing lower values than the actual environment (Fig. 5).
The only parameter that was not found to be significantly
different was speed (p = 0.072). On the other hand, no
significant differences were found between the actual and
VR environment for the BC task (Fig. 6). Interestingly,
Pearson’s r revealed similar correlations in the range of
motion in both tasks, with EROM to display the highest
value (r = 0.79 for BC; r = 0.78 for NP) and the WROM
the lowest value (r = 0.21 for BC; r = 0.20 for NP).
Surg Endosc
Fig. 5 Comparison of the time
to task completion, distance and
speed of the instrument tip
during the task, wrist and elbow
range of motion between actual
and VR environment in the NP
task. All parameters except
speed displayed significantly
greater values for the actual
environment (*p \ 0.05)
Fig. 6 Comparison of the time
to task completion, distance and
speed of the instrument tip
during the task, wrist and elbow
range of motion between actual
and VR environment for the BC
task. There were no significant
differences (p C 0.05)
For the EMG, dependent t-tests between the actual and
VR environments revealed that there were no significant
differences for the BC task, but there were significant
differences (p \ 0.05) for the NP task in most parameters,
with the actual environment always producing higher
values (Figs. 7 and 8). Specifically in the NP task, EMGr
for both FCR (p = 0.003) and ED (p = 0.023) were
significantly different. EMGv was also found to be significantly different for both muscle groups (p = 0.001 for
FCR and p = 0.007 for ED). Pearson’s correlations
between the two environments revealed r values from
0.41 to 0.93 for the NP task and from 0.17 to 0.91 for the
BC task (Table 1).
Discussion
The purpose of the present study was to evaluate a proposed VR environment for robot-assisted surgery. We
developed a dVSS robotic VR training simulator and
compared the performance of two actual inanimate surgical
tasks (NP and BC) between dVSS and VR simulator. Our
results showed no significant differences for the BC task in
all parameters between the actual and the VR environment,
with moderate correlations for spatiotemporal parameters
and high correlations for most EMG parameters. For the
NP task, our results showed significant differences in most
parameters with moderate correlations for spatiotemporal
123
Surg Endosc
Fig. 7 Comparison of selected
EMG parameters between
actual and VR environment for
the NP task. All parameters
besides EMGfm displayed
significantly greater values
during the actual environment
(*p \ 0.05)
Fig. 8 Comparison of selected
EMG parameters between
actual and VR environment for
the BC task. There were no
significant differences
(p C 0.05)
Table 1 Pearson’s correlation r values for selected EMG parameters
from the FCR and ED muscle groups between actual and VR environments in both BC and NP tasks
Task
EMGr
EMGv
RMGfm
FCR
ED
FCR
ED
FCR
ED
BC
0.83
0.27
0.91
0.18
0.17
0.57
NP
0.93
0.53
0.86
0.46
0.87
0.41
parameters and high correlations for most EMG
parameters.
The noticeable differences in almost all parameters for
the NP task imply that the VR environment did not sufficiently simulate the actual environment. Values during the
VR environment appeared to be consistently lower.
123
A possible explanation for this behavior is the lack of the
physical interaction feature of the simulated objects in the
VR environment. Needle passing was simulated as a
simultaneous overlap of two touch sensors attached to a
cylindrical tube. Grasping the needle using the instrument
tips and gravitational forces were not available. Therefore,
the absence of physical interaction might have resulted in
different motions of the telemanipulators, reduced muscle
activation, and thus different motion of the instrument tips.
The only parameters that did not yield significant differences were the median frequency of the FCR and ED
muscle groups. Frequency analysis of the electromyographic signals from these muscles has proven to be an
effective method of measuring muscle fatigue. Specifically,
increased muscle fatigue is associated with a decreased
median frequency of the power spectrum [20, 21]. Given
Surg Endosc
the short time to completion in both tasks as well as the
small number of repetitions, it was expected that EMGfm
would not yield significant differences. However, based on
the reduced activation of the EMGr during NP on the VR
environment, more repetitions of this task will probably
elicit different results in terms of EMGfm.
On the other hand, the VR environment for the BC task
appeared to elicit similar behavior with the actual environment in both the FCR and ED muscle groups. The
moderate to high correlations for most of the parameters
and the lack of significant differences between the actual
and VR environment imply that BC was simulated
effectively.
A number of studies have investigated the feasibility and
validity of performance assessments with virtual-reality
simulators. The main drawback in this procedure is the fact
that the validity of performance assessment is limited by
the reliability of such measurements [22]. Most of these
studies did not make a direct comparison between the
actual and virtual environment, but rather based the comparison on simple parameters, such as time to completion,
to evaluate the effectiveness of such environments [22–24].
The strength of the present study is the direct comparison
between the actual and VR environment, as well as the
variety of measured parameters.
A possible limitation of the present study is the high
inter-subject variability due to the inexperience of the
participants. As mentioned above, the average of five
trials was used for data analysis. The differences in
response among trials, especially from first to last, may
distort the actual performance of the participants. However, participant selection was based on the fact that the
proposed form of training is designed for the initial steps
of a potential robotic surgical training program. The
replication of the present study with more trained participants may increase the power and validity of the
current findings.
In conclusion, our study showed that the VR environment adequately simulated the BC task. The significant
differences found for the NP task may be attributed to an
oversimplification in the VR environment. Thus, they do
point to the need for improvements in the complexity of
our VR simulation. Along with the improvement of the
NP task, by incorporating more physical interaction and
the presence of a simulated needle, we are currently
developing more complex and advanced training tasks for
robotic surgery. These tasks will be evaluated in order to
meet the specific physical and mechanical needs of the
inanimate tasks. Further research is needed to develop
effective and reliable VR trainers for robotic surgery and
identify the training procedures needed to optimize
learning and transfer these skills into the operating
room.
Acknowledgements This work was supported by NIH
(K25HD047194), NIDRR (H133G040118), and the Nebraska
Research Initiative.
References
1. Satava RM, Bowersox JC, Mack M, Krummel TM (2001)
Robotic surgery: state of the art and future trends. Contemp Surg
57:489–499
2. Robinson TN, Stiegmann GV (2004) Minimally invasive surgery.
Endoscopy 36:48–51
3. Moorthy K, Munz Y, Dosis A, Hernandez J, Martin S, Bello F,
Rockall T, Darzi A (2004) Dexterity enhancement with robotic
surgery. Surg Endosc 18:790–795
4. Chang L, Satava RM, Pellegrini CA, Sinanan MN (2003) Robotic
surgery: identifying the learning curve through objective measurement of skill. Surg Endosc 17:1744–1748
5. De Ugarte DA, Etzioni DA, Gracia C, Atkinson JB (2003)
Roboticsurgery an resident training. Surg Endosc 17:960–963
6. Hernandez JD, Bann SD, Munz Y, Moorthy K, Datta V, Martin S,
Dosis A, Bello F, Darzi A, Rockall T (2004) Qualitative and
quantitative analysis of the learning curve of a simulated surgical
task on the da Vinci system. Surg Endosc 18:372–378
7. Gutt CN, Oniu T, Mehrabi A, Kashfi A, Schemmer P, Buchler MW
(2004) Robot-assisted abdominal surgery. Br J Surg 91:1390 –
1397
8. Hanly EJ, Marohn MR, Bachman SL, Talamini MA, Hacker SO,
Howard RS, Schenkman NS (2004) Multiservice laparoscopic
surgical training using the da Vinci surgical system. Am J Surg
187:309–315
9. Donias HW, Karamanoukian RL, Glick PL, Bergsland J, Karamanoukian HL (2002) Survey of resident training in robotic
surgery. Am Surg 68:177–181
10. Guru KA, Kuvshinoff BW, Pavlov-Shapiro S, Bienko MB, Aftab
MN, Brady WE, Mohler JL (2007) Impact of robotics and laparoscopy on surgical skills: a comparative study. J Am Coll Surg
204:96–101
11. Moorthy K, Munz Y, Dosis A, Hernandez J, Martin S, Bello F,
Rockall T, Darzi A (2004) Dexterity enhancement with robotic
surgery. Surg Endosc 18:790–795
12. Sarle R, Tewari A, Shrivastava A, Peabody J, Menon M (2004)
Surgical robotics and laparoscopic training drills. J Endourol
18:63–67
13. Mehrabi A, Yetimoglu CL, Nickkholgh A, Kashfi A, Kienle P,
Konstantinides L, Ahmadi MR, Fonouni H, Schemmer P, Friess
H, Gebhard MM, Buchler MW, Schmidt J, Gutt CN (2006)
Development and evaluation of a training module for the clinical
introduction of the da Vinci robotic system in visceral and vascular surgery. Surg Endosc 20:1376–1382
14. Hanly EJ, Zand J, Bachman SL, Marohn MR, Talamini MA
(2005) Value of SAGES Learning Center in introducing new
technology. Surg Endosc 19:477–483
15. Dohi T (2004) The overview of robot surgery. Nippon Rinsho
62:824–830
16. Weiss H, Ortmaier T, Maass H, Hirzinger G, Kuehnapfel U
(2003) A virtual-reality-based haptic surgical training system.
Comput Aided Surg 8:269–272
17. Kypson A, Nifong LW, Chitwood Jr WR (2004) Robot-assisted
surgery: training and re-training surgeons. Int J Med Robot 1:70–76
18. Narazaki K, Oleynikov D, Stergiou N (2006) Robotic surgery
training and performance: identifying objective variables for
quantifying the extent of proficiency. Surg Endosc 20:96–103
19. Judkins T, Oleynikov D, Narazaki K, Stergiou N (2006) Robotic
surgery and training: electromyographic correlates of robotic
laparoscopic training. Surg Endosc 20:824–829
123
Surg Endosc
20. Basmajian JV, De Luca CJ (1985) Muscles alive, their functions
revealed by electromyography. Williams & Wilkins, Baltimore,
MD
21. Bonato P, Roy SH, Knaflitz M, Luca CJD (2001) Time-frequency
parameters of the surface myoelectric signal for assessing muscle
fatigue during cyclic dynamic contractions. IEEE Trans Biomed
Eng 48:745–753
22. Rosenthal R, Gantert WA, Scheidegger D, Oertli D (2006) Can
skills assessment on a virtual reality trainer predict a surgical trainee’s talent in laparoscopic surgery? Surg Endosc 20:1286–1290
123
23. Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003)
Learning curves and impact of previous operative experience on
performance on a virtual reality simulator to test laparoscopic
surgical skills. Am J Surg 185:146–149
24. Lehmann KS, Ritz JP, Maass H, Cakmak HK, Kuehnapfel UG,
Germer CT, Bretthauer G, Buhr HJ (2005) A prospective randomized study to test the transfer of basic psychomotor skills
from virtual reality to physical reality in a comparable training
setting. Ann Surg 241:442–449