C
I
U
SMART Lab
Virtual Reality Simulators for
Minimal Invasive Surgery
Training
Shamyl Bin Mansoor,
SMART Lab, SEECS, NUST, Islamabad
School of Electrical Engineering & Computer Science,
1
National University of Sciences & Technology, Pakistan
About the Team
• School of Electrical Engineering & Computer Science
–
–
–
–
–
–
–
–
Dr Osman Hasan, Project Director
Dr Muddassir Malik Co-PI,
Dr Arshad Ali, DG, SEECS, NUST
Shamyl Bin Mansoor, Co-PI
Researchers
Zaheer Mukhtar (Software Engineer)
Taimur Hassan (Mechanical Engineer)
Farrukh Hijaz (Electrical Engineer)
• Holy Family Hospital, Rawalpindi
– Dr Asif Zafar, Professor of Surgery, Head Surgical Unit -2,
– Dr Qasim Ali, Senior Registrar (Surgeon)
– Dr Faisal Murad, Senior Registrar (Surgeon)
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
2
Outline
•
•
•
•
•
•
I
Motivation
Our Project
Challenges
Developing the System
Our Approach
About the Lab
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
3
Motivation
• Minimal Invasive Surgery is currently in vogue
• Operations once performed “open” are now done almost
exclusively using laparoscopes
• The latest trend is to perform robots assisted surgery
• All these new skills require special training
– Hand Eye Coordination
– Navigation of Camera and Laparoscopic instruments in a confined
space
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
4
Background
• Conventional Open Surgery Vs Minimal
Invasive Surgery
Conventional Surgery
I
C
U
SMART Lab
* Images courtesy of Dr Asif Zafar (Holy Family Hospital, Rawalpindi)
MIS
Background
• Robots Assisted Minimal Invasive Surgery
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
6
State of the Art
ProMIS by Haptica, Ireland
LapSIM By Surgical Science, Sweden
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
7
Our Project
• Objectives
– Developing a cost effective training system for Minimal Invasive
Surgery
• Major Components
– Virtual Reality Simulator for Minimal Invasive Surgery
– Training Robot for Robot assisted Minimal Invasive Surgery
• Value Addition
–
–
–
–
I
C
U
SMART Lab
Simulating Real Surgery Scenes recorded at Holy Family Hospital
Using open source tools making it cost effective
Haptic Intercface with the Simulator
Developing basic and advanced exercises (Fundamentals of
Laparoscopic Surgery (FLS) curriculum)
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
8
The Concept
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
9
Challenges
• The first challenge is to model the human anatomy
– Making 3D models that exactly look like the original ones
• The second challenge is to simulate this 3D model to act like the
real thing
– For example a 3D model of heart can only be called a heart model if it
acts like a human heart too, beating and pumping blood
– Simulating Tissue behavior
• The third challenge is to simulate the environment within the
human body
– What happens if an external object pokes a tissue?
– In real-life the tissue or that part of the body would start to bleed
internally
– How can that be simulated
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
10
System Components
•
•
•
•
•
•
•
•
I
Dataset
Mechanical Object
Tissue Modeling and Characterization
Illumination Mechanism
Collision Detection
Teaching Resources
Assessment through metrics and feedback
Performance Monitoring
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
11
Teaching Resources and
Assessment
• Implementing Laparoscopic Training Manual and
specific scenarios
• Cues, Feedback and comparisons
• Assessment through metrics and qualitative
feedback
– Errors
– Movement
– Time
– Forces
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
12
Four Modules
• Simulation Engine
– Based on Simulation Open Framework Architecture
– Modeling and Simulation
• Basic Training Module
– Fundamentals of Laparoscopic Surgery Curriculum for Surgeon
Training
• Advanced Training Module
– Simulation of complex laparoscopic procedures
– Videos provided by Holy Family Hospital of different scenarios of
laparoscopic surgery
• Evaluation Module
– Evaluation of surgeons on the basis of simulator training
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
13
Our Approach
• We are utilizing open source framework for
developing our system
• Using SOFA (Simulation Open Framework
Architecture) as the core of our system
• Creating simulations of real scenarios from Holy
Family’s database of over 100 videos
• Formal Verification of the system
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
14
Formal Verification
• A Safety Critical Application
– A minor glitch in the analysis phase could result in disastrous
consequences
• Potential Analysis Aspects
– Evaluating Behavioral Correctness of Telesurgical Robotic
Systems
– Formal Reliability Analysis of Telesurgical Robotic Systems
– Formal Verification and Performance Analysis of Telesurgical
Network Protocols
I
C
U
SMART Lab
Current Progress
• Developed basic exercises of FLS Curriculum
• Integrated a conventional joystick for camera
navigation
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
16
Demo Video
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
17
Demo Video 2
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
18
Research Areas at SMART lab
• We are also interested in a number of other areas
which include
– Android based Disaster Management Systems
– Telemedicine Systems using mobile telephony
– Intelligent Robotics
– Brain Machine Interface
– Visualization Techniques
More Information Available at
http://smart.seecs.nust.edu.pk
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
19
Thank You
Contact Us!
smartlab@seecs.edu.pk
shamyl.mansoor@seecs.edu.pk
I
C
U
SMART Lab
Smart Machines And Robotics Technology
Lab, School of Electrical Engineering &
Computer Science, NUST
20