Workshop i Visualisering Presentation
Haptic and Visual Simulation
of a Material Cutting Process
Using Patient Specific High Resolution CT-data
for Haptic- and Graphic rendering
Magnus G. Eriksson
Supervisor: Professor Jan Wikander
Co-supervisor: Professor Hans von Holst
CTV (Center for Technology and Health Care)
The Mechatronics Lab/Machine Design, KTH, Stockholm
Department of Neuronic Engineering KTH-STH, Huddinge
Mechatronics Lab
Background
Since 1980s
Since 1990
Since 1990s
Mechatronics Lab
Temporal Bone Surgery Simulator
Mechatronics Lab
Education of Surgeons
•
“See one, do one, teach one”
•
Patients risky situation
•
Ethically and economically unacceptable
•
Cadavers, plastic models or animals (high cost, ethical problems,
difficulties of training results)
Exampel
Mechatronics Lab
Training in VR Simulators
•
Avoid patients and cadavers
•
Performance feedback
•
Not time related
•
Pre-operation planning
•
Older, experienced surgeons
•
Rare pathologies
•
Reduce expensive costs
•
First 60 patients <-> Simulator training (Ahlberg et al.)
Mechatronics Lab
VR and Haptic Simulators Used Today
Film example from Simulatorcentrum
•
AccuTouch® Endoscopy Simulator
•
LapSim® Laparoscopy Simulator
Metrics and Certified by US Food and Drug Administration (FDA)
Mechatronics Lab
The Temporal Bone Surgery System
3D graphic
patient
Anatomical Model
Visual
Feedback
Visual
Feedback
”Calc.
Force”
Drilling
Operation
Pos.
Force
Feedback.
surgeon
”Real Force”
Pos.
slave
Mechatronics Lab
master
Pos.
Research Goal and Focus
Goal:
•
To develop a haptic and VR system for training and educating surgeons
who practice bone milling
Focus:
1. To develop a VR system for realistic 3D representation of the human
skull, including the changes resulting from the milling process
2. To develop an efficient algorithm for realistic haptic feedback to mimic the
milling procedure using CT-data of the skull
Real time demands, without artifacts or delays when removal of material
Mechatronics Lab
System Design
SenseGraphics H3DAPI
Initialization
Graphic Thread 30 Hz
Haptic Thread 1000 Hz
•3D Visualization
•Material Removal
•Collision Detection
•Calculate Force
Mechatronics Lab
Possible VR Haptic and Milling Applications
•
Temporal bone surgery
•
Craniofacial surgery
•
Dental tooth milling
•
Vertebral operating procedures
•
Freeform design
Research Goal: To develop a haptic and VR system for training and
educating surgeons who practice bone milling
Mechatronics Lab
Conclusion
•
A haptic and VR surgical milling simulator prototype
•
Patient specific DICOM data
•
Efficient graphical rendering
•
Haptic rendering to avoid fall-through problems
•
Real time requirements when removing material
Film, tooth milling
Mechatronics Lab
Future Work
•
Investigate various force models (3-DOF vs 6-DOF, mill is turned onoff, material removal rate) and benchmark
•
Other applications, bone fractures, sculpting etc…
•
User interface and virtual environments / visualisation
•
Validate the simulator together with Simulatorcentrum
•
Re-Design the haptik device and API (Matlab/Simulink)
•
Investigate the economical and ethical benefits of using simulators
Film, skull bone milling
Mechatronics Lab
Initialization
Initialization
• Volumetric data from a DICOM-file
• Density and gradient 3D matrices
• Octree node structure containing the
voxel data
Mechatronics Lab
 Range of x, y, and
z coordinates.
 Max/min density
values.
Graphic Rendering
•
Update rate > 30Hz and a latency of less than 300ms
Graphic thread, 30 Hz
Y_max
r
Check for milling
Update density and
gradient values
d
Y_min
X_min
Marching cubes
algorithm on updated
tree nodes
OpenGL to create the
shape of the object
Film, skull bone milling
Mechatronics Lab
X_max
Haptic Rendering
Haptic thread, 1000 Hz
n̂1
Collision detection
A force based on a
proxy-probe method
p proxy_ tng
p proxy n̂2
a2
a
a1
p probe
If milling, add
vibration force
Send force to the
haptic device
Mechatronics Lab
n2
F  k  ( p proxy_ surface  p probe)
Verification of the Haptic Algorithm
A Stiff Surface
Mechatronics Lab
A Soft Surface