Model-Guided Therapy and the
role of DICOM in Surgery
Heinz U. Lemke, PhD
Chair of Working Group 24 “DICOM in Surgery“
Content
1. Introduction (problems and solutions)
2.
3.
4.
5.
Model guided therapy with TIMMS
Classification and model classes
Virtual human model examples
Conclusion
Computer Assisted Digital OR Suite for Endoscopic MISS
Problems: Multiple Data Sources
Video Endoscopy
Monitor
Image Manager Report
C-Arm Images
MD’s
Staff
RN, Tech
EEG Monitoring
MRI Image PACS
C-Arm
Fluoroscopy
Left side of OR
Laser
generator
Image view
boxes
EMG
Monitoring
Digital endoscopic OR suite facilitates MISS
Teleconferencing
- telesurgery
Courtesy of Dr. John Chiu
Model Guided Therapy and the
Patient Specific Model
• Model Guided Therapy (MGT) is a methodology
complementing Image Guided Therapy (IGT) with
additional vital patient-specific data.
• It brings patient treatment closer to achieving a
more precise diagnosis, a more accurate
assessment of prognosis, as well as a more
individualized planning, execution and validation
of a specific therapy.
• By definition, Model Guided Therapy is based on
a Patient Specific Model (PSM) and allows for a
patient specific intervention via an adapted
therapeutic workflow.
Model Guided Therapy and data structures
• Model Guided Therapy based on patient specific
modelling requires appropriate IT architectures
and data structures for its realisation.
• For PSMs, archetypes and templates allow
different levels of generalisation and
specialisation, respectively.
Model Based Patient Care
Modalities
(X-ray,CT, US,
MR,SPECT,
PET,OI)
Biosensors
(physiology,
metabolism,
serum, tissue, …)
Data bases
(Atlas,
P2P repositories,
data grids, ...)
Omics
EMR
Model Creation
and Diagnosis
(Data fusion,
CAD, …)
EBM
Workflow
IHE
Mechatronics
(Navigation,
ablation, …)
Model Maintenance
and Intervention
(Simulation,
decision support,
validation, …)
IT Communication Infrastructure
Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3.
4.
5.
6.
Classification and model classes
Virtual human model examples
PM data structures (SDTM and OpenEHR)
Conclusion
Interventional Cockpit/SAS modules
IT Model-Centric World View
Therapy Imaging and Model Management System (TIMMS)
Images
and
signals
Modelling
tools
IO Imaging
and
Biosensors
Modelling
Computing
tools
Simulation
WF and
K+D
tools
Kernel for
WF and K+D
Management
Repository
Data Exch.
Engine
Rep.
tools
Visualisation
Rep. Manager
Control
Devices/
Mechatr.
tools
Validation
tools
Intervention
Validation
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Data and
information
Models
(Simulated
Objects)
WF`s, EBM,
”cases”
Models and
intervention
records
Model Guided Therapy with TIMMS
• For a therapeutic intervention it is assumed that
human, mechatronic, radiation or pharmaceutical
agents interact with the model.
• MGT provides the scientific basis for an accurate,
transparent and reproducible intervention with the
potential for validation and other services.
• TIMMS is an IT meta architecture allowing for
interoperability of the agents to facilitate a MGT
intervention.
Model Guided Therapy
The basic TIMMS patient model must have the following features:
1.
2.
3.
4.
The TIMMS patient model must have components which
represent the patient as an n-dimensional and multiscale
(in space and time) data set.
The TIMMS patient model must facilitate interfacing to the
surgeon and other operative personnel, the TIMMS engines,
TIMMS repositories, and the IT infrastructure.
The TIMMS patient model must be capable of linking these
components, which may be static or dynamic, in a meaningful
and accurate way.
For dynamic components, the TIMMS patient model must be
able to process morphological and physiological data and
perform the necessary mathematical functions to maintain the
model in an up-to-date state.
Model Guided Therapy
5.
6.
7.
8.
9.
The TIMMS patient model must be capable of being incorporated
by the TIMMS executing workflow and responding to its changes.
The TIMMS patient model must be amenable to be developed
using readily available, standardized informatics methodology.
Tools may include UML, XML, Visio, block diagrams, workflow
diagrams, MATLAB, Simulink, DICOM (including surgical DICOM),
Physiome, CDISC SDTM, openEHR and similar products and tools.
The TIMMS patient model must comply to software engineering
criteria, for example, to open standards and service-oriented
architectures to allow for multi-disciplinary information exchange.
The TIMMS patient model must allow for further extensions to
incorporate advances in molecular medical imaging, genomics,
proteomics and epigenetics.
The TIMMS patient model must be amenable to be used for clinical
trials, predictive modeling, personal health records and in the long
term contribute to a Model Based Medical Evidence (EBME)
methodology.
Interventional Cockpit/SAS modules
IT Model-Centric World View
Therapy Imaging and Model Management System (TIMMS)
Images
and
signals
Modelling
tools
IO Imaging
and
Biosensors
Modelling
Computing
tools
Simulation
WF and
K+D
tools
Kernel for
WF and K+D
Management
Repository
Data Exch.
Engine
Rep.
tools
Visualisation
Rep. Manager
Control
Devices/
Mechatr.
tools
Validation
tools
Intervention
Validation
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Data and
information
Models
(Simulated
Objects)
WF`s, EBM,
”cases”
Models and
intervention
records
Generic and patient specific
n-D modelling tools
•
•
•
•
•
•
•
•
•
Geometric modelling
Prosthesis modelling
Properties of cells and tissue
Segmentation and reconstruction
Biomechanics and damage
Tissue growth
Tissue shift
Properties of biomaterials
...
Modelling
tools
Model Guided Therapy
• MGT in its simpliest instantiation is an intervention with
a subset, a single or a set of voxels representing
locations within the patient body. With this view, it is an
extension from Image (pixel) Guided Therapy (IGT) to
model (voxel) guided therapy. Examples of model
guided therapy are:
a) interventions within a subset of a voxel, e.g. cells,
organelles, molecules, etc.
b) interventions with a voxel, e.g. small tissue parts of
an organ or lesion, etc.
c) interventions with a set of voxels, e.g. part of
functional structures of organs, organ components,
soft tissue, lesions, etc.
Model Guided Therapy
In a simple PSM, voxels may be associated
with several dimensions of data
1.
2.
3.
4.
5.
6.
7.
8.
9.
1-D signals (e.g. EEG)
2-D projection and tomographic images
3-D reconstructions
Temporal change
Tissue/cell type
Ownership to organ, lesion, system, prothesis, chronic
condition, etc.
Spatial occupancy/extension
Permeability (blood brain barrier)
Flow (e.g. electric, heat, liquid, perfusion, diffusion, etc.)
Model Guided Therapy
In a simple PSM, voxels may be associated
with several dimensions of data
10. Level of oxygenation (e.g. level of hypoxia)
11. Pharmacokinetics (e.g. effect of tissue on
pharmaceutical agent, flow parameters, time to peak,
etc.)
12. Pharmacodynamics (effect of pharmaceutical agent on
tissue, ablation parameters)
13. Biological marker types (in vitro and/or in vivo
molecular spectrum)
14. Reference coordinate system (e.g.
Schaltenbrand/Warren, Talaraich/Tourneaux)
15. Value (life critical to life threatening)
16. Neighbourhood (e.g. 3³, 5³, 7³, etc.)
17. ...
Example: ENT model elements
Source: G. Strauss
Example: ENT model elements
Source: G. Strauss
Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3. Classification and model classes
4. Virtual human model examples
5. Conclusion
Strategies for multiscale modelling
• Modelling is essential for understanding the
knowledge of human characteristics such as, anatomy,
physiology, metabolism, genomics, proteomics,
pharmacokinetics, etc.
• Because of the complexity of integrating the
knowledge about the different characteristics the
model of a human has to be realised on different
levels (multiscale in space and time) and with different
ontologies, depending on the questions posed and
answered delivered.
• The problems associated with using reduced-form
components within large systems models stem
primarily from their limited range of validity.
Source: J. Bassingthwaighte
Patient specific and associated
modelling functions
In the Model-Centric World View a wide variety of
information, relating to the patient, can be integrated
with the images and their derivatives, providing a more
comprehensive and robust view of the patient.
By default, the broader the spectrum of different types of
interventional/surgical workflows which have to be
considered, the more effort has to be given for designing
appropriate multiscale PSM’s and associated services.
Patient specific and associated
modelling functions
Management of n-D and multi resolutional
knowledge (model of the biologic continuum in
space and time) is still a research and
development challenge.
If solved successfully, it will transform surgery
into a more scientifically based activity.
Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3. Classification and model classes
4. Virtual human model examples
5. Conclusion
Patient Specific CMB
Human Laser
Scan (CAESAR DB)
Multimodal Imaging
(MRI, CT, Angio,..DT-MRI)
Visible Human
Anatomical Template
organ surface meshes
Roberts JHU
Spitzer 2006 Virtual Anatomy
PKPD
FEM Mesh (Roberts JHU)
Content
1.
2.
3.
4.
Introduction (problems and solutions)
Model guided therapy with TIMMS
Classification and model classes
Virtual human model examples
5. Conclusion
Solutions and Research Focus
(medical)
• Transition from image guided to model guided
therapy (e.g. through workflow and use case
selection/creation/repositories)
• Concepts and specification of patient specific
models in a multiscale domain of discourse
• Concepts and design of a canonical set of low
level surgical functions
• Prototyping
Interventional Cockpit/SAS modules
IT Model-Centric World View
Therapy Imaging and Model Management System (TIMMS)
Repository
Data Exch.
Engine
Control
Prototyping
Images
and
signals
Modelling
tools
IO Imaging
and
Biosensors
Modelling
Computing
tools
Simulation
WF and
K+D
tools
Kernel for
WF and K+D
Management
Rep.
tools
Visualisation
Rep. Manager
Devices/
Mechatr.
tools
Validation
tools
Intervention
Validation
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Data and
information
Models
(Simulated
Objects)
WF`s, EBM,
”cases”
Models and
intervention
records
Solutions and Research Focus
(technical)
• Concepts and data structure design of patient specific
models (e.g. with archetypes and templates)
• Model management with open architectures (e.g. SOA)
• SOA modulariation with repositories, engines, LLM´s and
HLM´s
• LLM´s as adaptive (cognitive/intelligent) agents
• HLM´s as application modules (competitive differentiation)
• LLM´s possibly as open source
• Kernel (engine and repository) for adaptive workflow and
K+D management
• Cooperative and competitive R+D framework for engine
and repository building
• Therapy based open standard ( e.g. S-DICOM)
• Transition from CAD to CAT modelling
Interventional Cockpit/SAS modules
IT Model-Centric World View
Repository
Therapy Imaging and Model Management System (TIMMS)
Data Exch.
Engine
Control
Archetypes and Templates
Images
and
signals
Modelling
tools
IO Imaging
and
Biosensors
Modelling
Computing
tools
Simulation
WF and
K+D
tools
Kernel for
WF and K+D
Management
Rep.
tools
Visualisation
Rep. Manager
Devices/
Mechatr.
tools
Validation
tools
Intervention
Validation
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Data and
information
Models
(Simulated
Objects)
WF`s, EBM,
”cases”
Models and
intervention
records
Solutions and Research Focus
(medical and technical)
• Transition from image guided to model guided therapy (e.g.
through workflow and use case
selection/creation/repositories)
• Use cases for adaptive workflow, exception handling and
K+D management for selected interventions
• Cooperative and competitive R+D framework for low
(open source) and high level (competitive differentiation)
surgical function computerisation
• Information/model flow from diagnosis (e.g. CAD) to CAT
(i.e. interdisciplinary cooperation)
• Development of standards for patient modelling in
WG24 “DICOM in Surgery”
Interventional Cockpit/SAS modules
IT Model-Centric World View
Repository
Data Exch.
Engine
Control
Candidate components for open source
Open Source
Images
and
signals
Modelling
tools
IO Imaging
and
Biosensors
Modelling
Computing
tools
Simulation
WF and
K+D
tools
Kernel for
WF and K+D
Management
Rep.
tools
Visualisation
Rep. Manager
Devices/
Mechatr.
tools
Validation
tools
Intervention
Validation
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Data and
information
Models
(Simulated
Objects)
WF`s, EBM,
”cases”
Models and
intervention
records
WG 24 “DICOM in Surgery“
Project Groups
•
•
•
•
•
•
•
•
•
•
•
PG1
PG2
PG3
PG4
PG5
PG6
PG7
PG8
PG9
PG10
PG11
WF/MI Neurosurgery
WF/MI ENT and CMF Surgery
WF/MI Orthopaedic Surgery
WF/MI Cardiovascular Surgery
WF/MI Thoraco-abdominal Surgery
WF/MI Interventional Radiology
WF/MI Anaesthesia
S-PACS Functions
WFMS Tools
Image Processing and Display
Ultrasound in Surgery
Definition of Surgical Workflows (S-WFs)
• Micro Laryngeal Surgery (MLS)
(PG2
ENT/CMF)
• Foreign Body Excision
(PG2 ENT/CMF)
• Total Hip Replacement Surgery
(PG3
Orthopaedic)
• Total Endoscopic Coronary Artery Bypass (TECAB) (PG4
Cardiovascular)
• Mitral Valve Reconstruction (MVR)
(PG4
Cardiovascular)
• Laparoscopic Splenectomy
(PG5
Thoraco-abdominal)
• Laparoscopic Cholecystectomy
(PG5
Thoraco-abdominal)
• Laparoscopic Nephrectomy left
(PG5
Thoraco-abdominal)
• Angiography with PTA and Stent
(PG6
Interventional Radiology)
• Hepatic Tumor Radio Frequency Ablation
(PG6
Interventional Radiology)
• Trajugular Intrahepatic Portosystemic Shunt
(PG6
Interventional Radiology)
CARS 2008 Computer Assisted Radiology and Surgery
CARS / SPIE / EuroPACS
9th Joint Workshop on
Surgical PACS and the Digital Operating Room
Barcelona, 28 June, 2008
12th Meeting of the
DICOM Working Group WG 24 “DICOM in Surgery“
Barcelona, 28 June 2008
http://www.cars-int.org
WG24 “DICOM in Surgery”
Secretariat:
Secretary:
Howard Clark, NEMA
Franziska Schweikert, CARS/CURAC Office
fschweikert@cars-int.org
General Chair: Heinz U. Lemke, ISCAS/CURAC, Germany
Co-Chair:
Ferenc Jolesz, Harvard Medical School, Boston
(Surgery/Radiology)
Co-Chair:
(Industry)
tbd