Neurobase Project
27/11/2006
« Re-use or Re-invention - a Roadmap for Data Integration »
27 - 28 November, 2006, e-Science Institute, Edinburgh
Integration of Distributed and
Heterogeneous resources in
Neuroimaging:
The NeuroBase Project
Primary Investigator:
Investigator: C. Barillot
VisAGeS Unit/Project - U746
INSERM/INRIA
IRISA, CNRS 6074 , Univ. of Rennes I
Campus de Beaulieu, 35042 Rennes, France
http:
http://www.irisa.fr/visages/Neurobase
//www.irisa.fr/visages/Neurobase
Context
Framework:
Framework: Concerted Action of the French ministry of
research « Technologies for health »
Period: 2002-2005
Principal Investigator : Christian BARILLOT
Major Contributors :
VISAGES, IRISA/Inria/INSERM/Cnrs, Rennes (Image processing
and Ontologies) : B. Gibaud, A. Gaignard, L. Temal
IFR 49 " Functional Neuroimaging » Paris & Orsay (CEA,
INSERM U678, CHR Pitié Salpétrière) (clinical and cognitive
neuroimaging): H. Benali, M. Pélégrini-Issac, S. Kinkingnéhun
CARAVEL, INRIA Project-team
Medience Inc.
Business
Object Inc. (mediation, data bases) : E. Simon, J-P. Matsumoto
INSERM U594, Grenoble (processing in cognitive neuroimaging
and Ontologies): M. Dojat
Grant Funding:
Funding: 100k€
100k€
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
1
Neurobase Project
27/11/2006
Federation of information
resources in clinical neuroimaging
General Objectives
Follow the growth of the communication and exchange
infrastructures (e.g. Internet)
Follow the emergence of "virtual" organizations of users (e.g.
clinical groups of research)
Applications of information and grids technologies in health:
Creation of "virtual" cohorts
Research on the singular diseases (search for «unlikely facts»)
through data mining and knowledge discovery from image
descriptors
Validation / certification of new drugs
Research Issues
Combine Grid Computing and Data/Semantics Grid
technologies in the field of medical imaging
Evolutive and adaptive workflows in Medical Imaging (user
interactions, heterogeneity, …)
Integrate the semantic web technologies into clinical research
Initial Objectives of Neurobase (1)
Specify how to integrate heterogeneous and
distributed information resources in neuroimaging
Applications in neurology and clinical neurosciences
Definition of a datadata-processing architecture allowing :
Access and sharing of experimental neuroimaging
protocols and results
Access and sharing of image processing procedures (on
anatomical and functional data)
in order to:
carry out large scale experiments
re-use existing image processing tools
validate new image processing tools
Access to validation data sets
Comparison to existing processing tools
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
2
Neurobase Project
27/11/2006
Initial Objectives of Neurobase (2)
At the end,
end, such a system must allow to index
information accessible from different heterogeneous
and distributed data bases for :
The search of experiments according to a specific protocol
(e.g. allow the retrieval of specific descriptions of experiments, allow the
examination of experimental results, and retrieving eventually the related
images)
The search of similar results (e.g. for the study of anatomofunctional networks)
The search of images containing singularities (spatio-temporal
particularities for instance)
Transverse searches to highlight possible regularities
(similarly to a "data mining" type approach) (e.g. possible
similarities of the protocol corpus, of the experimental results and the
related images, or encore spatio-temporal invariants).
« Neurobase »
Phase I
Specifications
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
3
Neurobase Project
27/11/2006
“Neurobase” system architecture for wrapping
neuroimaging information resources
Users
applications
Internet Access
Uniform View
Mediation
services
Common access service (query/retrieve)
(query/retrieve) using the
« Neurobase » semantic model
Wrapper 1
Heterogeneous &
Distributed
Information
Data Base
Information
data base 1
•C++
•Java
•Php
•.dim
Wrapper i
Wrapper n
Information
data base i
Information
data base n
•Delphi
•Perl
•Matlab
•.hdr
•C
•Perl
•Vtk
•.dcm
Action:
Elaboration of a reference model for
the sharing of neuroimaging data
This covers the modeling of the :
☺ Source images
☺ Symbolic data describing acquisition conditions
ROI’s
Indexation processes
Interpretations performed on experimental data
(labeling of anatomical region or functional responses)
Matching procedures between images and patients
Image processing procedures for segmentation and
quantification
Stimulation procedures (activation paradigms)
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
4
Neurobase Project
27/11/2006
Initial work plan
Elaboration of a demonstrator based on some
existing modules:
Le Select (http://www.le-select.com/)
BrainVISA/Anatomist (http://brainvisa.info/)
VIsTAL
(http://www.irisa.fr/visages/software/VIsTAL/VIsTAL.html)
BALC (U594, Grenoble)
FSL (http://www.fmrib.ox.ac.uk/fsl)
Implementation of functionalities such as data
wrappers & mediators, medical image processing
methods (data access, registration, segmentation,
visualization, …) and data flow approaches.
This demonstrator ambitioned to be developed and
Example of indexation from a
probabilistic atlas of visual areas*
A 2D unfolded map showing
VFR (color) and low level
visual areas borders
Descriptor =
Activation V3v
V3A
probability
V2d
V1
Delineation of low level visual
areas borders from MRI and fMRI
V2v
V3v
V4
unfolded retinotopic
map of a subject
Probability similarity index
of visual areas borders
Matching to a training set
*I. Corouge, M. Dojat, and C. Barillot, Medical Image Analysis, 8(3), 2004.
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
5
Neurobase Project
27/11/2006
Exemple: Search for SPECT perfusion
abnormalities in Epilepsy
Query : Seek the perfusion data showing a " hyperhyperperfusion in the frontal lobe region " in SPECT
(e.g. " Frontal lobe epilepsy »)
Search result: retrieved to the client application:
SPECT volumes
descriptions of the clinical cases
If needed, MRI volumes and the MRI-SPECT registered data
Examples of search of SPECT data by using a digital approach :
search in images from a model (e.g. an atlas)
Example « frontal lobe region»
Use of different methods of inter-individual fusion :
SPECT Template (SPM)
Registration Atlas-MRI + Registration MRI-SPECT
Quantification operators (hypo-perfusion, hyper-perfusion)
Query guided by a spatial discrete model
2. Registration
to the SPECT
SPM template
1. Definition of a 3D ROI in a digital
model (Example «frontal lobe region»)
3. Transfer of the
region of interest
4. <Descriptor> = quantitative parameter
to the 3D ROI
(e.g. comparison with the reference region )
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
6
Neurobase Project
27/11/2006
Exemple: Search for transversal
search of MS lesion classification
Query : find all T2 hyper intense MS lesions on
multisequences longitudinal MRI with lesion
load < 10mm3 on the right hemisphere
Preprocessing workflow
Retrieve MR-T1 head volume
Correct for image artifacts (bias, noise, …)
Segment brain structures into hemispheres and cerebelum
MS lesion workflow :
Classify the multisequence longitudinal MRI for all retrieve
subjects
Detect lesion
Classify lesions
Compute lesion load for Hyper intense lesions (on ↑MR-T2 and
↑T1-Gd)
Spatio-temporal analysis of
imaging data
Image registration (rigid / nonnonrigid) [HellierHellier-TMI03]
TMI03]
Image preprocessing
noise reduction [Ogier04, 06]
bias field correction [Prima-MedIA05]
Segmentation of brain structures
[Ciofolo04,05,06]
Ciofolo04,05,06]
Definition of ROIs
Computation of global
quantitative indexes (volume,
length, thickness, …)
Active shape (e.g. level sets) for
refining the atlas-based
segmentation
11/27/2006
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
14
7
Neurobase Project
27/11/2006
Automatic spatio-temporal
segmentation of MS lesions
t1
t1
Parametric
estimation of
“normal” tissues
t2
t2
Identification of
“irregular” data
Introduction a
priori knowledge
on the pathology
tn …
… tn
11/27/2006
[AitAit-Ali 05,06]
05,06]
15
Automatic spatio-temporal
segmentation of MS lesions
Time 3
Time 3
Time 1
3 acquisitions, 4 modalities : T1, T1 Gd, T2 and PD (46 × 256 × 256 - 3mm slices)
T1w
T1-Gd
T2w
PD
Results
Segmentation of new lesions evolving to lesion edema
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
8
Neurobase Project
27/11/2006
Neurobase Ontology
Scope
Studies (subjects, experimental context, clinical
aspects, etc.)
Datasets and description of their content (images, ROI,
registration data, etc.)
Image processing (processing tools, processing, etc.)
Method
Integration of multiple sources (fMRIDC, DICOM,
Neurobase partners’ experience)
Representation : UML, then Protégé
B. Gibaud’s Talk
« Neurobase »
Phase II
Test Bed
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
9
Neurobase Project
27/11/2006
Test Bed
Web Application
Dataset selection based on user-defined criteria (subjects,
studies, datasets)
Execution of « dataflows »
Display of results
Integration of a Java-Based 3D Imaging Viewer (JIV http://www.bic.mni.mcgill.ca/~crisco/jiv/)
→ Software environment: Servlet container Tomcat
Mediation system
Publish and access the data
Invoke the programs
→
based on « Le Select » (Medience/Business Object Inc.)
Test Bed : Ressources distribution
Local data repositories based on heterogeneous data
organizations:
Paris: BrainVISA/Anatomist (http://brainvisa.info/)
Rennes: PostGres SQL / Dicom
Grenoble: BALC
Image processing workflows based on heterogeneous
(proprietary and public domain) modules:
Rennes: VIsTAL (http://www.irisa.fr/visages/software/VIsTAL/VIsTAL.html)
Grenoble: BALC
Paris: FSL (http://www.fmrib.ox.ac.uk/fsl)
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
10
Neurobase Project
27/11/2006
Le_Select
(Medience SA
Business Objects Inc.)
Initially developed at Inria (CARAVEL projectprojectTeam, Rocquencourt)
Rocquencourt)
Major features
Uniform access to distributed heterogeneous data
Application of transformations to data
Data « published » according to a relational model, and
accessed in SQL (e.g. using JDBC)
Invocation of data processing programs on arbitrary
datasets
Fully distributed
“Neurobase” Test Bed Architecture:
Principal
•C++
Image
data
base
Server
S1
PostGres
•Perl
•Java
WD
WD
Client 1
WP
WP
2D/3D
Viewer
WD
WD
LeSelect™
™
LeSelect
LeSelect™
Web
Browser
jdb
c
Internet Access
Data
Data
Tomcat
Tomcat Flow
Flow
LeSelect™
™
LeSelect
LeSelect™
WD
WD
Image
data
base
WP
WP
htt
p
jdbc
WD
WD
•C++
•Perl
PostGres
•Java
Server S2
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
11
Neurobase Project
27/11/2006
“Neurobase” Test Bed Architecture:
Exploitation
INTERNET
Firewall
Firewall
Firewall
Firewall
5517; 3060
5517; 3060
5517; 3060
Le Select
Le Select
Le Select
Le Select
IRISA (putamen)
Grenoble
Jussieu
U. Rennes I
boot server
boot server
boot server
IRISA_NET
boot server
5517; 3060
5517; 3060
Client Demo
WebApp
IRISA (w3ext)
TomCat
Apache
8080
8080
Connect thru https
and passwd
INTERNET
“Neurobase” Test Bed : Execution of
distributed processing on shared data
g2a
IRM 3T
(8 bits, Analyze)
Head MRI (8 bits, Analyze)
Brain Mask (8 bits, Analyze)
a2g a2g
Classification
GM/WM
VISTAL
2D/3D Display
(client BrainVisa/
Anatomist)
Classification
GM/WM
BALC
a2g
Classified Volume
(8 bits, Gis)
Gis)
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
Brain Mask
BET/FSL
Brain MRI (8 bits,
Analyze)
g2a
Head MRI (8 bits, GIS)
Grenoble
(8 bits, GIS)
a2g
Data Flow A
Rennes
IRM 1.5 T
Classified Volume
(8 bits, Analyze)
2D/3D Display
(Client MRIcro/FSL)
12
Neurobase Project
27/11/2006
“Neurobase” Test Bed : Execution of
distributed processing on shared data
Grenoble
IRM 3T
(8 bits, GIS)
(8 bits, Analyze)
a2g
Head MRI (8 bits, Analyze)
Restored Head MRI (8 bits, Analyze)
Restoration
VISTAL
Brain Mask
BET/FSL
Brain MRI (8 bits,
Analyze)
g2a
Head MRI (8 bits, GIS)
Brain Mask (8 bits, Analyze)
a2g a2g
Classification
GM/WM
VISTAL
2D/3D Display
(Client BrainVisa/Anatomist)
a2g
Classified Volume
(8 bits, Gis)
Gis)
Classification
GM/WM
BALC
g2a
Data Flow B
Rennes
IRM 1.5 T
Classified Volume
(8 bits, Analyze)
2D/3D Display
(Client MRIcro/FSL)
NeuroBase Web Application: Query
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
13
Neurobase Project
27/11/2006
NeuroBase Web Application:
Retrieve
NeuroBase
WebApp:
Data Flow
Results
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
14
Neurobase Project
27/11/2006
Neurobase project: the good ☺
☺ Generic architecture to share distributed and
☺
☺
☺
heterogeneous resources (data, processing tools)
Shared semantic model on data resources
Build around a mediation middleware (Le Select /
Medience Server™
Server™)
OnOn-site deployment of a demonstrator:
☺ Generic hierarchical data wrapper
☺ Interfacing heterogeneous data (BALC, BrainVisa,
DICOM, PostGres SQL)
☺ Interfacing heterogeneous image processing tools
(Vistal, FSL, BrainVisa, BALC, ?Matlab)
☺ Implementation and exploitation of dataflows working
on distributed and heterogeneous resources
Neurobase project: the bad
No ontology on processing tools and activation paradigms yet
No generic definition for designing distributed dataflow/workflow
dataflow/workflow
Test Bed exploitation still limited
External management of processing capabilities (processing
middleware vs semantic middleware)
Security issues
Robustness w.r.t. network events (toward P2P)
Integration of processed data
Dependency to network and processing resources capabilities
Exploitation with “real”
real” applications
Deployment around Virtual Organization of users
Deployment around relevant applications (MS, Strokes, Epilepsy,
Dementia, Parkinson, Tumors…)
Need specialized ontology
? Extension to new application areas and new scales (e.g. biological
imaging, animal, genetics, …)
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
15
Neurobase Project
27/11/2006
Summary
We proposed the “Neurobase”
Neurobase” architecture based
on mediation services to cope with the distributed
nature of neuromaging resources
Worked have been done to define a “Neurobase”
Neurobase”
common ontology to cope with the
heterogeneous aspects of the information
resources (data, processing methods)
Proof of concepts has been shown with a
demonstrator (demo next)
Dedicated neuroimaging applications need to be
implemented in the system with dedicated
ontologies
Perspectives
Works continue in a new French Research National
Agency on « Software Technologies » (2007(20072009):
Extend to address workflow issues and Grid computing
(experiments on EGEE and G5K)
Extend to address ontology aspects on processing tools
Extend to address security issues on data
Extend to test 3 test-bed applications (MS lesions,
Stroke, Brain tumors)
Good fit with FP7 3.5.2.1 Objectives on “Virtual
Physiological Human”
Human” (call #2)
#2)
Looking at extending to EU partnerships
Already advanced on MS pathology
Christian Barillot, PI, Visages U746
(INRIA/INSERM)
16