Content-based Retrieval of 3D Medical Images
Y. Qian, X. Gao , M. Loomes, R. Comley, B. Barn
School of Engineering and Information Sciences
Middlesex University, UK
R. Hui, Z.Tian
Department of Neurosurgery, General Navy Hospital, P.R.China
Contents
1. Background
2. Methodology
3. Experiment Results
4. Conclusion and Future Work
1. Background
MIRAGE
(Middlesex medical Image Repository with a CBIR ArchivinG Environment)

Aim: To develop a repository of medical images benefiting MSc and research
students in the immediate term and serve a wider community in the long term in
providing a rich supply of medical images for data mining, to complement MU
current online e-learning system.

So far 100,000 2D images and 100 images in 3D form.

http://image.mdx.ac.uk/
JSIC


Innovation in the use of ICT for education and research.
http://www.jisc.ac.uk/
Content-Based Image Retrieval (CBIR)
CBIR can index an image using visual contents that an
image is carrying, such as colour, texture, shape and
location.
e.g. Query by Example Image(QBE)
Proposed Framework for MRIAGE
GIFT Framework
GIFT(GNU Image Finding Tool) is open framework for
content-based image retrieval and is developed by
University of Geneva.
 Query by example and multiple query
 Relevance Feedback
 Distributed architecture (Client - Server)
Demo:
Content-Based 3D Brain Image Retrieval
2D brain images ----- 3D Brain
 Shape-based
Surface of a 3D object(e.g. tumor)
 Texture-based
Inside of a 3D object( e.g.textures representing tissue structure properties
Aim: To develop a fast texture-based 3D brain retrieval method
2. Methodology
Proposed framework for 3D image retrieval
Pre-processing
1) Spatial Normalization---Statistical Parametric Mapping
(SPM5)
Transform each individual brain into a standard brain template
2) Divide 3D brain into 64 non-overlapping equally sized
blocks
Extraction of Volumetric Textures
1) 3D Grey Level Co-occurrence Matrices (3D GLCM)
2) 3D Wavelet Transform (3D WT)
3) 3D Gabor Transform (3D GT)
4) 3D Local Binary Pattern (3D LBP)
1) 3D Grey Level Co-occurrence Matrices (3D
GLCM)
3D GLCM is two dimensional matrices of the joint
probability of occurrence of a pair of gray values
separated by a displacement d = (dx,dy,dz).

52 Displacement vectors:
4 distance * 13 direction = 52

4 Haralick texture features:
energy, entropy, contrast and homogeneity

Feature vector:
208 components (=4 (features) * 52 (matrices)).
2) 3D Wavelet Transform (3D WT)
3D WT provides a spatial and frequency representation
of a volumetric image.

2 scales of 3D WT

Mean and Standard deviation

Feature vector:
30 components (2 (features) +15 (sub-bands))
3) 3D Gabor Transform (3D GT)
3D GT generates a set of 3D Gabor filters
Gabor filters
g  x, y, z , F ,  ,    g  x, y, z  exp  j 2 F sin  cos x  F sin  sin y  F cos z 
^
Gabor Transform:
GTi  f x, y, z  * g x, y, z, Fi ,i , i  i  1,2,3...144

144 Gabor filters
4 (F) *6(θ)*6(Φ) =144

Mean and Standard deviation

Feature vector:
288 components (2 (features) +144(filters))
4) 3D Local Binary Pattern (3D LBP)
Local binary pattern(LBP) is a set of binary code to
define texture in a local neighbourhood. A histogram is
then generated to calculate the occurrences of different
binary patterns.

59 binary patterns

Feature vector:
177 components (=59(patterns)*3(planes)
Similarity Measurement
Histogram Intersection(3D LBP)
DQ, I    minQi , I i 
i
 Normalized Euclidean distance (3D GLCM,3D WT,3D GT)
DQ, I  
 Qi  I i 





i
i


2
Lesion Detection
Assume bilateral symmetry of a normal brain along its mid-plane
3. Experimental Results
Test Dataset
 100 MR brain images
 Size: 256  256  44
 DICOM (Digital Imaging and Communications in Medicine) format
 Collected from Neuro-imaging Centre at Beijing General Navy Hospital,
China
Experimental Results ------ Lesion Detection
Experimental Results -------Retrieval
Experimental Results -------Query time
4. Conclusion and Future work
1) Conclusion:
 Comparative results demonstrate that LBP outperforms four 3D texture
methods in terms of retrieval precision and processing speed.
 The query time with VOI selection offers 4 times faster operation than
that without.
2) Future work:
 Test on the larger dataset
 Plug 3D image retrieval into GIFT framework (MIRAGE 2011)
Thank You.