From: AAAI Technical Report SS-94-05. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved.
Representing
Institute
spatial
knowledge using the "Intelligent
Volumes" concept
K.H. H~hne, A. Pommert, M. Riemer,
Th. Schiemann, R. Schubert, U. Tiede
of Mathematicsand ComputerScience in Medicine (IMDM)
University Hospital Eppendorf
Martinistrafle 52
20246 Hamburg, Germany
e-marl: hoehne@uke.uni-hamburg.de
Introduction
Method
[’erent fields of computer science contribute to medL imaging. Image processing and computer graphdeal with the creation and enhancement of visual
sentntion of medical information. Its interpretation
,erally remains in the hands of the viewer. Thus the
resentation of semantics does not play a role there.
:ontrast to this, computervision tries to extract sentical information from images. Despite numerous
Jroaches there are no general operational methods
[lling the requirements of clinical application yet.
.~refore there was no need to develop representation
emes for the extracted information. Artificial intelnee, however, is just dealing with such knowledge
~esentation schemesin great detail, but the link to
pictorial world has mostly been ignored.
Vhen going into the field of making volume based
tondcal atlases,
where not much work has been
e so far [Memoet al., 1990], we encountered the
~I of s spatial knowledge representation,
which
s image volume data with symbolic knowledge.
s ]ink has not much been considered by typiapplications for representation of human anatomy
az [Robinson et al., 1993, Wahler-L~cket el., 1991,
loine et al., 1991, Greits e~ aL, 1991]. Thus we
e developed a new scheme for the combined repntation of the pictorial and symbolic knowledge of
ge volumes, that we preliminarily call "intelligent
,me". This paper describes its structure, the tools
its generation and exploration, and (despite of rening problems) the advantages for describing and
[oring spatial structures such as they occur in cliniimaging or anatomy education.
According
to the factthatany spatialobjectpossesses
a spatial representation ("the object itself") and
the knowledge about it, the a/ntelligent volume" data
structure consists of two levels (Fig. ??). The lower
level contains the spatial knowledge, which is based
on intensity volumes derived from tomographic hnaging techniques like computer tomography (CT), rangnetic resonance imaging (MRI), or positron emission
tomography (PET). If several imaging techniques are
used together (e.g. MRI and PET) these have
be spatially registered, which is not a trivial task
[van den Elsen e~ al., 1993]. Besides the intensity vo]umesthere are one or more attribute volumeseach for a
difi’erent domain of knowledge. A voxel in an attribute
volumeis assigned a numberfor the object the voxel belongs to in this domain. In the word of medical atlases
some possible domains are:
¯ anatomical structure
s functional area
s blood supply area
¯ radiation dose
The upper level of the data structure is a symbolic
knowledgerepresentation, which is stored in a database
and contains both further descriptive knowledgeon the
objects and the relations between them. Examples of
descriptive knowledgeare:
¯ object names in different languages
11
¯ visualization
Singie detailed structures (e.g. gyri or basal ganglia of
the brain) need up to half an hour, and a complete atlas, in which every voxel is classified, might take up to
several manmonth in difficult cases. In order to obtain
a space filling modelit is necessary to segmentat] structures as solid objects. Pure surface definition results in
hollow objects, whose appearance and handling during
later exploration contrast reality.
After segmentation of an object, its spatial extent
is saved in an attribute volume with an automatically
chosen id, which is assigned to every voxel contained
in the object. If the object is already described in the
generic knowledgebase, it is sufficient to specify the ohject name in order to link all available symbolic knowledge (possibly modified) to the new spatial definition.
If the object is still missing in the generic knowledge
base, its symbolic description can be added now.
parameters
¯ textual descriptions
¯ links to reference images (e.g. histological
radiographs)
images,
Relations are defined between objects, in order to
structure the set of objects and to facilitate navigation
amongthe hierarchy of objects during the following exploration session. It depends on the meaning of the
domain, which interpretation of a relation might be useful. In the current implementation in a specific domain
the semantics of all relations is the same and may be
interpreted as:
¯ part of
Application
¯ next to
It is the essential advantage of a data structure like the
aintelligent volume", that it can be explored by arbitrarily browsing through the symbolic and the pictorial
context after the data structure is filled. The information content of the Uintelligent volume"can be explored
in basically three ways, which are not possible with classical image data structures:
¯ suppfiedby
¯ involved in function
The symbolic knowledge is represented in a semantic
network which is defined via a description language,
that can be understood also by non computer scientists
and can therefore easily be maintained or modified.
The declaration of the numeric value (U/d="), which
represents an object in the corresponding attribute volume, is the only direct link between the symbolic and
the spatial knowledge representation. Apart from this
llnk the symbolic description is completely independent
of the spatial knowledge. Thus, if a symbolic description of a part of the body has been defined, this description can be used as a generic knowledge base for
any data set of the same body region. Newdata sets
are first linked with a copy of the generic knowledge
base. A copy of this generic knowledge base can then
be extended or modified for an individual case in order
to take care of pathologies, individual variabilities, or
simply missing objects. After these modifications, the
Imowledge base has become specific for the particular
¯ Composition of views on the basis of conditions to
the symbolic knowledge
(e.g. show all structures supplied by the carotic
artery)
¯ Extraction of symbolic knowledge in the pictorial
context
(e.g. anatomical annotations of object names in the
actual 3D view)
¯ Simulation of medical techniques
(e.g. show all objects along the path of an endoscope)
It is a decisive feature of the approach, that the
knowledgeabout the objects to be visualized is exclusively contained in the ~inte]]igent volume" and not in
the exploration program. Thus visuaiisation environme,is of any object may be created by just filling a
new ~intelligent volume" without changing the visualization program. In our implementation the "intelligent volume" is explored with the program VOXELMAN,which offers a large set of functions for interactire creation, manipulation, and exploration of 3D iraages in an intuitive way. The visualization system uses
volume visualization methods, which have been devcloped in previous projects [Tiede et al., 1990I. More de.
tailed information on the implementation can be found
in [H~hneet al., 1992].
The "intelligent volume" concept has first been applied to the creation of a 3D anatomical atlas of the human brain (VOXEL-MAN/bxain)based on an MRI volume of a head [H~hne ctal., 1992, Tiede et al., 1993,
Schubert et al., 1993]. Three attribute volumes provide
Defining the spatial extent of the objects is a
more lengthy step and has to be carried out for
every case again. For this purpose we are using a very user friendly interactive 3D segmentation
and volume editing system [H~hne and Hanson, 1992,
Automatic segmentation
Schiemannet a/., 1992].
methods are desirable, but currently there is no approach, that would succeed sufficiently independent of
the input data.
The expense of segmentation with theinteractive system depends on the objects that she]] be defined. Gross
anatomy (e.g. brain) can be obtained in less than
minutes from a state of the art MRIvolume data set.
12
owledge on the domains of anatomical structure (curitly 176 objects), blood supply areas (44 objects),
d functional areas (43 objects). Apart from visual~tion parameters and textual descriptions of the oh:ts, there are also ]inks to histological reference iraes stored in the descriptive part of the ~intelligent
[ume’. The spatial definition of all structures of the
~u took several months, as every object is subdivided
great detail.
phological operations. J: Comput. Assist. Tomogr.,
16(2):285-294, 1992.
[H~hne et aL, 1992] K.
H.
HShne,
M. Bomans, M. Riemer, R. Schubert, U. Tiede, and
W. Lierse. A 3D anatomical atlas based on a volume
model. IEEE GompuLG~phica Appl., 12(4):72-78,
1992.
[Lemoine et al., 1991] D. Lemoine,
C. Barillot, B. Gibaud, and E. PasqualJni. An anatomical
based 3d registration system of multimodality and
atlas data in neurosurgery, In A. C. F. Colchester
and D. ~. Hawkes, editors, l~formation Processing
in Medical Imaging, Proc. IPMI ’91, volume 511 of
Lecturc Notes in Computer Science, pages 154-164.
Spnnger-Verlag, Berlin, 1991.
[Mano et al., 1990] I. Mano, Y. Suto, M. Susu]d, and
M. Iio. Computerised three-dimensional normal atlag. Radiat. Mad., 8(2):50-54, 1990.
[Robinson et al., 1993] Glynn P. Robinson, Alan C.F.
Colchester,
and Lewis D. Griffin. Model-Based
Recognition of Anatomical Objects from Medical Images. In Information Processing in Medical Imaging, Proc. IPMI’93, pages 197-211. Springer, Berlin,
1993.
[Schiemann et al., 1992] T. Schiemann, M. Bomans,
U. Tiede, and K. H. H6hne. Interactive
3Dsegmentation. In R. A. Robb, editor, Visualiza~iolt
in Biomedical Computing Ils Proc. SPIE 1808, pages
376-383, Chapel Hill, NC, 1992.
[Schubert etal.,
1993] R. Schubert, K. H. H~hne,
A. Pommert, M. Riemer, T. Schiemann,
and
U. Tiede. Spatial knowledgerepresentation for visualization of human anatomy and function. In H. H.
Barrett and A. F. Gmitro, editors, Information Proceasing in Medical Imaging, Proc. IPMI ’93, pages
168-181. Springer-Verlag, Berlin, 1993.
Conclusions
." show that the ~intelllgent volume" data structure
very useful tool for representing both pictorial and
nbo]ic knowledge in a common scheme. As both
: semantics and visualisation features are contained
the ~intelligent volume", many different kinds of
nes may be visualised and explored with the same
ual exploration environment. This has been shown
;h the example of anatomical atlases using the pro~n VOXEL-MAN
with "intelligent
volumes" deserib.different parts of the humanbody. The function;y of such atlases exceeds by far that of existing
Ltomicai data bases or hypermedia representations of
san anatomy. It is the substantial advantage of the
¯ ~ented approach to enable an infinite numberof dif.~nt possibilities to represent the models knowledge,
~ending on the user’s choice. Hypermedla systems
~w navigation on a possibly high but fixed number
~redefined images and textual descriptions only.
Phe flexibility of the data structure will enable future
ensions without substantial modifications of the sys1’s structure. The space tilting characteristic of the
de/allows to add any kind of knowledge, which can
correlated to a point in space, e.g. the contents of
..ady existing anatomical databases and hypermedia
dications. It is needless to say, that the approach
y be applied for knowledge representation and exration of any other complex volume object.
[Tiede ei al., 1990] U. Tiede, K. H. HShne, M. Bomans,
A. Pommert, M. Riemer, and G. Wiebecke. Investigation of medical 3D-rendering algorithms. IEEE
CompuLGraphics Appl., 10(2):41-53, 1990.
[Tiede et al., 1993] U. Tiede, M. Bomans,K. H. H~hne,
A. Pommert, M. Riemer, T. Schiemann, R. Schubert,
and W. Lierse. A computer~ed three-dlmensional atlas of the humanskull and brain. Am. J. Neuroradiolog~, 14:551-559, 1993.
[van den Elsen et al., 1993] Petra A. van den Elsen,
Evert-;Ian D. Pol, and Max A. Viergever. Medical
image matching -- a review with classification.
IEEE
E~gng. Mad. Biol., 12(1):26-39, 1993.
[Wahler-Lfick et al., 1991] Marion
Wahler-L~ck,
Thomas Sch~ts, and Hans-Joachlm Kretschmann. A
new anatomical representation of the human visual
pathways. Grae/e’s Arch. Clin. Ezp. Oph~halmol.,
229:201-205, 1991.
Acknowledgements
are grateful to the late Prof. Lierse (Dept. of Neunstomy), Prof. Richter (Dept. of Pediatric Radl~y), and all members of our department, who have
ported this work. The original MRIimage sequence
~he head was kindly provided by Siemens Medical
terns (Edangen).
References
eits et 61., 1991] Torgny Greits, Christian Bohm,
yen Holte, and Lars Eriksson. A computerized brain
flag: Construction, anatomical content, and some
pplications. 3. Comput. Assist. Tomogr., 15(1):268, 1991.
hue and Hanson, 1992]
~. H. HShne and William A. Hanson. Interactive
D-segmentation of MRIand CT volumes using mot-
13