Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
SECTION 4: DATA STRUCTURE AND QUERIES
SPECIFIC AIMS
The overall objective of Section 4: Data Structures and Queries is the
organization, representation, classification and visualization of our
collections of structure and function features of the human brain. We
will research and develop the neuroinformatics approaches necessary
to achieve a content based query system capable of answering
database questions about the relationship between image features. We
will accommodate different spatial scales from macroscopic in vivo to
microscopic post mortem, and different modalities from
chemoarchitecture to PET. This neuroinformatics section will focus
on the development of tools and techniques needed for the diversity of
data types and queries about the human brain. We will build on the
progress made during the active grant, remain compatible with the
database BrainMap and create a system that is extensible and
consistent with a multimodality, multisubject probabilistic
representation.
Section 1, Figure1: This section of the grant focuses
on organization, representation, classification and
visualization of structural and fuctional data.
Specific Aim 1. Develop an efficient storage technology for brain images, associated demographic and experimental
descriptors, and derived data such as probability maps, statistical parametric maps, 3D registration vector fields and
summary statistics. Where appropriate, we will build on commercial software, including support for 3D and 4D
representations so that time-series of functional and structural data can be added to the underlying database.
Specific Aim 2. Develop new mathematical querying systems to allow searches through these archived images using a
large range of possible search criteria. Searches based on text key words will be complemented by the ability to search
each database based on the content of the images themselves. We will write a querying interface that is suitable for
experts as well as a broader public. We will create a representational schema that is consistent with each of the image
types (see Sections 1 and 3).
Specific Aim 3. Develop an index of the images, incorporating image content and ancillary demographic data. This
system will permit rapid searches and stratification of the database according to an extendable list of attributes.
Analysis and content-based search and classification will support a variety of services. We will be able to access
baseline information, to gain an understanding of normal brain function for a range of normal parameters and recognize
deviation in morphology and function beyond normal variation, with a long-term objective toward the recognition of
similarities and sub-population abnormalities.
Specific Aim 4. Build visualization tools for presenting the query results. We will include approaches that are based
on volumetric/voxel data structures as well as surface-based approaches that explicitly define geometry.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
BACKGROUND AND SIGNIFICANCE
Brain mapping efforts have made tremendous strides in the acquisition and analysis of image data describing structure
and function. Maps that provide integrated spatial representations of data from multiple modalities and subjects are
also active research areas. The active grant has resulted in the first probabilistic reference system to accommodate the
variabilities inherent in population studies.
Whereas these maps can result in comprehensive collections of data, representing our understanding of the brain, a
database is needed with more interaction in order to utilize this information efficiently. Only with sufficient
intelligence, efficiency, adaptability and extensibility will we be able to take advantage of the richness of brain maps
(Jones, 1996). Not only are multiple data structures a key component of modern imaging methods but so is an everincreasing data volume. The libraries of such images are increasing in size, requiring the use of modern database
technology and content-based search over the images stored in such databases.
One of the early (yet modern) attempts at construction of a brain database was developed by Bloom et al. (1995). They
created an electronic version of the Paxinos and Watson rat atlas (Paxinos & Watson, 1986). Although based upon
Hypercard and only representing structure outlines, the utility was immediately recognized and appreciated. Swanson
(1992) also provided a digital version of his delineations and Felleman and Van Essen (1991) organized image data on
the Macaque monkey as a database. More recently, Fox and Lancaster (1994) have produced BrainMap as part of the
active grant. This database provides a unique view into the spatial location of sites of activation and connections to the
bibliographic source of studies in which data were originally reported. See Preliminary Results – Progress Report,
below.
The desire to incorporate image volumes into the database for more complex and non-textual queries presents some
difficult challenges, the most obvious of which is the need for embedded processing capabilities to answer queries of
content. Arya (1996) developed one of the first database systems that attended to image content from image volumes
in the medical literature. It was called QBISM (Query By Interactive, Spatial Multimedia). However, it was a testbed
and never included the notions of probabilistic representations or the 4th dimension.
Searching Image Databases: Queries based on Key-Words and Image Content
Rapid increases in the size, content and heterogeneity of current medical image databases have resulted in several
technical challenges. In response to these challenges, we propose to develop modern database technologies that support
complex queries and rapid searches over the images stored in our databases. Substantial advantages will result from
combining existing resources with new database technology, particularly as the size and content of the underlying
databases increase.
Today, most databases perform searches using simple Boolean combinations of key words and text. However, largescale databases often handle multi-media objects, such as graphical surface models, 2D, 3D images, animations and
video time-series. Paradoxically, these systems do not yet provide for image retrieval by content; instead they rely on
accessing images and objects in the database via ancillary parameters, such as identifiers, keywords, etc., which must
be affixed to the images and graphical objects when they are created, or typed in later. Content-based querying is
designed to allow direct access to a range of database objects, which are similar in content, without consulting
accessory text labels for each image or object type.
Image database systems, such as QBIC, designed by IBM, and Virage, the product of recent databasing research at UC
San Diego (Niblack et al., 1993; Gupta, 1997), are among the first systems to perform image-based searches. Earlier
work from our Stanford collaborators became an integral part of the IBM system, QBIC. In our current proposal this
work provides vital preliminary information for developing systems for brain image indexation and querying.
Why are content-based searches useful?
The main goal of content-based search systems is the rapid recognition of subtle group patterns in image databases.
Content-based searches rely on a mathematical decomposition of shape or image data into a concise set of feature
vectors. These feature vectors allow efficient comparisons among large ensembles of image data. Similarities can be
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
identified and used to retrieve groups of data with common characteristics. Feature vectors are calculated using image
transformation algorithms that also support compression, rapid transfer and reconstruction of brain image data. We
propose to adapt and extend content-based search algorithms to handle brain image data. The resulting algorithms will
allow services such as:
(1) access to a baseline of information, to gain an understanding of normal brain function for a range of normal
parameters, (2) support for existing projects which aim to recognize deviation in morphology and function that go
beyond normal variation (Thompson et al., 1997a,b), and (3) recognition of group similarities and sub-population
abnormalities, providing triggers to alter or strengthen therapy.
Members of the consortium will develop these strategies. This section of the ICBM project will establish and validate
technology for rapid and advanced querying and indexing of the expanding ICBM brain image databases.
Talairach Atlas Citations
350
Composition of Talairach-space Literature
200
300
250
200
150
Methodolog
y
Morphometr
y
Drug
Activation
Pop
Contrasts
Task
Activation
150
100
50
0
80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96
Year
Section 4, Figure 2: Talairach Coordinates as a Spatial Reference
System for Human Brain Mapping.
Pu
100
bli
ca
tio
ns
50
0
1994
PRELIMINARY RESULTS – PROGRESS REPORT
1995
1996
Section 4, Figure 3: Different topics of papers
published utilizing Talairach space.
The active grant focused on the development of a functional activation database. We created a system to accommodate
a redacted form of the data, storing experimental results. Points of activation are stored in the database with reference
to the original experiment and bibliographic source. Sophisticated interfaces and displays also were created. Progress
on specific aims of the active grant are abbreviated and summarized below.
Structure Probability Maps Database
Structure Probability Maps Database. A special database and server, the Talairach Daemon (TD) (Lancaster, 1997)
was created to provide on-line access to data for structure probabilities. The SP Map database consists of structure
probability anatomical maps (SP_AMs) for brain structures derived from brain regions automatically segmented from
MR images in normal subjects using ANIMAL (Collins, 1995).
The data can be viewed at
http://ric.uthscsa.edu/projects and following the link to the Talairach Daemon. The current SP_AMs for age range of
20-40 years were derived from 100 subjects and this will be refined to include all 450 subjects from all three sites
(MNI, UT, and UCLA). This proposal seeks to expand the age range to 40-90 years for SP_AMs.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Interactive Data Sharing. The Talairach Daemon database server was developed to provide access to the SP Maps as a
memory resident database. This server was expanded to include a database of anatomical structures from the 1988
Talairach Atlas (Talairach, 1988) called Talairach Labels database (Lancaster, 1997). Software interface tools were
developed to easily interface UNIX applications to the two TD databases. An interface to the TD server from
BrainMap was coded into the most recent version of the BrainMap Search & View client. The BrainMap link to the
TD server allows one to ask for probability information or Talairach labels using coordinates retrieved from a query of
the BrainMap database. This software is called BrainMap Search & View 4.0 (Sun Solaris version - freeware at
http://ric.uthscsa.edu; see Projects: BrainMap). An interface to the TD databases was also incorporated into a new
version of the spatial normalization software SN (Lancaster, 1995). The updated application is DIVA (Digital Image
Visualization and Analysis), and the interface provides interactive access to the TD database data by pointing-andclicking on spatially normalized brain images. The software interfaces for both of these applications are interactive
with very good response times from the Talairach Daemon database server. Command line interface (CLI) versions of
the TD interface software were also developed. A query is by x-y-z Talairach coordinates and retrieved data is the set
of data attributes (anatomical names, probability) from the selected database. Recent testing indicates query-retrieval
transaction times of approximately 1-4 seconds for most users. Recent access statistics (Jan-July 1997) for the TD
server indicate widespread (18 countries) and continued use (high number of visits) of the database (375,143 accesses).
Principal users were US based labs (282,548 accesses with up to 4 users per lab) followed by European users (UK
Academic, Switzerland, France, Germany, Australia, and Italy all exceeded 100 accesses) and users in Canada (568
accesses).
Computerized Atlas. The Talairach Daemon database server provides for display of 50% probability boundaries for all
SP_AMs (structure probability atlas). The visualization of these probabilistic structures is provided in the TD Java
Applet accessible at http://ric.uthscsa.edu/projects. Viewing of SP_AMs is enhanced by plotting them as overlays onto
axial section graphical images derived from the 1988 Talairach atlas. Work is underway to provide probability
contours at 10% intervals in this applet. Since the Talairach Daemon viewer was developed using the Java
programming language this solves the problem of distribution, updating, and managing of code. Applets developed
using the Java programming language can be readily accessed using common web browser applications (Netscape and
Microsoft Explorer). Additionally, this can be done from practically any computer platform of interest. As updates are
made to the structures in the probability atlas, users automatically have access to them. Full 3-D versions of the
SP_AMs are available from the MNI ftp site.
BrainMap Modifications. BrainMap Search and View 4.0 is now available for Sun Solaris 2.X. This version of the
BrainMap client contains many enhancements over the previous version that could only be run on a MacIntosh. It was
written using the Galaxy software development environment and can be ported to other platforms. The link from this
application to the Talairach Daemon database server to interactively access SP_AM and Talairach Labels data is
complete. BrainMap supports the data formats for MRI, PET and fMRI through the use Talairach coordinates. Recent
modifications of BrainMap Search & View 4.0 support the use and development of Functional Volumes Models
(FVM).
Section 4, Figure 4: We (Fox) reviewed
seven functional imaging studies of coherent
motion perception, reporting on a total of 74
subjects. The resulting functional volumes
model is illustrated in a BrainMap layout
with bounding boxes for the 95% confidence
intervals applicable for a mean activation
when the group size is ten (the average
group size in the meta-analysis).
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Research Database and Image Archives. A complete set of forms was developed for use in interviewing subjects,
managing data input, and guiding the staff in the proper management of subjects and their data. The
neuropsychological evaluation software application NEUROCOG was developed to automate the screening and
evaluation of subjects. It includes objective measures for tests such as finger tapping, word recognition memory, figure
recognition memory, reaction times, Stroop color-word test, and synonym-antonym test.
The ICBM subject database was developed using the SQL database management system software from Oracle. A web
application, ICBMview, was developed to enter subject information in addition to that which is automatically entered
using a NEUROCOG results file. This program supports browsing, searching, report generation, and editing/entering
of specific subject data into the database. It is accessible on the ICBM web page and is username/password protected
for keeping data secure. The entry/edit feature of ICBMview allows users to search for subject data with many
combinations of criteria including < > and * as a wild card for the fields. The user interface was designed to simplify
access to the data and improve the navigation from subject to subject and between forms for different views of data.
The design was such that it would be easy to use for data verification or to enter new data. Together, ICBMview and
NEUROCOG provide simple and efficient methods to enter all subject data into the ICBM subject database.
The organization, operational strategy, and basic structure of the image archive has been established. The image
archive consists of five high-speed high-capacity disks (45 Gbytes expandable to 90 Gbytes) managed by a Sun Sparc
20/50 computer. This high speed image archive was chosen in favor of an optical archive because the costs are now
similar, and the random data access and higher read/write capability will facilitate better usage by all sites. All images
are stored in the minc file format with a .des file to help others use these files. Archived images are accessed using ftp
from the directories of interest.
Daemon Database Server DATABASE SERVER. The Talairach Daemon database and server were developed to
provide high-speed access to data within the 3-dimensional (3D) Cartesian brain space of the 1988 Talairach atlas
(Talairach, 1988). The global geometric features that define this brain space (origin, orientation, and dimensions) are
well documented (Lancaster, 1995), and methods to perform global spatial normalization of tomographic brain images
well established (Fox 1985, Friston 1989,1995, Evans 1995]. The standardized space provides a natural 3D database
structure for storing and retrieving data attributes of volumetric brain structures using x-y-z coordinates. Currently two
databases are served by the Talairach Daemon, the Talairach Labels database and the Structure Probability Maps
database. For the Talairach Labels database a naming scheme was devised to label the entire 1988 Talairach brain
volume with named structures ranging in scale from hemispheres to cells. This was accomplished using a volumefilling hierarchical anatomical labeling scheme (Lancaster, 1997). The SP Maps database maintains structure
probability anatomic maps (SP_AMs) that provide percent incidence for brain structures derived from measurements in
a population of normal subjects (Mazziotta, 1995).
Daemon. A daemon is a UNIX program that runs as a background process independent of the terminal. It is similar to
the terminate-and-stay-resident (TSR) applications common to personal computers and is used to perform a specific
function without user interaction and with minimal time delay. An example is the UNIX printer daemon that comes
into action when a print service is requested. An attractive feature of a daemon program is that it can be written to
monitor system activities such as communications at some port address, input a message, perform processing based on
the message, and send a response. This feature enables a database developer to fully automate its client-server network
activities. This automated networking capability led to the adoption of the daemon class application as the means to
develop the Talairach Daemon (TD) server.
Daemon Software Design. The Talairach Daemon software was implemented as a multi-threaded application, with
network communication using UNIX sockets, queries and responses with text strings as the data, and a streamlined
query structure and processing stream. The multi-threaded design of the TD server allows new transactions to begin
while prior transactions are processing. This helps to insure that a remote client application will have access to the
server regardless of the level of server activity. UNIX sockets communications occur at a lower level than the file
transfer protocol (ftp) or the hypertext transfer protocol (http) of the control program/internet protocol (TCP/IP)
commonly used for network communications. This process-to-process networking method was deemed to be more
efficient (fewer layers to unravel) and therefore a faster means to communicate than the other protocols. Additionally,
the sockets communication can be easily managed using its own port address and does not have to work through the ftp
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
or http servers that can become busy with other activities. By using text strings for queries and responses the data
formatting task for TD client and server applications is greatly simplified. Additionally, the amount of text for both the
query and the response is very small, generally less than 50 characters for a single transaction.
Volumetric Database Structure. The Talairach Daemon uses a 3D array to establish a volume data structure with xdimension = 170 mm, y-dimension = 210 mm, and z-dimension = 200 mm with 1x1x1 mm3 voxels. These dimensions
are approximately 25% larger than the Talairach atlas brain for x and y directions and extend from above the top of the
cerebrum to below the cerebellum in the z direction. This volume contains 7.14 x106 voxels, each with a unique x-y-z
coordinate. A new database structure was needed for the TD volumetric space to support overlapping of numerous
brain structures, allow rapid random access to data, and handle multiple, simultaneous databases.
Volume Databasing Scheme. The Talairach Daemon was designed for support of the multi-valued SP_AMs and a
voxel encoding scheme was adopted for its initial implementation. The voxel encoding scheme uses a 3-D array of
pointers (one for each voxel) for direct access to the voxel data (Section 4, Figure 5). Using this scheme input
coordinates directly address each 1 mm voxel in the Talairach Daemon’s x-y-z field of view (170 mm, 210 mm, 200
mm). The pointers are used to access voxel records that contain data attributes for brain structures that are present
within the voxel. Voxel records are managed as linked lists to labels, values, or combinations. Though the voxelbased encoding scheme is very efficient for query and retrieval of data, it demands a great deal of memory. The
memory needed for the 3D array of pointers (4 bytes per pointer) is 28.56x106 bytes. Also, each new structure entered
into the database requires additional memory, proportional to its volume in mm3. A secondary data pointer is used
within the voxel records to reduce their memory requirement. By using the data-pointer method for the left hemisphere
(656,630 voxels) the memory voxel-record requirement is reduced from 9,849,450 bytes to 2,626,530 yielding a 4:1
reduction. The data-pointer scheme is used for all structures managed by the Talairach Daemon database.
Talairach
Daemon
Schematic
y (200 mm)
Volume Array
z (210 mm)
1 mm3
voxel
Volume Array
Talairach
x-y-z voxels
Pointers
x (170 mm)
Voxel Record
Voxel Record
Linked List
Pointers
DB, lbl, val
1, Caudate, 90
2, Gyrus Cinguli
2, Left Cerebrum
Section 4, Figure 5: Talairach Daemon Schematic. Note that the voxel record contains data from two databases,
anatomic and functional (SP_AMs & SP_FMs). The design and implementation of the daemon database/server allow it
to represent numerous databases for volumetric brain data.
RESEARCH DESIGN AND METHODS
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Development of New Brain Image Databasing Technologies
The next sections describe the proposed development and implementation of new databasing technologies for brain
image data. Building on the progress made during the active grant, we will develop several systems for indexing,
searching and classification of multidimensional brain images. These systems will allow searching and stratification of
the database according to key attributes, including the image content itself. Development of state-of-the-art
technologies for rapid, content-based image searches will be conducted in collaboration with experts in the field at
Stanford University. The resulting systems will preserve compatibility with the existing database BrainMap. They will
also create a versatile, accessible brain image repository that is expandable and consistent with a multimodality,
multisubject probabilistic representation.
Specific Aim 1:
Develop advanced storage technologies for archiving structural and functional brain image data. We will also archive
associated data such as demographic parameters, structure and function probability maps, and 3D non-linear
registration vector fields (Sections 1-3). The databasing technologies developed for this task will build on commercial
software, and will include specifically-designed support for complex multidimensional brain data, including 3D scalar
and vector volume data and time-series.
Source Data
Appraisal of database storage technologies requires experience with a wide range of clinical and research image data.
Sample image data, including ancillary demographic information, will be derived from two main sources:
(1) ICBM repositories of structural and functional image volumes, time-series, probability maps, and derived
meta-data. Linked information on experimental parameters will include descriptions of stimuli, such as mode of
stimulation, type, length, repeat rate, experiment length and behavioral and psychophysical information. Metadata will include means and statistical distributions for selected descriptive parameters. Results of the automated
neuropsychological exam (administered using the "NEUROCOG" software developed during the active grant)
will also be included, as well as itemized textual information such as prior neuropsychiatric test data.
(2) Collaborators in the ICBM consortium. Outside collaborators will provide a small but complementary testbed of related brain image data. Data acquired elsewhere will be transformed according to the registration
standards for brain images adopted by ICBM for our baseline work (Talairach and Tournoux, 1988) and
compiled into the probabilistic reference system.
Multimedia Storage
Media. The images themselves are of considerable size, but modern commercial database technology is being scaled to
deal with the storage problems. Our current plan is to use Oracle Corporation's version 8 product, which has recently
been released for Sun Solaris platforms. ICBM servers will provide access to large image files for researchers and will,
eventually, provide broader community access. Problems of storing images for effective retrieval are being tackled in
current databasing systems, but this technology must be employed with care to achieve the right balance of cost and
performance. Experience with the PACS systems by several ICBM members has provided valuable guidelines in the
choice of storage devices (Wong, 1996). Storage on optical media supports very large image databases, and emerging
devices (such as DVD) can augment existing storage technology as well.
Oracle-8 Server Architecture. We will incorporate our developments into the Oracle system. The new Oracle-8 server
architecture includes the concept of cartridges, plug-in modules for many image manipulation functions that users of
the databases will employ. It also includes an object request broker (ORB) to facilitate collaborative computing on the
World Wide Web. Several types of cartridges can be purchased from Oracle or associated vendors. For example, we
expect to obtain cartridges for basic image storage and textual analysis. Our own research products will follow the
same paradigm, so that eventually we will develop new application cartridges to aid in multimodal image registration,
content-based image analysis, and indexing for image retrieval
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Compression. We expect to employ compression to a larger extent than in earlier investigations. While transmission
of brain image data for research purposes requires the use of lossless compression, content-based searching of images
can make use of highly condensed versions of the image data derived using ‘lossy’ compression schemes. These
compression schemes distill salient feature vectors from the data at a range of spatial scales, and only the feature
vectors required for subsequent querying and retrieval of images need to be accessed. We will monitor the performance
on different image modalities in order to make practical choices of compression schemes for storage and transmission
(Wong, 1995).
Advances in Data Transmission Technology. Criteria, which make image data manageable for access, visualization
and statistical combination are changing, and will continue to change during this renewal period. We will focus on
developing techniques at a scale where image size and granularity match current resources, rather than devoting efforts
to overcome temporary barriers. We will consider the scalability of chosen technologies, so that our systems will
improve as available data transmission devices improve. Our experience in large-scale PACS operations and
neuroimaging database systems is crucial to this phase (Wong et al., 1997, Wong, 1996), so that the research will lead
to tools that will remain viable for storing, processing, and indexing large-scale image collections.
Managing Multidimensional, Multi-Modality Brain Image Data. The images we will process are much more complex
than those handled by currently available commercial image databases systems [e.g., QBIC, Virage (Niblack et al.,
1993, Gupta, 1997)]. Brain maps and images typically contain 3D spatial plus temporal observations, requiring in
principle an increase of one or two orders of magnitude of effort for their processing. Retention of full spatial and
temporal resolution is absolutely required for accurate identification and segmentation of neuroanatomical regions, and
for precise localization of functional maps. 4D images (i.e., time series of 3D spatial data) must also be archived. The
granularity of these images will differ, because of the variety of acquisition equipment in 3D, and the sampling rate
will differ for observations in the time dimension. For example, functional response to a stimulus may be monitored
over msecs, and perfusion responses may be tracked over periods spanning minutes. This provides additional
challenges in registration and normalization of the acquired images (Toga, Thompson and Payne, 1996). In general,
however, the fourth, temporal dimension is much less densely sampled, although functional imaging of pre- and poststimulus responses are becoming commonplace.
Hierarchical Object-Oriented Database.
To organize data storage for effective retrieval, we will analyze and
adapt the modeling bases now in use at UT and other ICBM sites. We will design an object-oriented, hierarchical data
model of the brain for classifying, describing, and organizing multimodal and multimedia brain information at various
levels of abstraction (Wiederhold, 1995, Brinkley, 1997). Since the summary descriptions, in the form of numerical
statistics, probability maps or statistical parametric maps will exist for some data sets but not others, we will ensure
that missing information does not create difficulties for storage and retrieval algorithms. The consistent use of a
standard, but modern database technology will enhance the shareability of these models (Wiederhold, 1991). Building
on the success of BrainMap, the ICBM database will be strengthened using recent advances in mediation technology,
which allows for heterogeneity in representation and cross-referencing to standard bibliographic resources, as
maintained by the National Library of Medicine (Wiederhold, 1992).
Specific Aim 2:
We will develop new mathematical querying systems to allow rapid searches through large multidimensional brain
image archives. Several types of search will be supported, adding versatility to the system. Searches based on text keywords will be complemented by the ability to search each database based on the content of the images themselves. We
will write a user friendly querying interface.
Background: Encoding of Image Content
Brain image archives, and image archives in general, differ from databases of purely numeric or textual information. In
particular, clinicians and researchers often search through image and video information using knowledge of what the
image or video actually contains, rather than by referring to a list of keywords or descriptions associated with the visual
information. Textual descriptors such as keywords are inadequate to describe a brain image, because the same image
might be described in different ways by different investigators.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
One mathematically rigorous method for encoding the content of a multidimensional brain image (which may be 2D,
3D, or a 4D time-series) is to use algorithms, which extract specific features in the image, and these features are
retained as the basis for subsequent analysis. Basis function methods are widely used in mathematics and engineering
to represent images, and multidimensional signals and data in general. These methods re-express any image as a
weighted sum of features (basis images). Since it is known which system of basis functions was used for the encoding,
only the weights of a small number of features need to be saved, and these can be used subsequently to reconstruct the
image to any required degree of accuracy. In other words, the feature weights alone are mathematically capable of
encoding the content of an image or video. Once extracted, these feature weights represent a very compact set of
parameters which computer algorithms can use to organize, search, and locate necessary visual information in given
images or large sets of images (Gupta, 1997).
Feature vectors are especially useful for rapid searches through image archives based on similarity of image content,
since images, which are similar in content, also have similar feature vectors. Crucially, feature vectors require less
memory to store and are much easier to manipulate than the images themselves. Feature vectors can, therefore, be
regarded as proxies for the image itself, in that they can express the image’s salient features in an optimally compact
format.
Basis Functions in Medical Imaging
For this reason, basis function representations are used to represent images at various stages of algorithms for
computed tomography, image compression and reconstruction, noise suppression, non-linear image registration. More
importantly for this application, basis function representations also make it easier to design algorithms for rapid access
to images with similar content in brain image databases.
ICBM members have extensive expertise in the use of basis functions in the development of image analysis algorithms.
In particular, non-linear image registration algorithms using basis functions have been developed during the active
grant period. These algorithms used radial basis functions in 3 dimensions (Thompson and Toga, 1996b,1997a,b) and
spherical harmonic basis functions to represent components of 3D registration fields and multi-scale features of
spherically-parameterized surfaces (Thompson and Toga, 1996c).
Pattern Recognition Applications
For purposes of pattern recognition and image encoding, feature vectors which represent a given brain image can be
distilled from the image itself in a variety of ways. The success of image compression algorithms relies on the fact that
images may be expressed as linear combination of some basic set of fundamental images, defined by discrete cosine,
wavelet, Fourier series or other systems of basis functions which form a complete orthonormal system defined on the
image domain. In what follows, we focus on one particular basis function representation for brain images, the wavelet
representation, which is a powerful and widely used basis for compression and indexation of multidimensional data in
large digital image databases.
Encoding Image Content using Wavelet Basis Functions
Wavelet algorithms decompose signals (in 1D) or images (in higher dimensions) into different frequency components
(wavelets). Like Fourier series, and other complete orthonormal systems, wavelets are a particular system of basis
functions with substantial advantages for analyzing images. In wavelet analysis, a signal is decomposed into a sum of
wavelets differing in position and scale. A particular wavelet, the Daubechies-wavelet, has mathematical properties
(continuous derivatives, zero integral, and compact support) that provide remarkable results in image analysis and
synthesis (Daubechies, 1988, 1992).
We have developed a system using a combination of Daubechies' wavelets (Wang, 1997) and additional feature vectors
derived from intensity histograms, has been demonstrated to be capable of classifying a 2D image as objectionable or
benign. This algorithm is designed to be coupled with a Web browser, with the goal of screening out pornographic and
other images. This technology makes it possible for young children to use the Internet without being exposed to
objectionable images. The system correctly classifies images with an outstanding recall of 97% on a database of benign
and objectionable 2D color images, depicting a wide range of subjects.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Wavelet Encoding of Multidimensional Brain Images
We will refine and extend the 2D wavelet algorithms (for gray-level and color images) for analyzing 3D and 4D
medical images. We have extensive experience in content-based analysis of 3D medical images (Wong, 1996a, 1997)
and will apply this experience for this project. The strategy is briefly described in the following.
Definition of Functional Maps for Wavelet Analysis
Regions of interest corresponding to major lobar and functional units will first be defined in a range of normal MR
image volumes, using a combination of manual and automated approaches. Each MR image will be registered in
standard ICBM stereotaxic space. For each subject, functional brain maps from a variety of modalities will then be
registered to each individual’s segmented and labeled MR data, defining a range of corresponding regions of interest
(ROIs) for comparison and quantitation.
Features with particular relevance in the analysis and classification of functional brain images include the shape,
magnitude and spatial distributions of functional responses. Once images are co-registered in 3D stereotaxic space,
functional and structural data can be correlated and quantified over time for each subject in the group. Such functional
maps might include, for example, PET blood flow measurements for a range of subjects, in left and right temporal lobe
ROIs. For time-series data, this strategy can be modified to track the progression of functional response within specific
anatomical ROIs over time. Each functional map (in one to four dimensions) for each subject will then provide a basic
element of data for wavelet analysis.
Wavelet decomposition will then be used to process and extract relevant image features from each functional map.
These features will be organized with associated clinical and etiological parameters into the brain image database. Note
that images will be projected onto a wavelet basis of the same dimension as the functional map. The resulting arrays of
wavelet coefficients, together with other image-derived numerical measures, will be stored as a feature vector along
with each functional map to facilitate content-based image retrieval and analysis.
Searching for Similar Functional Maps after Wavelet Encoding
There are several ways to compare functional maps after wavelet encoding. Each method involves computing one or
more than one similarity distance, ranging from Mahalanobis distances in N dimensions to other multivariate random
field statistics for different groups of primitive vectors. One such distance for comparison of deformation maps,
without basis function encoding, has been used by us for the detection of structural abnormalities in Alzheimer’s
Disease (Thompson et al., 1997a). The computation of the similarity distance between wavelet-encoded feature vectors
is performed in two steps. First, for each primitive such as a set of wavelet coefficients or spatially-indexed lists of
local intensities, a similarity distance is computed in the appropriate Hilbert space. These Hilbert space distances are
then combined with empirically-determined weights to produce a final score, used to rank results by similarity. Of
course, the definition of similarity at this point is determined by the set of weights used (Gupta, 1997). Other functional
maps in the database can be retrieved according to their similarity to a given functional map, and clusters of functional
maps, which are similar in the multidimensional feature space, can be identified.
In this project, we will focus on features of functional maps, which differ greatly, in disease from those in normal
populations. Wavelet coefficients derived from spatial maps or 4D time-series are each expected to show significant
inter-group variability, related to metabolic factors of the underlying physiological events. We will attempt to
determine effective wavelet parameters and parameter-based classifiers for features of neurobiological interest.
Conjunction of Wavelet Features with Other Scalar and Textual Search Parameters
Metadata will be important for retrieval as well. The summaries may be in numerical form as coordinate locations of
functional activation sites together with their significance levels. In some cases we have brief time-courses, moving the
associated image data to a fourth dimension. From the retrieval point of view, we will see how well our wavelet
technologies capture this information. We will, of course also adopt existing parameters and search heuristics found
effective at other sites. In time, an ability to search for a functional pattern, and then retrieve references to experiments,
publications, and classification of causal bases for patterns of brain activities is a challenging and vital objective of the
planned research (Wiederhold, 1995).
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Specific Aim 3:
We will develop an index for our entire archive of brain maps and images, incorporating image content and ancillary
demographic data. This system will permit rapid searches and stratification of the database according to an extensible
list of attributes.
Indexing Approaches for Multidimensional Brain Images
The development of image indexing techniques is an active area of research (Faloutsos et al., 1997), and is addressable
directly using wavelet feature vectors derived from each functional map. Conceptually challenging will be the
selection, prioritization, weighting, and combining of parameters for indexing both images and multidimensional brain
maps. Since wavelets allow the extraction of a near-infinite number of coefficients at multiple temporal frequencies
and spatial scales, and other image parameters will be collected as well, structuring and determining an effective
granularity of the index will be crucial to make the link from image-description to retrieval (Wiederhold, 1987).
Existing methods combining multiple indices (e.g., combinations of text-based and content-based queries) do not scale
well. There are many approaches, such as weighting of parameters or hierarchical structuring of searches. However,
their selection is typically ad hoc, and will work where the objectives and users' paradigm are known, or at least fixed.
Within the ICBM consortium we have a diversity of hypotheses and queries. This will allow us to develop an
understanding of the necessary criteria, and allow the investigation of general and adaptive approaches. We do have the
advantage that the indexing parameter space is much smaller and hence easier to manage than the images themselves,
so that techniques, which would not be feasible on the base data, can be investigated.
Algorithms Proposed for Image Indexing
Section 4, figure 3: Content-Based Querying of MR Brain Image Data (data courtesy of Steven Wong, USCF). Images
with similar content can be accessed together based on analysis of their wavelet representations. Mathematical
decomposition of multidimensional image data using the fast wavelet transform produces a sequence of coefficients
which are assembled into a feature vector to represent each databased image. Metrics defined on these vectors allow
identification and retrieval of images with similar content, for subsequent comparison and analysis.
We propose to use a multidimensional (2D to 4D, color and/or gray-scale) image indexing scheme using Daubechies'
wavelet transforms. For large image databases, feature vectors obtained from multi-level wavelet transforms will be
stored in our system to speed up the search. We will apply a fast wavelet transform (FWT) to each image in the
database, using the Daubechies wavelet basis in the appropriate number of dimensions. Wavelet projections will be
determined independently for each of the data channels (3 in the case of color or multi-spectral data, and a single 8 or
16-bit channel in the case of gray-scale images). An array of coefficients of the wavelet transform, and their standard
deviations, will be stored as feature vectors. Given a query image, a multi-step metric will then be used for the search,
as follows.
Queries Based on Image Content
When a user submits a query (see Figure 6), the feature vector for the querying image is computed and matched to the
pre-computed feature vectors of the images in the database. This is done in two phases.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Section 4, Figure 6: Content-Based
Querying of MR Brain Image Data
[preliminary results (Wong)]. Images with
similar content can be accessed together based
on analysis of their wavelet representations.
Mathematical decomposition of multidimensional image data using the fast wavelet
transform produces a sequence of coefficients
which are assembled into a feature vector to
represent each databased image. Metrics
defined on these vectors allow identification
and retrieval of images with similar content,
for subsequent comparison and analysis.
For each image to be inserted into the database, a 4-layer fast wavelet transform is computed using Daubechies'
wavelets. The case of 2D images will be discussed here, although the same principles apply in higher dimensions. For
2D images, the upper-left 8x8 corner of the transform matrix represents the lowest frequency band of the 2D matrix for
the level of wavelet transform we used. The lower frequency bands in the wavelet transform usually represent object
configurations in the images and the higher frequency bands represent texture and local color variation. Extracting a
submatrix of size 16x16 from that corner, we obtain a compression of 256:1 over the original 2D image of 256x256
pixels. We store this as part of the feature vector. Standard deviations of the 8x8 corner submatrix elements are stored
as part of the feature vector as well. Since the standard deviations are computed based on the wavelet coefficients in
the lowest frequency band, this eliminates disturbances arising from image noise and other high-frequency artifacts.
In the first phase of a content-based query, an image is suggested by the user with the goal of finding analogous images
in the database. We compare the feature vector stored for the querying image with the feature vector of each of many
target images in the database. If a logical acceptance criterion fails, then we set the distance of the two images to -1,
which means that the target image will not be further considered in the matching process. Having first a fast and rough
cut, followed by a more refined pass, maintains the quality of the results while improving the speed of matching.
A weighted variation of Euclidean distance is used for the second phase comparison. If an image in the database
differs from the querying image by more than empirically-defined threshold when we compare the wavelet feature
vector, we discard it. The remaining image vectors are used in the final matching, using an extended set of wavelet
vector components.
To further speed up the system, we use a component threshold to reduce the amount of Euclidean distance
computation. That is, if the difference at any component within the feature vectors to be compared is higher than a predefined threshold, we set the distance of the two images immediately to -1 so that the image will not be further
considered in the matching process.
Sensitivity to Fluctuations in Image Brightness. The angle between any two feature vectors in the N-dimensional
feature vector space provides an alternative measure to the Euclidean, Mahalanobis and other statistical distances
discussed above. The cosine value of the angle can be obtained after computing the vector dot product of the two
vectors being compared, in a normed vector space. This alternative metric makes comparisons among images of
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
functional maps robust to color or brightness shifts. Such non-biological intensity shifts in brain data can also be
compensated for using a variety of dedicated tools for radio-frequency inhomogeneity correction, and color uniformity
correction (see Sections 1,3).
A preliminary version of this algorithm has been used, in the context of a 2D photographic image database, to handle
partial hand-drawn sketch queries. We have tested this algorithm on a database of more than 10,000 general-purpose
images and the accuracy is in general better than traditional algorithms. We intend to further develop and refine this
promising algorithm for medical image retrieval purposes.
We expect that actual retrieval will be very rapid. Relevant subsets of the indices can be brought in their entirety into
the memory of modern computers, and then further refinement, ranking and selection of images can take place, without
further reference to the source images. All computations are performed on the feature vectors, rather than the images
themselves. Eventually the highest ranked images can be presented for perusal or further processing. Storage of all
wavelet information would allow reconstitution of the source image, since the full set of wavelet parameters represents
a compressed form of the image, but such recovery is not the intent of ICBM. We will, in fact, often use images of
reduced resolution, or only selected segments for creating the indexes, depending on the search objectives.
Specific Aim 4: Visualization
We will develop methods for quantitative visualizations based on multimodal 4D data structures contained in the
database. Although many visualization tools exist as mature computer programs in the 2D world (apE, Image,
Analyze, Explorer, etc.), there are few comprehensive packages appropriate for very large multidimensional data sets.
We will build extensions to our Display and MARS software packages that produce reconstructed multimodal models
for the measurement of spatial, intensity (functionally and histologically relevant) and correlative statistics. Because
our models can be based on either explicit geometry using surface models or voxels, we will accommodate either. We
also will provide capabilities to transform spatial and intensity data to the appropriate range, field of view and
representation.
All visualizations will be compatible with the calculation of spatial and intensity statistics, both volumetric and
surface-based. Significant restrictions apply to surface models for such calculations to be valid. The algorithms must
be certain that the surface is consistently oriented, that is all surface patches on an outer boundary must face outward
and all surface patches on an inner boundary must face inward. Patches must abut exactly without gaps, so all inner
and outer boundaries are completely specified. Patches cannot intersect. These requirements will be satisfied
automatically with surfaces constructed as isosurfaces, but must be forced explicitly with surfaces formed from
contours. Since we will generate either representation, we will attend to both. For volume models based on voxel
enumeration, volume based statistics such as volume or moments of inertia will be computed by simply summing over
all voxels in the model.
While volumetric quantities like volume or moments are straightforward to calculate for volume models, surface area
computation will be a particular problem. Each voxel could be thought of as a rectangular block and its rectangular
faces treated as part of the surface whenever the voxel is adjacent to a voxel external to the model. This approach is
incorrect, for the same reason the sum of lengths and risers of a staircase are not the same as the diagonal distance the
staircase traverses. Diminishing the step (voxel) size will not improve accuracy of the estimate, which will tend to
remain approximately constant. A better estimate of surface area can be made from an isosurface of the volume model.
Thus, within limits of computational expense, we will employ corroborating methods to assure statistical accuracy.
More difficult visualizations have to do with the results of correlative and comparative queries. We will build
visualization modules that display data using a variety of presentation approaches. For example, isoprobability clouds,
probability thresholds, variability color maps, and other methods will be developed to communicate complex spatial
and intensity relationships across modalities and subjects. Examples of our preliminary work in visualization are found
throughout this proposal.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
LITERATURE CITED
Amarnath Gupta, Ramesh Jain, Visual Information Retrieval, Communications of the ACM,.40 (5), pp 69-79, 1997.
Arya, M., Cody, B., Faloutsos, C., Richardson, J., and Toga, A.W. (1996) Design and implementation of QBISM, a 3D medical image database system, in Multimedia Database Systems; Issues & Research Directions, V.S.
Subrahmanian, and S. Jajodia (eds.) Springer Verlag:79-100.
Arya, M., Cody, W., Faloutsos, C., Richardson, J. & Toga, A.W. 1996 A 3D medical image database management
system. Comp. Med. Image & Graph. 20(4):269-284.
Bloom FE (1995) Neuroscience-knowledge management: slow change so far, Trends Neurosci 1995 Feb;18(2):48-49
Brinkley JF, Rosse C (1997) The Digital Anatomist distributed framework and its applications to knowledge-based
medical imaging.J Am Med Inform Assoc 1997 May;4(3):165-183.
Ch.Faloutsos, H.V. Jagadish, and Yannis Manolooulos: "Analysis of the n-dimensional Quadtree Decomposition for
Arbitrary Hyperrectangles"; IEEE Trans Knowledge & Data Engineering, Vol 8 No. 3, May-June 1997, pp 373383.
Collins DL, Holmes CJ, Peters TM, Evans AC (1995). Automatic 3D model-based neuroanatomical segmentation.
Human Brain Mapping 3:190-208.
Evans 95
Felleman DJ, Van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex.Cereb Cortex,
1991 Jan;1(1):1-47.
Fox 85
Fox and Lancaster 94
Fox PT, Lancaster J, Stewart M, Ganslandt T (1995) BrainMap Database, Research Imaging Center, U. of Texas at San
Antonio, 1995; accessible at http://ric.uthscsa.edu
Friston 95
Friston KJ, Passingham RE, Nutt JG, Heather JD, Sawle GV, Frackowiak RS (1989) Localisation in PET images:
direct fitting of the intercommissural (AC-PC) line, J Cereb Blood Flow Metab 1989 Oct;9(5):690-695.
Gio Wiederhold , Thierry Barsalou, Walter Sujansky, and David Zingmond: "Sharing Information Among Biomedical
Applications"; in T.Timmers and B.I.Blum (eds): Software Engineering in Medical Informatics, IMIA, NorthHolland, 1991, pages 49-84
Gio Wiederhold: "Digital Libraries, Value and Productivity"; Com. ACM, Vol.38 No.4, April 1995, pages 85-96.
Gio Wiederhold: "Mediators in the Architecture of Future Information Systems"; IEEE Computer, March 1992, pages
38-49.
Gio Wiederhold: "Objects and Domains for Managing Medical Knowledge"; Methods of Information in Medicine,
Schattauer Verlag, Vol.34 No.1, pages 1-7, March 1995.
Ingrid Daubechies, Orthonormal bases of compactly supported wavelets, Communications on Pure and Applied
Mathematics, 41(7):909-996, October 1988.
Ingrid Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, 1992.
James Ze Wang, Gio Wiederhold, Oscar Firschein, System for Screening Objectionable Images Using Daubechies'
Wavelets and Color Histograms, Interactive Distributed Multimedia Systems and Telecommunication Services,
Proceedings of the Fourth European Workshop (IDMS'97), Ralf Steinmetz and Lars C. Wolf (Eds.), Darmstadt,
Germany, Springer-Verlag LNCS 1309, September 1997.
Jones SB (1996). Linking Databases in Biotechnology, NeuroImage 4:S59-S60.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.
Principal Investigator/Program Director (Last, first, middle):
Mazziotta, John C.
Lancaster J (1995). SN: Spatial Normalization Software for Human Brain Mapping, Research Imaging Center, U. of
Texas at San Antonio, 1995; accessible at http://ric.uthscsa.edu
Lancaster, J.L., Rainey, L.H., Summerlin, J.L., Freitas, C.S., Fox, P.T., Evans, A.C., Toga, A.W. and Mazziotta, J.C.
(1997) Automated labeling of the human brain: A preliminary report on the development and evaluation of a
forward transform method. Human Brain Mapping (in press).
Lemke H, Huang HK. Applicability of ATM to distributed PACS environment. SPIE vol. 2435, 1995:68-79.
Mazziotta, J.C., Toga, A.W., Evans, A., Fox, P., and Lancaster J. (1995) A Probabilistic Atlas of the Human Brain:
Theory and Rationale for Its Development. NeuroImage, 2:89-101.
Mazziotta, J.C., Toga, A.W., Evans, A.C., Fox, P.T. and Lancaster, J.L. 1995
Neuroscience. 18 (5): 210-211.
Digital Brain Atlases. Trends in
Paxinos, G. and Watson, C. (1986). The Rat Brain: In Stereotaxic Coordinates. Academic Press.
Swanson, L.W. (1992) Brain Maps: Structure of the Rat Brain, 240 pp., Elsevier, Amsterdam.
Talairach J, Tournoux P, (1988): Co-Planar Stereotaxic Atlas of the Human Brain. New York: Thieme Medical
Publishers.
Thompson, P.M. and Toga, A.W., (1996b). Visualization and Mapping of Anatomic Abnormalities using a
Probabilistic Brain Atlas Based on Random Fluid Transformations, Proc. of the IEEE International Conference on
Visualization in Biomedical Computing: Hamburg, Germany, September 1996, 4:383-392. Also in: Lecture Notes
in Computer Science 131, K.-H. Höhne, R. Kikinis, [eds.], Springer-Verlag.
Thompson, P.M. and Toga, A.W., (1996c). A Surface-Based Technique for Warping 3-Dimensional Images of the
Brain, IEEE Transactions on Medical Imaging, 15(4):1-16, Aug. 1996.
Thompson, P.M. and Toga, A.W., (1997b). Detection, Visualization and Animation of Abnormal Anatomic Structure
with a Deformable Probabilistic Brain Atlas based on Random Vector Field Transformations (Invited Paper),
Medical Image Analysis; paper, with video sequences provided on CD-ROM with Journal Issue [to appear,
November 1997].
Thompson, P.M., MacDonald, D., Mega, M.S., Holmes, C.J., Evans, A.C. and Toga, A.W., (1997a). Detection and
Mapping of Abnormal Brain Structure with a Probabilistic Atlas of Cortical Surfaces, Journal of Computer
Assisted Tomography, 21(4):567-581, Jul.-Aug. 1997.
Toga, A.W., Thompson, P.M. and Payne, B.A., (1996a). Modeling Morphometric Changes of the Brain during
Development, in: Developmental Neuroimaging: Mapping the Development of Brain and Behavior; [eds:] R.W.
Thatcher, G. Reid Lyon, J. Rumsey, N. Krasnegor, Academic Press, pp.15-27, September 1996.
Wang, James Ze, Gio Wiederhold, Oscar Firschein, and Sha Xin Wei: "Wavelet-Based Image Indexing Techniques
with Partial Sketch Retrieval Capability"; IEEE Advances in Digital Libraries (ADL-97), Library of Congress,
Washington, DC, 7 May 1997.
Wayne Niblack et al, The QBIC project: Query image by content using color, texture and shape, Storage and Retrieval
for Image and Video Databases, pages 173-187, San Jose, 1993. SPIE.
Wiederhold, Gio: File Organization for Database Design; McGraw-Hill Book Company, New York, NY, March 1987.
Wong STC, Huang HK. A hospital integrated framework for multimodal image base management. IEEE Trans
Systems, Man, Cybernetics, 26(4), 1996:455-469.
Wong STC, Huang HK. Networked multimedia in medical imaging. IEEE Multimedia, 4(2), 1997:24-36.
Wong STC, Knowlton RC, Hawkin RA, Laxer KL. Multimodal image fusion for noninvasive surgical planning of
epilepsy. IEEE Computer Graphics & Applications, 16(1), 1996:30-38.
Wong STC, Zaremba L, Gooden D, Huang HK. Radiologic image compression. Special issue of Proc. IEEE on Image
and Video Compression, 83(2), 1995:194-219.
PHS 398 (Rev. 5/95)
Page
Number pages consecutively at the bottom throughout the application. Do not use suffixes such as 3a, 3b.