A Web-Based Tutorial for Statistical Analysis of
fMV4RI Data
by
Ian Lai
Submitted to the Department of
Electrical Engineering and Computer Science
in partial fulfillment of the requirements for the degrees of
Bachelor of Science in Electrical Engineering and Computer Science
and
Master of Engineering in Electrical Engineering and Computer Science
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June 2003
© Massachusetts Institute of Technology 2003. All rights reserved.
MASSACHUSETTS INSTITUTE
OF TECHNOLOGY
JUL 3 0 2003
.........
Author .....
..
LJBRARIES
Department of
Electrical Engineering and Computer Science
May 21, 2003
C ertified by ..
.....................
Julie Greenberg
Research Scientist
Thesis Supervisor
Certified by..
.......................
Randy Gollub
H3T-Affiliated Faculty
Supervisor
Accepted by . .
Arthur C. Smith
Chairman, Departmental Committee on Graduate Students
BARKER
2
A Web-Based Tutorial for Statistical Analysis of fMRI Data
by
Ian Lai
Submitted to the Department of
Electrical Engineering and Computer Science
on May 21, 2003, in partial fulfillment of the
requirements for the degrees of
Bachelor of Science in Electrical Engineering and Computer Science
and
Master of Engineering in Electrical Engineering and Computer Science
Abstract
A dearth of educational material exists for functional magnetic resonance imaging
(fMRI), a relatively new tool used in neuroscience research. A computer demonstration for understanding statistical analysis in fMRI was developed in Matlab, along
with an accompanying tutorial for its users. The demo makes use of Dview, an existing software package for viewing 3D brain data, and utilizes precomputed data to
improve interactivity. The demo and client were used in an HST graduate course in
methods for acquisition and analysis of fMRI data.
For wider accessibility, a Web-based version of the demo was designed with a
client/server architecture. The Java client has a layered design for flexibility, and the
Matlab server interfaces with Dview to take advantage of its functionality. The client
and server communicate via a simple protocol through the Matlab Web Server. The
Web-based version of the demo was implemented successfully. Future work includes
implementation of additional demo features and expansion of the tutorial before dissemination to a wider group of medical and neuroscience researchers.
Thesis Supervisor: Julie Greenberg
Title: Research Scientist
Thesis Supervisor: Randy Gollub
Title: HST-Affiliated Faculty
3
4
Acknowledgments
I would like to thank my advisors, Julie Greenberg and Randy Gollub, for all their help
and support with the project. Rick Hoge, the developer of Dview, was instrumental
in getting the prototype running, and assisted greatly with his implementation of
the statistical analysis code. Doug Greve and Mark Vangel helped me understand
the analysis behind the project and offered a starting point for the statistical analysis
code. Along with everyone mentioned above, Russ Poldrack read through the tutorial
and helped fine-tune it for HST.583.
I must also thank Mark D'Avila for assisting with the administrative portions of
the project, and the students of HST.583 for testing out the tutorial and demo for
their Lab 5. Vijay Choudhary, the successor for the project, offered invaluable input
while I was drafting the thesis. And lastly, I would like to thank Laura Cerritelli,
Emily Marcus, and others at Epsilon Theta for their support this past year, as well
as my family, without which I could not have completed this project.
5
6
Contents
1 Introduction
9
1.1
Intended Audience
1.2
Scop e
1.3
Roadmap.
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
2 Background
3
9
11
. . . . . . . . . . . . . . . . . . . . .
11
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
. . . . . . . . . . . . . . . . . . . . .
12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
2.1
M otivation ............
2.2
Previous Work
2.2.1
Courses and Workshops
2.2.2
Dview
Standalone Version
15
3.1
Demo
. . . . . . . . . . .
15
3.1.1
Overall Structure .
15
3.1.2
Interface . . . . . .
16
3.1.3
Processing . . . . .
19
3.1.4
Precomputation and Bookkeeping
20
3.2
Tutorial
. . . . . . . . . .
21
3.3
Brain Data Acquisition . .
21
3.4
Current Use . . . . . . . .
21
3.5
Limitations
22
. . . . . . . .
7
4
5
Web-Based Version
25
4.1
Additional Data Sets for Tutorial . . . . . . . . . . . . . . . . . . . .
25
4.2
Web-Based Demo . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
4.2.1
Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
4.2.2
Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
4.2.3
Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
Client Design
31
5.1
User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
5.2
Data Abstractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
5.2.1
View Parameter . . . . . . . . . . . . . . . . . . . . . . . . . .
34
5.2.2
Graph Data . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34
Layered Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
5.3.1
Graphical User Interface (GUI) Layer . . . . . . . . . . . . . .
35
5.3.2
Main Layer
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
41
5.3.3
Data Retrieval Layer . . . . . . . . . . . . . . . . . . . . . . .
42
Package Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
5.3
5.4
45
6 Server Design
Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
6.1.1
Main Server . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45
6.1.2
Dview Interface . . . . . . . . . . . . . . . . . . . . . . . . . .
48
6.2
Precomputation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
48
6.3
Communications Protocol
. . . . . . . . . . . . . . . . . . . . . . . .
49
6.3.1
Request Protocol . . . . . . . . . . . . . . . . . . . . . . . . .
49
6.3.2
Response Protocol
50
6.1
. . . . . . . . . . . . . . . . . . . . . . . .
7 Discussion
57
7.1
Web-Based Demo . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
7.2
Future Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
8
Chapter 1
Introduction
Functional Magnetic Resonance Imaging, or fMRI, is a relatively new tool that has
found widespread use in neuroscience research for a variety of applications, from mapping regions in the brain responsible for the sense of touch to studying the effect of
schizophrenia on the brain. fMRI detects activity in the brain by taking advantage
of the change in magnetic properties of the blood surrounding neuronal activation,
which produces a blood oxygen level dependent (BOLD) signal that can be picked
up by a regular MRI scanner. In a typical fMRI experiment, however, the signal can
be overshadowed by noise, and researchers must rely on statistical analyses to detect
any significant response [1]. This paper describes the technical details of online educational materials to aid researchers in understanding the methods and decisions that
underlie the analyses necessary for interpreting the results of an fMRI experiment.
These materials include the fMRI Data Analysis Demonstration (hereafter referred
to as the demo) and an accompanying lab tutorial.
1.1
Intended Audience
The demo and tutorial target primarily researchers, from physicists studying magnetic
resonance to cognitive neuroscientists, who may have any kind of background in the
subject matter. The materials provide continuing education to these researchers who
9
are interested in fMRI data but might not have the requisite knowledge of statistical
analysis. Graduate students studying fMRI may also find these materials useful for
their ongoing education.
1.2
Scope
The demo and tutorial cover the basic preprocessing steps and parameters commonly
used in analysis of fMRI data, as described in detail in section 3.1.2. More advanced
methods for preprocessing and analysis, as well as the physics of the actual data
acquisition and experimental design, are not included in the tutorial.
1.3
Roadmap
Chapter 2 discusses the motivation for this project and the work that has been done in
this field by other researchers. Chapter 3 describes the standalone, off-line prototype
developed over the summer and fall of 2002. Chapter 4 covers the overall requirements
and platform chosen for the Web-based version of the demo, and Chapters 5 and 6
detail the client and server.
Chapter 7 discusses the progress made and suggests
further improvements for the demo in the future.
10
Chapter 2
Background
2.1
Motivation
Although MRI has been used since the 1950s, fMRI was only recently discovered in
the early 1990s [2]. Because the field of research using fMRI is still relatively young,
it is dominated by investigators who either know the how fMRI works inside out, or
simply know how to use the popular data analysis software packages for their research
data. These packages often include a multitude of parameters for their analysis, which
can be readily optimized by the expert fMRI researcher, but are hardly touched by
anyone else [3]. Since the packages have preset defaults that may not be appropriate
for all situations, it is possible for researchers to draw false conclusions from their
data if they do not have a proper grounding in data analysis. This could affect the
field adversely as it undergoes explosive growth; in the years 1999-2001 alone, more
than 900 abstracts were submitted to the International Conferences on the Functional
Mapping of the Human Brain
[4].
For these reasons, the educational materials will be invaluable to researchers utilizing fMRI in their work. The demo should simulate data analysis so that researchers
can gain insight into the type of decisions they need to make. Each parameter choice
in each part of the preprocessing and statistical analysis should be explained and its
implications made clear, so that a researcher can see how it affects the overall out11
come of the analysis and how to select appropriate settings for analyzing a particular
data set.
2.2
Previous Work
The vast majority of educational materials for fMRI focus on the physics and experimental design, but few exist for fMRI data analysis, and those that do focus primarily
on theory or one specific software package. Textbooks such as [5] have chapters that
provide overviews of data analysis and delve into detail about the theory behind the
statistics of general linear model (GLM) analysis.
Several researchers have posted
online material on the Web to explain the basics of fMRI with some detail [6-13],
but they cover data analysis briefly and sometimes in the context of software analysis
packages [8,12], such as SPM [14] or Brain Voyager [15]. A few journal and conference
publications explain statistical analysis, but more often they describe new methods
at the cutting edge [16-19].
2.2.1
Courses and Workshops
Semester-long courses on fMRI exist at various universities. Courses specifically cov-
ering fMRI at University of Michigan [20], University of Waterloo [21], UCLA [22],
and the Harvard-MIT Division of Health Sciences and Technology (HST) [23] include
several lectures covering data analysis and the statistics underlying the analyses. The
fMRI courses at University of West Ontario [24] and Brown Medical School [25] cover
some data analysis, while the programs at CalTech [27] and Columbia University [26]
do not seem to emphasize analysis in lectures.
Another source of fMRI education comes in the form of workshops that range from
a one-day analysis session at the Oxford Centre for Functional Magnetic Resonance
Imaging of the Brain (FMRIB)
[28] to a five-day workshop organized by the Institute
of Neurofunctional Imagery (IFR 49) in France [29].
In between there is a three-
day workshop at the Functional Imaging Research Center at the Medical College of
12
Wisconsin (FIRC-MCW) [30] with an hour of analysis lecture and a two-hour session
using AFNI, a popular analysis package [311. The NMR Center at Massachusetts
General Hospital (MGH) [32], as well as fMRI Club Nederland (FCN) [33], offer
workshops with three hours worth of material on data analysis. The Institute for
Advanced Magnetic Imaging and Centrum for Neurosystems (AMI and NeuroHUT)
at the Helsinki Institute of Technology also offered multiple lectures and hands-on
sessions of analysis [34].
In addition, conferences sponsored by the Organization
Human Brain Mapping (OHBM) [35] and the annual meeting of the International
Society for Magnetic Resonance in Medicine (ISMRM) [36] also feature tutorials in
fMRI analysis.
While the educational opportunities for fMRI are growing as the courses and
workshops spread throughout the country, they have limits to the number of people
that can enroll or sign up, and often prove to be quite expensive. The proposed demo
and tutorial, on the other hand, will be provided freely to the public, and do not
place a time constraint as the courses and workshops do on the attendees.
2.2.2
Dview
One interactive educational tool that exists for fMRI is Dview, a Matlab program
developed by Richard Hoge at the MGH-NMR Center. Dview was used in the HST
course "Functional Magnetic Resonance Imaging: Data Acquisition and Analysis," or
HST.583, in several of the lab sessions [37-39] for navigating through brain volumes
and for performing some simple analyses. It allows students to click through a run of
fMRI data, explore its statistical properties, and run several built-in analyses. A lab
manual with a self-paced tutorial accompanied each lab session in the course, guiding
students through using Dview to examine and compare various data sets.
Although Dview was simple to use and worked well for the introductory lab sessions, it did not support more sophisticated analyses and was not suited for teaching
core concepts of GLM analysis; it also contained many features not used in the labs
and served more as a general viewer of brain data than a tool specifically geared
13
toward teaching. However, because Dview has a powerful general-purpose viewer engine and a framework for processing brain data, it would be useful to incorporate into
the demo. GLM analysis and other processing required by the demo could be added
to Dview, and the extra features that might otherwise confuse students could then
be disabled.
14
Chapter 3
Standalone Version
Over the course of the summer and fall of 2002, an off-line, standalone prototype of the
fMRI Data Analysis Demonstration and its accompanying lab tutorial were developed
for use the same fall in the HST fMRI course, HST.583. Below is a description of the
prototype demo and tutorial.
3.1
3.1.1
Demo
Overall Structure
Implemented in Matlab, the standalone demo utilizes Dview as its viewer for brain
volumes, and takes advantage of the existing infrastructure for computing statistical
information from the time-varying brain data. The structure of the demo can be
logically divided into several parts. The interface collects parameters for preprocessing
and statistical analysis from the user and provides a way to select analyses or brain
data for display. The processing backend does the computation for the analyses. The
bookkeeping portion keeps track of precomputed brain volumes for operations that
take a substantial amount of time.
15
3.1.2
Interface
The interface of the demo consists of a panel of menus and buttons separate from
the Dview window; a screenshot of the panel window is included in Figure 3-1 in the
Appendix. The menus allow the user to select the following parameters:
" Input data
" Cost function for motion correction algorithm
" Reference point for motion correction algorithm
" Filter width for spatial filtering (smoothing)
" Signal model
" Order of polynomial for detrending
" Noise model
In addition, parts of the processing, such as motion correction, spatial filtering,
and detrending, can be disabled by toggling the appropriate checkboxes.
The user can also choose to display the following multidimensional plots and maps
with various pushbuttons:
" View the time series brain image (4-D)
" Graph the temporal autocorrelation (4-D)
" Plot the temporal standard deviation map (3-D)
" Plot the statistical (activation) maps (3-D)
" Plot the design matrix of regressors used in analyses
" Plot the histogram of t-values
" Plot the false-positive rate
16
Figure 3-1: A screenshot of the fMRI Data Analysis Demonstration prototype.
17
Organization
The panel divides the menus and buttons into frames that match the flow of processing and analysis: Input Data, Motion Correction, Spatial Filtering, Statistical Model,
and Inference. Since Autocorrelation and Standard Deviation Map are used to examine statistical properties of the data, but are not in the normal flow of processing,
they exist in a separate frame. At each step of processing, the user can change an
appropriate parameter from the ones listed above, and view the corresponding map
and plots of resulting data.
Recall/Save
To faciliate comparison between data arising from applications of different processing
parameters, such buttons as Save Parameters and Recall Parameters are provided
for the user to switch between different sets of parameters. When the user presses
Recall Parameters,a dialog box appears, with a list of the previously saved sets of
parameters. When a set of parameters is chosen, it is automatically applied to the
panel, and the user can now view the data corresponding to the chosen parameters.
A simplified Recall function is used for Autocorrelation and Standard Deviation Map,
which automatically saves the appropriate parameters and type of plot that the user
views; pressing Recall results in a list of previously viewed plots. This special case
allows the user to compare statistical properties of the data more easily.
Dview
Almost all of the maps and graphs are plotted in Dview. When the user presses one
of the pushbuttons to view a map or plot, the demo starts Dview if a session is not
already running, and invokes the appropriate graphing function. For three- and fourdimensional brain data, Dview displays in a column the standard transverse, sagittal,
and coronal views of the brain that correspond to the point in space occupied by a
yellow cursor; the user may click on any of the three views to move the cursor and
18
change the view. A larger axis to the right displays the time-course data for the point
that corresponds to the cursor, if the data has four dimensions.
Since Dview does not have functions for all of the maps and plots, it was modified
to add the necessary functionality. A number of the functions in Dview were also tied
directly to its user interface, so they were changed to allow the demo to invoke them
programmatically, and to automatically select the optimal settings for the maps and
plots used in the demo.
Run Button
A previous version of the demo prototype had pushbuttons labelled Run for the
preprocessing steps of Motion Correction and Spatial Filtering in order to simulate
software analysis packages. If the user changed a parameter that affected data downstream in the processing flow, the pushbuttons for viewing data downstream were
disabled, since the processing had to be redone in order for the data to become available. Although the Run buttons give the user a sense that he is actually performing
the processing, they were dropped because they were felt to hinder interactivity, as
the user has to click on multiple Run buttons in order to view and compare processed
data. Moreover, the preprocessed data in the demo is actually precomputed, so the
Run buttons are more cosmetic than useful in nature; more on the precomputation
will be discussed in section 3.1.4.
3.1.3
Processing
The demo takes advantage of Dview's existing framework for handling and processing both three-dimensional brain maps and four-dimensional time course data when
computing maps and plots. Temporal autocorrelation and standard deviation computation are also built in, so the demo merely invokes the appropriate functions, and
added the ability to calculate temporal autocorrelation and standard deviation after polynomial detrending. The author consulted Douglas Greve at the MGH-NMR
Center for the formulae required for the GLM analysis used in the demo, and Richard
19
Hoge, the creator of Dview, implemented the analysis for use in an earlier lab of the
HST.583. Further modifications were made to adapt the analysis for the demo and
to supplement the existing t-map, p-map, and signal model calculations implemented
for the earlier lab.
3.1.4
Precomputation and Bookkeeping
The preprocessing steps of motion correction and spatial filtering take more computational power than Matlab and Dview can feasibly support while maintaining an
adequate level of interactivity, so motion-corrected and spatially-filtered data are precomputed, using the scripts developed by Richard Hoge for invoking AFNI. Perl is
used to automate the precomputation process for the various permutations of preprocessing parameters, and to add appropriate metadata to each run data file. Certain
metadata that have been corrupted or dropped in the processing are also corrected
at this point. The precomputed brain data is stored in the MINC (Medical Image
NetCDF, or Network Common Data Form) format [40], a common format for MR
brain image data which allows the coordinate mapping used and other arbitrary information to be stored alongside the data.
A system of standard file names was devised to keep track of the precomputed
volumes. Each file has in its name an identification number for the run and the
preprocessing parameters used to generate the file. For instance, the human experiment data used in the demo, motion-corrected using weighted least squares as the
cost function and the first timepoint as the reference point, would have the name
tutorial-3-MC-wls-first .mnc. A bookkeeping module specifically generates these
names given the user's choice of parameters, and passes them on to Dview to load;
if a file has already been loaded into memory, then Dview is instructed to switch its
display to that file.
20
3.2
Tutorial
A self-paced lab tutorial was developed to help students use the demo for learning
key concepts of preprocessing and analysis. It includes step-by-step instructions for
using the demo, as well as explanatory text that provides students with a background
knowledge of the processing steps and questions that probe the students' understanding of the material. The four goals of the tutorial as stated in the text are to help
the students with the following:
" Understanding temporal and spatial correlation in fMRI data;
" Understanding how to construct a statistical model for fMRI data;
" Identifying sources of noise and their contribution to fMRI signals; and
* Understanding the effects of motion correction and spatial filtering on the outcome of statistical analysis of fMRI data.
3.3
Brain Data Acquisition
In order to collect the fMRI brain data for use in the demo and tutorial, two scanning
sessions were performed during the summer at the MGH-NMR Center. The three data
sets ultimately used were a run of the phantom, or a jug of paramagnetic salt solution
in distilled water; a human subject not exposed to stimuli; and the same human
subject viewing a flashing annulus with a black-and-white checkerboard pattern and
a gray background, shown in Figure 3-2.
3.4
Current Use
The demo was deployed in the Athena computing environment for the HST fMRI
course HST.583, and the lab tutorial posted on the course web site as the fifth laboratory exercise in the class. Students in the course attended two 1.5 hour Monday lab
21
Figure 3-2: The checkerboard annulus pattern used as the stimulus in data acquisition.
sessions, on November 18 and November 25, 2002, during which they worked through
the lab tutorial. During the two weeks of the lab sessions, the students were also
exposed to various lectures on experimental design and statistical analysis, both univariate and multivariate. In addition, they had assigned readings of chapters from [5]
and various papers discussing various methods of statistical analysis. Informal feedback from the students gathered during the lab sessions was positive and indicated
that those who had prepared themselves with the background reading benefited from
the lab [41].
3.5
Limitations
Because of the limited scope of the demo and tutorial, and because of time constraints
in implementing both, not all of the concepts that the author had hoped to include
were operational in time for HST.583 students. In addition, since the collected data
did not display the variety of imaging artifacts and other problems that would be
educationally beneficial to show in the tutorial, these artifacts and problems could
22
not be included.
The interactivity of the demo was limited by both the size of
the precomputed brain data-which were about 30 megabytes for each precomputed
volume-and the fact that they were stored in the "course locker" in the Athena
environment, which required that each workstation running the demo retrieve each
of the files over its network connection.
Even with a high-speed internal campus
connection, and even when Dview retrieves only part of the image data for viewing,
the latency is often on the order of a minute when the brain volume is first loaded.
The Matlab GUI, still in its early stages, also had problems coping with displays in
multiple windows, and sometimes displayed buttons and menus in the wrong window.
23
24
Chapter 4
Web-Based Version
The standalone prototype offered a glimpse of what is possible with the demo and
tutorial, but suffered several limitations described in Section 3.5. Additional brain
data was acquired to address the limitations of the tutorial.
To make the demo
more accessible to researchers via the Internet, the demo was transformed into an
application running over the World Wide Web.
4.1
Additional Data Sets for Tutorial
An additional scanning session was performed at the MGH-NMR Center during January 2003. The same flashing annulus described in 3.3 was presented to a human
subject, with varying degrees of contrast, to obtain brain data where activity was
near the threshold of detection. The human subject was also asked to make several
facial and bodily motions that would disrupt the data acquisition, including yawning,
swallowing, twitching, and stretching. This data is used to illustrate the importance of
several pre-processing steps for detecting signals that are weak or marred by motion.
25
4.2
4.2.1
Web-Based Demo
Requirements
The Web-based demo should meet several requirements.
It should be sufficiently
interactive such that users do not experience an unreasonable delay when examining
and navigating brain data. An interface similar to that presented in the prototype
should appear in the Web-based demo, with a similar set of information available to
the user. To ensure development within a reasonable time frame, the implementation
of the prototype should be leveraged for the Web-based demo.
The demo should also be accessible to the vast majority of researchers interested in
fMRI. In order to have the demo be freely available and accessible to the largest population, it should not require that the user purchase a Matlab license, so downloading
the demo for local execution in Matlab would not be feasible solution. Matlab does
offer a Runtime Server [42] that allows Matlab programs to be converted to standalone applications, but this still does not solve the issue of requiring users to download
an enormous quantity of brain data. Thus it is desirable to keep the data and the
interface on separate computers.
4.2.2
Architecture
Given that the data and interface should be decoupled, a client/server model would
serve as a reasonable architecture; the data would reside on a server machine, and
the interface on the client side would communicate with the server to present the
appropriate graphs to the user. How much of the processing should be done on either
side was decided based on several constraints that govern how large the server and
client can be.
Location of computation
Even if the data all resides on the server, the amount
of disk space and memory required for computation may be too large to expect
of the client computer, especially if accessibility poses a major concern. Also, if the
26
bandwidth between the server and client is low, transmitting the data from the server
to the client for computation would take too much time and decrease interactivity,
given that each run of brain data takes about 30 megabytes. Hence the computation
should be done on the server side whenever possible.
Location of results
The desired model therefore has the server performing all of
the computations as requested by the user via the client, and that the server transmits
to the client the end result of the computations. Here a similar problem arises: the
server could send the client the entire result, but depending on the computation
requested, it may be anywhere in size from a simple line graph to a four-dimensional
brain volume. Assuming that the user only sees part of the data at a time, however,
it is sufficient to keep the result on the server side, and for the client to request the
portion that the user is interested in from the server. For comparison, Dview uses
a similar approach in displaying its three- and four-dimensional brain data, except
instead of having the data be transmitted over the network from server to client, it
is transmitted from the disk to memory; the bulk of the data is not loaded directly
into memory, but rather just the portion displayed to the user.
4.2.3
Platform
Server
Several options were considered for the server platform. One option was to move away
from Matlab and to use another programming language, both for the computation
and for serving data to the client. This approach presented one major advantage,
in that the code could be cleanly built from scratch, and not preserve any of the
complex features of the old viewer. The language of choice could be required to have
basic toolboxes for building a simple Web server, but the same functionality can be
achieved by deploying a standard Web server, such as Apache, which calls the server
program through CGI to handle requests.
A major limitation to this approach, however, is that a rewrite would involve
27
completely reconstructing the viewer in the new language, and common languages
such as C and Java do not have Matlab's matrix-manipulation facilities, which would
have to be written before the server can function, and would take a fair amount
of time to develop.
To be fair, there exist libraries for both that perform matrix
operations, and certain less well-known languages, such as Numeric Python
[43],
have these operations built in. However, the standalone demo also took advantage
of libraries for reading and processing data files in the common MINC format, which
does not exist for languages other than Matlab, C, and Fortran [40], and conversion
routines would still have to be written to bridge the gap from the MINC library
output to a format that a C matrix library would accept.
Another option involved using Matlab to precompute all of the data necessary,
and writing the actual server software in a different language.
This intermediate
approach would preserve the Matlab computational facilities while having the advantages of a server built from scratch. Depending on what output file format the Matlab
precomputation generates, this approach would also require additional libraries, both
for reading the precomputed data and for generating graphical output for the client.
One complication would be generating all of the precomputed data from Matlab; automating it would involve rewriting some of the demo code, which is not very difficult,
but it results in at least an order of magnitude more data than used in the prototype
demo, since only two stages of the processing were precomputed. Given enough disk
space on the server this might not necessarily be a problem.
Yet a third option kept Matlab for both computation and Web-serving.
This
approach would allow much of the old demo code to be readily used, but modified to
output data in the form of image files for the client. Aside from the dangers of code
reuse, the major disadvantages of this option include the limitations of the Matlab
Web Server. Normally in the environment of the Matlab Web Server, the user fills out
an input document which submits the appropriate input variables to the server via
URL-encoded form data in an HTTP POST request, and the server then computes
the output and sends back an output HTML document as the result [44]. The client
28
would have to communicate with the server in much the same way. This method
has been used successfully in other teaching contexts, such as the Spectral Analysis
Interactive Demo in HST.582J/6.555J Biomedical Signal and Image Processing [45].
Ultimately, the author chose to use a hybrid of the second and third options.
The data for the more time-consuming processing steps is precomputed, and the
Matlab Web Server dispatches the server scripts in Matlab to fetch the precomputed
data, perform any necessary minor computations, and return the output to the client.
This method still allows utilization of some of the old demo code, but also increases
interactivity, and makes it easier to switch to a different, non-Matlab environment for
the server. While switching entirely out of a Matlab-based environment would have
allowed for a cleaner design, the hybrid was deemed more feasible to implement in
the given time frame.
Client
Originally, standard HTML forms, perhaps supplemented with some JavaScript to
enhance the interface, were considered sufficient for the client. The interaction betweeen user and program does not go very far beyond viewing of static data delivered
by the server, which should be powerful enough to extract the appropriate views of
graphs and plots for the client. For instance, the brain viewer could be accomplished
using image maps, and the menus and buttons for selecting different processing steps
are standard in HTML.
However, while Matlab is capable of generating images for the Matlab Web Server,
it cannot produce plots in which the exact coordinates for the location of the axes are
specified. The standard methods for producing plots do not allow control over the
sizing or placement of the plot axes with respect to the entire plot. For the navigation
of time-series brain data, it is crucial that the user be able to click on the time-series
plot and move the cursor to that time point.
An intermediate solution of having a Java applet specifically for displaying interactive plots was explored. The server would transmit the data required for the plot,
29
and the applet would take care of plotting the data on the client side, allowing the
user to click on the plot and have the applet send back the coordinates. But the
communication between the applet and the HTML forms containing the processing
steps and the brain image data was found too cumbersome. Refreshing the image
data and plots would require extensive JavaScript or frames, and ultimately the author decided that having the entire client written in Java would offer more power,
robustness, and flexibility. The client then could be either an applet or a regular
Java application. Since ideally the demo would be used long term, the latest version
of the Java platform (Java 2 Platform, Standard Edition 1.4) was chosen, with the
assumption that most Web browsers will support the platform soon.
30
Chapter 5
Client Design
Running either as a Java applet or application, the client of the Web-based demo
is markedly different from the prototype in the interface and design. Below are the
details of the user interface of the demo client, the main data abstractions used, the
architecture, and the package structure.
5.1
User Interface
The user interface for the Web-based demo client is based on that of the standalone
prototype, with a few important differences. First, the prototype avoided changing
the interface of Dview itself to keep it as a separate viewer, but the demo client
integrates the viewer and the panel of processing parameters so that the user does
not have to switch his attention between windows. A screenshot of the integrated
interface appears in Figure 5-1.
Much like Dview, the viewer has a large panel to the right of the transverse,
sagittal, and coronal cross-sectional slices, in which it displays either a zoomed slice
for three-dimensional data or the time series data for four-dimensional data. However,
in addition to integrating the viewer and the panel, the demo client also displays the
design matrix and the motion correction parameters in the same window. When the
user requests a plot or the design matrix, the brain data is instead replaced by a panel
31
Figure 5-1: A screenshot of the client for the Web-based fMRI Data Analysis Demonstration.
32
displaying the appropriate data.
In the standalone prototype, the processing parameters control the computations
performed on the data, and the various View and Plot buttons at different stages of
processing display the data as processed up to that particular stage. However, since
nearly all of the data in the Web-based demo is precomputed, it does not make sense
to require the user to press a button to view the data after changing parameters, as
the response should be close to immediate. Instead, the user should be able to enable
or disable a processing step or alter its parameters, and see that the data is changed
automatically. Therefore, in the demo client, the View and Plot buttons have been
removed, and replaced by a list of different views to choose from:
" Time Series Data (raw, fitted, or hemodynamic response)
" Motion correction parameters
* Statistical properties (autocorrelation or standard deviation map)
" Design matrices
* Statistical maps (t-value or p-value maps)
" Inference plots (false positive rate or histogram of t values)
Since each view may require certain processing steps to be turned on, the view
selection defaults to raw data if turning off a processing step causes the view to become
unavailable.
For instance, if motion correction is turned off, the motion correction
parameters are no longer relevant; if the motion correction parameters were previously
selected, the display changes to show raw data instead.
33
5.2
Data Abstractions
5.2.1
View Parameter
The ViewParameters object represents whether a processing step is enabled, which
processing parameters are selected, and which view is currently requested by the user.
It has matching get and set methods for each parameter, and methods for toggling
and indicating whether each processing step is enabled.
5.2.2
Graph Data
A utility package of graph data abstractions provides a mechanism for communicating
the details of various graphs sent from the server to the client. For simplicity's sake,
each of the graph data objects is immutable.
The abstract GraphData object represents an arbitrary multi-dimensional graph
with axes and an optional title. Each Axis spans a predefined Range and can have an
optional label. The tick marks and labels of each axis can be specified or automatically
computed. Since the plots used in the demo client are all two-dimensional, the objects
representing them all derive from Graph2DData, which specifies a horizontal and
a vertical axis.
PlotData represents a line graph with one or more data series plotted on the
same axes. Each DataSeries encapsulates a list of GraphPoint objects, an optional
GraphStyle specifying the color, and an optional label that may be used for a legend.
MapData represents a two-dimensional image map. An ImageMap object specifies the actual image and its dimensions, and is translated on the graph to a specified
point.
The BrainPoint object represents a point in brain volume data, with or without
a time dimension.
34
5.3
Layered Architecture
The user's selection of different processing steps and navigation of brain data is generalized to a set of parameters, which the client sends to the server as a request for
data. After receiving a response from the server, the client then displays the data in
a manner appropriate to the data returned. This view of the client's responsibilities
suggests an architecture consisting of layers of interaction between the user and the
server. More centralized architectures were also considered for the client, but the
layered architecture seems to give the most flexibility for future changes to the user
interface and server.
The client consists of three different layers, the Graphical User Interface (GUI)
Layer, the Main Layer, and the Data Retrieval Layer, as shown in Figure 5-2. For
reference, Figure 5-3 shows a detailed view of the communications between layers of
the client.
5.3.1
Graphical User Interface (GUI) Layer
As the main interface to the user, the GUI Layer displays brain data and plots based
on the parameters that the user selects. To make the user interface easily modifiable or
adjusted, the GUI Layer is decoupled from the Main Layer via several Java interfaces,
each representing a different part of the user interface.
Interfaces
The Main Layer communicates with the GUI Layer via update methods that instruct
the user interface to refresh its data. To receive notification that the user has clicked
on a plot or changed a parameter, the Main Layer registers itself as a listener of events
caused by the GUI Layer.
Each interface of the GUI Layer can register and unregister listeners that are interested in its events via the addListener and removeListener methods, and return
a listing of listeners via getname-of-interfaceListeners.
35
The finishUpdates
Input
Display
Graphical User Interface Layer
Changed viewing parameters
Updated graph data
Main Layer
Request for graph data
Graph data response
Data Retrieval Layer
Server response
Request for data
Server
Figure 5-2: The layers in the Web-based demo client architecture.
36
GUI Layer
-
Brain2DViewerGU1
-
DesignMatrixViewerGU1
'0
'
U)
GUIListener
updt
BrainViewer
ParameterSelectorGUI
PlotViewerGUI
'0
U)
41)te3D
upd te
GUIListener
4O
41)es
'0
PiotViewerGUIListener
-
ParameterSelectorGUIListener
PlotViewer
Design MatrixViewer
getBrainao
Dispatc her
MainLayer
t
UgetData
(
DataRetriever
Data Retrieval
Layer
Remote DataRetriever
:]
HTTP
POST
Server
Figure 5-3: A detailed view of the communications among layers in the Web-based
demo client. Interfaces are labelled in italics and shown in white boxes, with their
accompanying implementations in adjacent shaded boxes Method calls are represented by thin arrows; the one method which returns data, getData, loops back to
the caller. The arrows between the GUI Layer and the Main Layer represent the
various MouseClicked and update methods. For instance, the update*() label
on the arrow from the DesignMatrixViewer to the DesignMatrixViewerGUI
represents the methods updateDesignMatrix, updateRegressor, and updateStimCovMatrix. See Section 5.3.1 for the list of methods.
37
method informs the GUI Layer that the updates for that interface are completed,
and that the given listener is read for events again.
An alternative to having an interface for each of the possible viewers is to have
a single monolithic interface for the entire GUI Layer, consolidating all of the update methods. While this approach is not altogether undesirable, the division of the
interfaces into separate types of viewer seems logical.
The list of interfaces for the GUI Layer follow, along with the interface-specific
update methods and the methods used to notify their listeners of events.
Brain2DViewerGUI The Brain2DViewerGUI represents a user interface for a standard brain viewer, using two-dimensional cross-sectional slices as the main navigational tool. In addition, it should be able to display a time series for a
four-dimensional brain data set. It must implement the following update methods:
updateMosaic Instructs the brain viewer to display a given mosaic of brain
slices.
updateSlice Instructs the brain viewer to display the given slice.
updateTimeSeries Instructs the brain viewer to display the given time series
data.
In addition, the Brain2DViewerGUI is also required to call the following methods of its registered listeners on the listed conditions:
mosaicViewRequested Invoked when the user requests a mosaic view of
the given cross-sectional view.
The type of view-sagittal, coronal, or
transverse-should be provided.
sliceMouseClicked Invoked when the user clicks on a cross section to move
the cursor in space. The coordinates for the point clicked should be provided.
38
timeSeriesMouseClicked Invoked when the user clicks on the time series plot
to move the cursor in time. The coordinates for the point clicked should
be provided.
For each of the methods that requires the point clicked, the point corresponding
to the axes of the graph displayed should be returned, rather than the physical
screen pixel coordinates.
DesignMatrixViewerGUI The DesignMatrixViewerGUI represents a user interface that displays a design matrix and a stimulus covariance matrix for a given
fMRI experimental paradigm, as well as a regressor used in statistical modelling.
It must implement the following update methods:
updateDesignMatrix Instructs the design matrix viewer to display the given
design matrix.
updateRegressor Instructs the design matrix viewer to display the given regressor.
updateStimCovMatrix Instructs the design matrix viewer to display the
given stimulous covariance matrix.
Also, the DesignMatrixViewerGUI is required to call the following methods of
its listeners on the listed conditions:
designMatrixMouseClicked Invoked when the user clicks on the design matrix to request the display of the regressor corresponding to the vertical
column of the matrix.
sliceMouseClicked Invoked when the user clicks on the regressor.
stimCovMatrixMouseClicked Invoked when the user clicks on the stimulous covariance matrix.
39
PlotViewerGUI The PlotViewerGUI represents a user interface for displaying one
or more two-dimensional line graphs. It must implement the following update
methods:
updatePlot Instructs the plot viewer to display the given plot.
updatePlots Instructs the plot viewer to display the given list of plots in some
appropriate visual arrangement.
When the user clicks on a plot, the PlotViewerGUI is required to call the
mouseClicked method of its listeners, providing the coordinates of the point
clicked and which plot the user clicked on.
ParameterSelectorGUI The ParameterSelectorGUI represents a user interface for
selecting view parameters. It must implement the updateParameters method, which instructs the parameter selector to change its display to reflect that
the given view parameters. When the user has changed the view parameters,
the ParameterSelectorGUI must call the viewParametersChanged method
of its registered listeners.
Implementation
The implementation of each of the viewer interfaces simply consist of standard Swing
panels, with different graphers (described in the next section) tiled to display the
brain and plot data. A StandardGUI object provides a frame divided into two
parts, an area where appropriate the viewer for the user-selected view is displayed,
and the area for the parameter selector, which simply comprises a collection of menus
and buttons for the different view parameters.
Grapher Utility Package
A package of graphing tools were developed to simplify the implementation of the
viewer interfaces. The packages consists of a LineGrapher, which plots line graphs,
40
such as the times series of brain data and the motion correction parameters, and a
MapGrapher, which plots image maps, such as the cross sections of the brain and
the design matrix. Each grapher can optionally display a cursor within the axes, and
uses the standard Swing event mechanism for notification of user input. The graphers
accept the graph data objects described in Section 5.2.2.
Because of its generality, the grapher package can be readily utilized by any other
user interface designed to sit in the GUI Layer. An abstract superclass AxisGrapher
is also provided for creating any other grapher that plots on a set of horizontal and
vertical axes.
5.3.2
Main Layer
The Main Layer constitutes the center of the client. It holds the state of the demo,
communicates with the GUI Layer to update the display, and sends data requests
to the Data Retrieval Layer. The Dispatcher, Brain2DViewer, PlotViewer, and
DesignMatrixViewer comprise the Main Layer. Though simple, the Main Layer
holds the core of the client logic, and is capable of serving and communicating with
different implementations of the GUI and Data Retrieval Layers.
Dispatcher
The Dispatcher serves as the main control of the demo. It keeps track of its contacts
in the Data Retrieval Layer and the GUI Layer, as well as the viewing parameters
chosen by the user.
When the various Viewers would like data from the server,
they call the Dispatcher, and the Dispatcher communicates the request to the Data
Retrieval Layer.
An alternative design would tease out the back-end for the parameter selector into
a separate entity, which would have the sole channel of communication with ParameterSelectorGUI and would communicate parameter changes back to the Dispatcher.
Since the view parameters are an integral part of the state of the client and are also
necessary for the Dispatcher's retrieving data on the Brain2DViewer and PlotViewer's
41
behalf, it seemed the Dispatcher should simply assume the role of the back-end as
well and maintain central control of the view parameter information.
Brain2DViewer
The Brain2Dviewer functions as the back-end to the Brain2DViewerGUI. It keeps
track of the current location of the cursor in the brain, and passes requests for brain
data to the Dispatcher when the user navigates through the brain. When the server
returns with the data, the Brain2DViewer calls appropriate update methods to refresh
the Brain2DViewerGUI.
DesignMatrixViewer
The DesignMatrixViewer, serving as the back-end to the DesignMatrixViewerGUI,
keeps track of the design and stimulus covariance matrices being viewed and the
regressors used in fitting the statistical model. It notifies the GUI to update when
the user requests the design matrix view. When the user clicks on the design matrix,
the DesignMatrixViewer fetches the regressor corresponding to the column that the
user clicked, and instructs the GUI to display it.
PlotViewer
As the back-end to the PlotViewerGUI, the PlotViewer keeps track of the current
plots requested by the user, and passes the plots to the PlotViewerGUI for display.
5.3.3
Data Retrieval Layer
The Data Retrieval Layer handles the client's communications with the server. It
sends the data request from the Main Layer to the remote server, and returns with
the response and data from the server.
A simple interface, DataRetriever, comprises the layer. The one method, getData, accepts a data request in the form of a DataRequest object, and returns the
42
response from the server in the form of a DataResponse object. Depending on the
type of data returned, the DataResponse could be either a BrainDataResponse, a
DesignMatrixDataResponse, or a PlotResponse.
The current implementation of the data retriever, RemoteDataRetriever, converts the provided DataRequest into an HTTP request via the POST method and
sends it to the server. It extracts the response from the server reply, an HTML document, and builds graph data from the corresponding key-value pairs, sending more
requests to the server if the URL for an image map is given. The appropriate type of
DataResponse is created and returned for the graph data received.
Because it is decoupled from the Main Layer via a simple interface, different versions of the Data Retrieval Layer can be used for different types of servers, including
ones serving local data and remote servers that use a different protocol from HTTP.
An alternative DataRetriever interface that was considered consisted of multiple methods for getting different types of data, instead of having a single getData
method. A balance should be sought in where the information about data types is
stored; having multiple methods stores that information in the different methods'
existence, whereas having a single method relies on the information being stored in
the data passed to the DataRetriever. Ultimately, as view parameters contain all of
the information needed for a request, it was decided that having that information
stored in the data was sufficient, and would offer more flexibility should more data
types arise in the future.
5.4
Package Structure
The demo client takes advantage of Java packages to divide the classes and interfaces
into logical groupings. Table 5.1 enumerates the packages in the demo client and
gives a description for each package.
43
Package
edu.mit
.hst583.fdad
edu.mit.hst583.fdad.client
edu.mit .hst583.fdad.client
edu . mit . hst583
.
.gui
fdad. client . gui. standard
edu.mit.hst583.fdad.client.gui.grapher
edu. mit . hst583
.
fdad. client . main
edu.mit.hst583.fdad.client.data
edu.mit .hst583.fdad.lib
edu.mit.hst583.fdad.lib.data
edu.mit .hst583.fdad.lib.graph
edu.mit .hst583.fdad.server
Description
Main package for FMRI Data Analysis
Demo
Package for client application
Client GUI Layer
Standard Client GUI implementation
Grapher utility package
Client Main Layer
Client Data Layer
Package for shared data structures between client and possible Java server
Package for client/server communications objects (requests and responses)
Package for graph data objects
Package for possible Java server
Table 5.1: The list of Java packages and their descriptions for the Web-based demo
client.
44
Chapter 6
Server Design
The demo server consists of Matlab scripts invoked by the Matlab Web Server running
as a CGI program on the server machine. The server architecture, precomputation,
and communications protocol are detailed below.
6.1
6.1.1
Architecture
Main Server
The FMRIDataAnalysisDemoServer script serves as the interface to the client.
It calls the request parser to extract the parameters from the input, loads the appropriate data, dispatches the corresponding data generator to compute or process
precomputed data, and passes it to the response generator to output the reply to the
client in the proper format. A diagram of the server architecture is shown in Figure
6-1. The subsequent sections detail each of the steps above.
Request Parser
The script GetRequestParameters checks the validity of the client input and generates a Matlab structure containing the parameters as requested by the client. Because
the Matlab Web Server automatically creates a structure for input variables, in the
45
FMRIDataAnalysisDemoServer
Request Parser
Response Generator
GenerateServerOutput
GetRequestParameters]
Data Generators
Data Setup
+GeneratePlot
ReadVolumeData H
ReadBetas
-+Generate3DSections
Generate*
GetCachedFiename
FNewUserData
Initialize
ShowSlic~e] ShowSignalStats
ReadFile
-;ShOwPlot-
Dview Engine
Figure 6-1: The architecture of the Web-based demo server. Matlab functions are
boxed, and each arrow indicates a call from one Matlab function to another. Functions that serve as interfaces between the server and the client or the underlying
Dview engine are italicized and represented in unshaded boxes. The Generate* box
represents miscellaneous Generate functions.
46
implementation the parser does a relatively simple conversion between fields in the
input structure and fields in the parameter structure.
Data Setup
After the structure of parameters is extracted, several Dview functions are called
to initialize the UserData structure used throughout Dview for manipulating brain
and plot data. The GetCachedFilename function retrieves the appropriate precomputed file for the given set of parameters, using the same file name convention
set up in the prototype. Actual loading of the data is performed via the function
ReadVolumeData, which reads in the brain data associated with the parameters.
ReadBetas makes use of ReadVolumeData to read in the fitted statistical model
(usually designated as /).
Data Generators
Once the data is loaded, the data generators call on various Dview functions to extract
the graph data requested by the client and to compute additional plots derived from
the data.
Generate3DSections is used for various three- and four-dimensional views of
the brain, such as the raw view and the t-value map view. GeneratePlot is used for
the time-series of the four-dimensional views. Other various Generate functions exist
for motion correction parameters, the design matrix, and various inference plots, such
as the histogram of p values.
Response Generator
A function GenerateServerOutput accepts the output data structure, transforming
each output data field into a format readily inserted into the output response by the
Matlab Web Server mechanism. The appropriate HTML template for the type of
data returned in the server response is chosen and applied to the output data.
47
6.1.2
Dview Interface
While Dview took care of reading the brain volume data and displaying the data very
well in the prototype, it required several modifications to make it a suitable back-end
for the demo server, as the computation was tied directly to the Matlab graphing and
user interface components.
The functions Initialize and ReadFile were used for initializing the UserData
structure and for reading the brain volume data into memory. However, quite a few of
the required fields in UserData are not set up in Initialize, but rather in the launch
script for Dview itself. To minimize change to the Dview interface, a new function
NewUserData is used to set up the remaining required fields.
The main Dview functions that produced graphing output were ShowSignalStats, ShowPlot, and ShowSlice. They processed the brain or plot data, automatically generated labels and axis limits, updated the Dview interface directly through
axis handles, and adjusted mouse handling and context menus. Because the functions
are not used to produce Matlab graphs, but to extract the graph data, the parts that
interacted with Matlab's graphing capabilities were teased out and disabled, and the
graph data explicitly saved in the return structure.
In addition, due to historic reasons, the bulk of the Dview functions necessary
for the back-end were present as subfunctions inside the Dview CallBack function.
The necessary functions were extracted into separate files so they could serve as the
Dview interface to the demo server.
6.2
Precomputation
In the interest of increasing interactivity, most of the data that was computed on
the fly in the prototype is precomputed for the demo server. This includes not only
the motion-corrected and spatially-filtered data, as was the case with the prototype,
but also the standard deviation map, the fitted statistical model, and the statistical
(t-value and p-value) maps. Data that can be trivially computed by Matlab, such as
48
the autocorrelation, and plots that can be extracted readily from three-dimensional
brain data, such as the histogram of t-values, were not considered for precomputation.
Despite the potential for exponential growth in the size of the data set due to
the numerous parameters (detrending, signal model, and noise model), the amount
of precomputed data does not actually pose a problem. This is because the threedimensional maps lack the time component, and the fitted models only require at
most the time data between successive stimulations in the experiments.
Since most of the data is precomputed, greatly simplifying the computations that
Matlab needs to do on the server side, the demo server could potentially be written
in a different language from Matlab for a speed improvement. This alternative was
not pursued because of time constraints.
6.3
Communications Protocol
The protocol for communications between server and client is designed with the constraint of the Matlab Web Server in mind. Because the Matlab Web Server handles
input in URL-encoded form data in an HTTP request using the POST method, and
returns output in an HTML document, the protocol must do the same. Nevertheless,
a great amount of flexibility is still possible, given that the client request need not
contain much information, and that the server response can link to other documents
of different types that the client can retrieve separately. The protocol is also sufficiently simple to implement in another language if the server does not use Matlab or
the Matlab Web Server.
6.3.1
Request Protocol
Because the client request is submitted in the form of URL-encoded form data, it
consists of a set of keys and values corresponding to the view parameters desired by
the client. Table 6.1 lists the different keys and their possible values. All of the keys
must be present, and all keys must have a legitimate value, for the request to be
49
Key
dataset
Possible values
resting, phantom, exp 1, exp 2, exp 3
motion-correction
mccostfunction
mcreference-point
true, false
wls
first, middle
spatial-filtering
sffilter_type
sffilterwidth
true, false
gaussian
2, 4, 6, 8
detrending
dtfunction
dt.polynomialorder
true, false
polynomial
0, 1, 2, 3, 4, 5
statisticalmodelling
stat-signalmodel
statnoisemodel
true, false
gamma, FIR
white, ARI
view
raw, autocorr, hdr, fitted, std dev map, t map,
p map, t histogram, mc params, fpr, matrices
Table 6.1: The keys for the client request and the valid corresponding values.
considered valid by the server; otherwise the server may respond that the request is
invalid.
6.3.2
Response Protocol
The response as returned by the Matlab Web Server is in the form of an HTML
document, although the content of the document is unspecified. For simplicity, the
server response will be plain-text data embedded in the HTML body, consisting of
key-value pairs that represent the graph data to be displayed by the client.
Syntax
The body of the server reponse is delimited by two special comments in the HTML
body, as shown in Figure 6-2 The delimiting comments must each be on a line by
itself. Content outside the server response body should be ignored by the client.
50
<HTML>
<HEAD>
<TITLE>Server response</TITLE>
</HEAD>
<BODY>
<PRE>
<!-- START FMRIDATAANALYSISDEMO DATA -- >
data-type = brain-3d
sagittal-image = "http://web.mit .edu/ilai/images/temp-image-37.png"
sagittal-horz-image-range = 0 252
sagittal-vert-image-range = -50 150
// Singularly useless property
useless-property = "Not used for other purposes"
<!-- STOP FMRIDATAANALYSISDEMO DATA -- >
</PRE>
I am text that should be otherwise ignored.
</BODY>
</HTML>
Figure 6-2: A sample response from the demo server.
51
The body consists of any number of blank lines and lines with a key-value pair,
separated by an equals sign (=). A key consists of a string of alphanumeric characters
or hyphens (-).
Keys are case-sensitive.
A value consists of a list of strings or
numbers, separated by white space; a string is delimited by double quotation marks
("), with the backslash (\) as the escape character. Thus, the string gt5"\f a would
be represented as "gt5\"\\f a".
Numbers are either positive or negative decimal
values.
Continuations are possible by setting the same key again. For instance, the lines
a = 2.0 and a = 4.7 yield the same list as a = 2.0 4.7. For documentation pur-
poses, a line can also be commented out by prepending it with two forward slashes
(//). Since the client considers only the data within the server response body, additional human-readable information can be included elsewhere in the HTML response
as well for testing purposes.
Semantics
Each server response should have the key data-type, which specifies the type of data
included in the response. The current possible values are brain-4d and brain-3d,
for four- and three-dimensional brain data; matrices, for the design and stimulus
covariance matrix; and plot, for one or more two-dimensional line plots. The client
should ignore any keys that it does not recognize.
Each graph (line plot or image map) returned by the server consists of a collection
of key-value pairs. The key-value pairs that specify each graph have keys with the
name of the graph as the prefix, with the suffixes in Table 6.2 expressing the different
properties of the graph. For instance, if the server is returning a line plot called
myplot, the title would have the key myplot-title, and the vertical range would
have the key myplot-vert-range.
52
Key Suffix
Required?
Graph Specifications
-title
-horz-range
no
yes
-vert-range
yes
-horz-axis-label
-vert-axis-label
-horz-tick-values
no
no
no
-horz-tick-labels
no
-vert-tick-values
no
-vert-tick-labels
no
Data Series (in a Line Plot)
-horz-data-n
yes
-vert-data-n
yes
-color-n
no
Image Map
-image
yes
-horz-image-range
yes
-vert-image-range
yes
Value
a string containing the title of the plot
the range of the horizontal axis (low value and
high value)
the range of the vertical axis (low value and high
value)
a string containing the label for the horizontal axis
a string containing the label for the vertical axis
the list of values along the horizontal axis where
tick marks should be drawn
the list of labels for the corresponding tick marks;
present if and only if the tick values are present.
same as -horz-tick-values, but for the vertical
axis
same as -horz-tick-labels, but for the vertical
axis
the list of values along the horizontal axis in the
nth data series
the list of values along the vertical axis in the nth
data series
the color for the nth data series
a string containing the URL for the image map,
usually a pointer to a PNG or JPEG image
the coordinates along the horizontal axis for the
left and right edge of the map
the coordinates along the vertical axis for the bottom and top edge of the map
Table 6.2: The suffixes for keys specifying the different properties of a graph in the
server response.
53
Required Data
For brain-3d, the server is required to return the image maps transverse, sagittal,
and coronal for the cross sections. In additional, the cursor coordinates should be
returned with the keys x-coord, y-coord, and z-coord. For brain-4d, the line plot
time-series-plot is also required, and the time coordinate should be returned with
t-coord.
For the data type matrices, the server is required to return the image maps
design-matrix and stim-cov-matrix. Also required are the regressors corresponding to each column n of the design matrix, regressor-plot-n.
For plot, the server should return the plots plot-n for however many plots the
client should display.
Alternatives Considered
Instead of using a list of key-value pairs for specifying graphs, an alternative protocol
design was considered where the graphs would be specified in a more hierarchical
structure. For instance, the graph would be specified within a block (possibly delimited by curly braces), and the horizontal and vertical axes would be specified in
their own sub-blocks, with different properties listed as key-value pairs within the
sub-blocks. This protocol design would provide a more logical structure for expressing the graph data, but was not utilized due to time constraints in developing the
demo.
Another way of returning data in the form of vectors (such as the x-values in a data
series) would be to save the data in a separate file, either in binary or plain text, and
provide the URL to the data. This approach seems gratuitous for simple line plots,
though it may prove useful if the plots run up against the limitations of a Matlab
Web Server bug, where variables that are too long crash the function that inserts
them into the HTML template for the server response
[46]. The approach could be
useful for caching plots and for matrix data (such as the brain cross sections or the
design matrix), the latter of which is currently saved as image files for simplicity.
54
The author also considered returning the entire response in a separate file in a
binary format, but it seemed that the benefit of having a compact response would
not outweigh the benefit of having an easily readable plain-text format, which would
make it transparent for testing and potentially reduce errors in development.
56
Chapter 7
Discussion
7.1
Web-Based Demo
The Web-based demo client and server were implemented during the spring of 2003
based on the designs detailed in the previous sections. Client-server interaction is operational, and most of the features are fully functional, including navigation through
three- and four-dimensional brain data, displaying statistical maps, and showing the
design matrix. The author has also successfully run the client on multiple platforms,
and the client has run as an applet as well as a standalone application.
7.2
Future Work
Several features present in the standalone prototype but not yet in the Web-based
demo could be implemented in a straightforward manner, such as keeping a parameter history, adding information about the cursor coordinates in the brain and the
statistics of the time-series data, and navigating through the brain by specifying the
coordinates. Other features, such as adding a mosaic view and color bars for thresholding color maps, require additional work on both the server and client, but are
feasible.
More work can be done on just the server as well. Currently the demo server
57
resides on a common machine in the Athena environment, with remote access to
the precomputed data. The author plans to install the server on a faster, dedicated
machine with the data copied locally to further improve the speed of the server. With
full access to the machine, the load on the server can also be analyzed for multiple
client connections.
The lab tutorial could be reworked to include the additional data acquired after
the class. Ideally its effectiveness would also be re-examined and improved by applying the "How People Learn" (HPL) framework [47-49], a strategy for designing
effective learning environments. In the HPL framework, students are presented with
a challenge problem for which they then brainstorm ideas, hear expert opinions, research the problem's background, assess their understanding, and present a solution.
In addition, in order to be useful to researchers outside HST, the tutorial needs to
be expanded so that it does not presume the knowledge that the students in the
HST fMRI course have from past lectures and readings. Material not covered in the
tutorial should be referenced via appropriate links to external sites.
A copy of the demo and tutorial materials will be eventually deployed for the
Biomedical Informatics Research Network (BIRN) [50], which brings together computational resources and data online in order to serve biomedical researchers on the
Internet. An eventual goal of this project is to have the fMRI Data Analysis Demonstration and its accompanying tutorial be disseminated to researchers across the country via BIRN. The demo and tutorial can also serve as an example of implementing
educational tools for other aspects and modalities of medical imaging.
58
Bibliography
[1] E. A. DeYoe, P. Bandettini, J. Neitz, D. Miller, P. Winans, "Functional magnetic resonance imaging (FMRI) of the human brain," Journal of Neuroscience
Methods, vol. 54, no. 2, pp. 171-87, 1994.
[2] M. S. Cohen, Susan Y. Bookheimer, "Localization of Brain Function with
Magnetic Resonance Imaging," Trends in Neurosciences, [Online document],
vol. 17, no. 7, pp. 268-77, 1994, [2002 Dec 1], Available HTTP: http:
//spinwarp.ucsd.edu/fmri/FMRI-TINS.html
[3] S. Gold, B. Christian, S. Arndt, G. Zeien, T. Cizadlo, D. L. Johnson, M. Flaum,
and N. C. Andreasen, "Functional MRI Statistical Software Packages: A Comparative Analysis," In Human Brain Mapping, vol. 6, no. 2, pp. 73-84, 1998.
[4] R. L. Savoy, "History and future directions of human brain mapping and functional neuroimaging," Acta Psychologica,vol. 107, pp. 9-42, 2001.
[5] P. Jezzard, P. M. Matthews, and S. M. Smith, Eds., Functional MRI: An Introduction to Methods. Oxford, UK: Oxford University Press, 2001.
[6] J. P. Hornak, "The Basics of MRI," [Online document], 2002, [2002 Dec 1],
Available HTTP: http://www. cis.rit.edu/htbooks/mri/
[7] "FIDAP Basics Home Page: Spatial Smoothing," [Online document], 8
Nov. 2001, [2002 Dec 1], Available HTTP: http: //lbc.nimh.nih. gov/f idap/
spatialsmooth.html
[8] M. Brett, "Cambridge Imagers: Tutorials," [Online document], 30 Mar. 2001,
[2002 Dec 1], Available HTTP: http://www.mrc-cbu. cam.ac.uk/Imaging/
tutorials
.html
[91 "Introduction to FMRI," [Online document], 14 May 2002, [2002 Dec 1], Available HTTP: http://www.fmrib.ox.ac.uk/fmrijintro/
[10] S. Smith, "FEAT: FMRI Expert Analysis Tool User Guide," [Online document],
10 Oct. 2002, [2002 Dec 1], Available HTTP: http://www.fmrib.ox.ac.uk/
fsl/feat4/
59
[11] D. C. Noll, "A Primer on MRI and Functional MRI," [Online document], Rev.
2.1, 21 June2001, [2002 Dec 1], Available HTTP: http: //www. bme.umich . edu/
~dnoll/primer2. pdf
[12] J. Culham, "fMRI for Dummies," [Online document], 24 Nov. 2002, [2002 Dec
1], Available HTTP: http://defiant.ssc.uwo.ca/Jody-web/fmri4dummies.
htm
[13] S. Clare, "Functional MRI: Methods and Applications," [Online document],
9 Aug. 2000, [2002 Dec 1], Available HTTP: http://www.fmrib.ox.ac.uk/
~stuart/thesis/
[14] Wellcome Department of Cognitive Neurology, "Statistical Paramteric Mapping," [Online document], 15 Nov. 2002, [2002 Dec 4], Available HTTP:
http://www.fil.ion.ucl.ac.uk/spm/
[15] "BrainVoyager: a product from Brain Innovation B. V.," [Online document], 26
Sep. 2002, [2002 Dec 9], Available HTTP: http://www.brainvoyager. com/
[16] S. M. Fitzpatrick and D. L. Rothman, "Meeting Report: Choosing the Right MR
Tools for the Job," Journal of Cognitive Neuroscience, vol. 14, no. 5, pp. 806815, 2002.
[17] D. J. McGonigle, A. M. Howseman, B. S. Athwal, K. J. Friston, R. S. Frackowiak, and A. P. Holmes, "Variability in fMRI: An Examination of Intersession
Differences," Neuroimage, vol. 11, pp. 708-734., 2000.
[18] T. E. Nichols and A. P. Holmes, "Nonparametric Permutation Tests For Functional Neuroimaging: A Primer with Examples," In Human Brain Mapping,
vol. 15, pp. 1-25, 2001.
[19] V. Solo, P. Purdon, R. Weisskoff, E. Brown, "A signal estimation approach
to functional MRI," In IEEE Transactions on Medical Imaging, vol. 20, no. 1,
pp. 26-35, 2001
[20] T. Nichols, "BME 499.098/Biostat 642: Introduction to Functional MRI,"
[Online document], 28 Aug. 2002, [2002 Dec 4], Available HTTP: http:
//www.sph.umich.edu/~nichols/fMRIcourse/
[21] "Introduction to functional magnetic resonance imaging (fMRI)," [Online document], 12 Sep. 2002, [2002 Dec 4], Available HTTP: http://www.arts.
uwaterloo.ca/~jdancker/fMRI/fMRI-index.htm
[22] M. S. Cohen,
Neuroimaging,"
"Topics and Reading Schedule for M285 - Functional
[Online document], 21 November 2002, [2002 Dec 4],
Available HTTP: http://airto.loni.ucla.edu/BMCweb/CourseWork/M285/
M285Schedule . html
60
[23] R. Gollub, "HST-583 Functional Magnetic Resonance Imaging: Data Acquisition and Analysis Fall 2002 Home Page," [Online document], 25 Nov. 2002,
[2002 Dec 4], Available HTTP: http: //web. mit. edu/hst. 583/www/
[24] J. Culham, "Psychology 554: Neuroimaging of Cognition," [Online document],
22 Nov. 2002, [2002 Dec 4], Available HTTP: http://defiant.ssc.uwo.ca/
Jody-web/Psy554/554_syllabus.htm
[25] J. Sanes and J. Eliassen, "Functional Magnetic Resonance Imaging: Theory and
Practice (BN293)," [Online document], 5 Sep. 2001, [2002 Dec 4], Available
HTTP: http://fmri.neuro.brown.edu/BN293/BN293.Syllabus.Fall2001.
pdf
[26] J. Hirsch, "Current fMRI Course: Functional Neuroimaging of Human Brain,"
[Online document], 30 Oct. 2002, [2002 Dec 4], Available HTTP: http: //www.
fmri.org/course
.htm
[27] J. Jovicich, D. Dubowitz, and G. Rees, "An introduction to functional magnetic resonance imaging (fMRI)," [Online document], 4 Apr. 2000, [2002 Dec 4],
Available HTTP: http://www.klab.caltech.edu/~geraint/mri/descrip.
html
[28] S. Smith, "FMRIB MEDx FMRI Analysis Course," [Online document], 21
Nov. 2001, [2002 Dec 4], Available HTTP: http://www.fmrib.ox.ac.uk/
internal/medx/course/
[29] M. Pesenti, "Advanced Course on FMRI: Atelier de Formation ' l'IRM fonctionnelle cerebrale," [Online document], 18 June 2001, [2002 Dec 4], Available
HTTP: http://www.neco.ucl.ac.be/mp/journeesIRMf .html
[30] "Functional Magnetic Resonance Imaging: An Introductory Course," [Online
document], 4 Nov. 2002, [2002 Dec 4], Available HTTP: http://www.firc.
mcw. edu/course/
[31] R. W. Cox, "AFNI Information Central," [Online document], 2 Dec. 2002, [2002
Dec 4], Available HTTP: http://afni.nimh.nih.gov/afni/
[32] "The MGH-NMR/Martinos Center Visiting Fellowship Program in Functional
MRI," [Online document], 20 Nov. 2002, [2002 Dec 4], Available HTTP: http:
//www.nmr.mgh.harvard.edu/fmrivfp/
[33] "fMRI course january 2001," [Online document], 25 Feb. 2002, [2002 Dec 4],
Available HTTP: http: //www. fmri.nl/General%20stuff /id45.htm
[34] R. Hari, "Workshop on fMRI-based experiments," [Online document], 15
Feb. 2002, [2002 Dec 4], Available HTTP: http://futu.hut .f i/fmri.html
61
[35] "fMRI Course - 10 June 2001 (Human Brain Mapping 2001)," [Online document], 6 June 2001, [2002 Dec 4], Available HTTP: http://www.hbm2001.
uci.org.uk/register/course10.html
[36] T. Nichols and S. Smith, "ISMRM Morning Categorical: fMRI Data Analysis,"
[Online document], 28 Feb. 2002, [2002 Dec 4], Available HTTP: http: //www.
ismrm.org/02/morningcat2('02).htm
[37] R. Hoge. "Dview Tutorial: Lab 0," [Online document], 27 Aug. 2002, [2002 Dec
10], Available HTTP: http://www.nmr.mgh.harvard.edu/~rhoge/HST583/
doc/Tutorial.html
[38] R. Hoge. "fMRI Data Acquisition Lab" [Online document], 30 Sep. 2002,
[2002 Dec 10], Available HTTP: http://www.nmr.mgh.harvard.edu/~rhoge/
HST583/doc/HST583-Lab1.html
[39] R. Hoge. "fMRI Physiology Lab" [Online document], 21 Oct. 2002, [2002 Dec
10], Available HTTP: http://www.nmr.mgh.harvard.edu/~rhoge/HST583/
doc/HST583-Lab2.html
[40] "Introduction to Minc," [Online document], [2002 Dec 4], Available HTTP:
http://www.bic.mni.mcgill.ca/software/minc/minc.html
[41] I. Lai, R. Gollub, R. Hoge, D. Greve, M. Vangel, R. Poldrack, J. E. Greenberg.
"Teaching Statistical Analysis of fMRI Data." American Society for Engineer-
ing Education Annual Conference Proceedings, June 2003, Session 2109.
[42] "The Mathworks: MATLAB Runtime Server" [Online document], 2002, [2002
Dec 9], Available HTTP: http://www.mathworks. com/products/runtime/
[43] "Numerical Python," [Online document], 30 Nov. 2002, [2002 Dec 9], Available
HTTP: http://www.pfdubois. com/numpy/
[44] "The Mathworks: MATLAB Web Server" [Online document], 2002, [2002 Dec
9], Available HTTP: http://www.mathworks. com/products/webserver/
[45] J. Greenberg, "HST.582J/6.555J/16.456J Biomedical Signal and Image Processing Spring 2002," [Online document], 7 Mar. 2002, [2002 Dec 9], Available
HTTP: http://web.mit.edu/6.555/www/
[46] "31237: Why do large vectors passed for processing to the MATLAB Web Server
1.2.1(R12.1) crash the server?" [Online document], 2003, [2003 May 20], Avail3 23 7 .
able HTTP: http: //www.mathworks. com/support/solutions/data/ 1
shtml
[47] J. D. Bransford, A. L. Brown, and R. R. Cocking, Eds., How People Learn
Brain, Mind, Experience, and School. Washington, DC: National Academy
Press, 1999.
62
[48] S. P. Brophy, "Guidelines for Modular Design," VaNTH Engineering Research Center Technical Report VANTHLSSPB200001Vi,
[Online document], 2000, [2002 Dec 11], Available HTTP: http://www.vanth.org/vanth/
thrustFiles/141.rtf
[49] D. L. Schwartz, X. Lin, S. Brophy, and J. D. Bransford, "Toward the de-
velopment of flexibly adaptive instructional designs," in Instructional-Design
Theories and Models: A New Paradigm of Instructional Theory, C. Reigeluth,
Ed. Mahwah, NJ: Erlbaum, 1999.
[50] University of California, San Diego, "Biomedical Informatics Research Network," [Online document], 7 Dec. 2002, [2002 Dec 9], Available HTTP: http:
//www
.nbirn.net/
63