Medical Imaging and Pattern
Recognition
Lecture 1
Medical Imaging Overview
Oleh Tretiak
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
1
Introduction
The first part of the course will provide an
introduction to medical imaging modalities, on
object-image relationship and the relationship
between the state of the object and image
features. The second part of the course will
cover statistical pattern recognition methods
and methodologies for performance
evaluation.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
2
Course Outline
• Part 1: Imaging models and methods
–
–
–
–
–
Introduction
Linear system theory and Fourier analysis
Probability and random processes
Image acquisition and storage
Image processing operations. Unary operations. Linear and
nonlinear filtering. Morphological operations. Segmentation.
– Laboratory 1. Introduction to ImageJ. Scanning.
Characterization of an imaging device.
– Laboratory 2. Image processing. Experiments with standard
operations. Problem solving experiments.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
3
Course Outline: More
• Diagnostic Imaging Modalities
– X-ray imaging: X-ray attenuation and scattering. Contrast
mechanisms. Dose. Three-dimensional imaging.
– Computer tomography: Radon transform and its inversion.
X-ray, SPECT, and PET tomography.
– Magnetic Resonance Imaging. Signal generation. Scanner
operation and inversion. Pulse sequences.
– Laboratory 3. Medical image processing. Experiments with
CT image stacks and image subtraction.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
4
Course Outline: Yet more
• Part 2: Statistical Pattern Recognition
– Foundations of statistical pattern recognition. Statistical
estimation, hypothesis testing.
– Estimation problems in patter recognition. Linear classifiers.
– Neural nets and training rules. Backpropagation.
– Evaluation methods: resubstitution, leave-one-out. ROC
methods: parametric and ordinal dominance. Bootstrap and
permutation.
– Review and overview: topics in medical image research.
– Lab 4. Methods of data plotting and analysis. Pseudorandom
numbers and modeling. Bootstrap.
– Lab 5. Classification with linear classifiers. Experiments on
neural-net training.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
5
Formal Details
• Instructor: Oleh Tretiak
– tretiak@coe.drexel.edu
• Course web site:
– http://www.ece.drexel.edu/faculty/tretiak/KPI/
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
6
This Lecture’s Outline
• Examples of medical images
• The imaging triangle: object, image,
observer
• Imaging categories in medicine
• Physics, biology, and imaging
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
7
Examples of Medical Images
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
8
Questions
• What does the image show?
• What good is it?
• How is it made?
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
9
X-ray Image
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
10
X-ray Image of Hand
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
11
What is it?
• Two X-ray views of the same hand are
formed on an single film by exposing
the hand onto half of the film while the
other half is blocked by an opaque
screen.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
12
What good is it?
• A fracture of the middle finger is seen
on both views, though it is clearer on
the view on the left. This image can be
used for diagnosis - to distinguish
between a sprain and a fracture, and to
choose a course of treatment.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
13
X-ray Imaging: How it works.
X-ray shadow cast by an object
Strength of shadow depends on
composition and thickness.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
14
Summary: X-ray Imaging
• Oldest non-invasive imaging of internal
structures
• Rapid, short exposure time, inexpensive
• Unable to distinguish between soft tissues in
head, abdomen
• Real time X-ray imaging is possible and used
during interventional procedures.
• Ionizing radiation: risk of cancer.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
15
CT (Computed Tomography)
CT Image of plane through
liver and stomach
Projection image
from CT scans
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
16
What Is It?
• Computer Tomography image of section
through upper abdomen of patient prior
to abdominal surgery.
• Section shows ribs, vertebra, aorta, liver
(image left), stomach (image right)
partially filled with liquid (bottom).
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
17
What Good Is It?
• The set of CT images, from the heart
down to the coccyx, was used in
planning surgery for the alleviation of
intestinal blockage.
• The surgery was successful (I’m still
here).
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
18
Computer Tomography:
How It Works
Only one plane is illuminated. Source-subject motion provides
added information.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
19
Fan-Beam Computer
Tomography
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
20
Summary of X-Ray CT
• Images of sectional planes (tomography) are
harder to interpret
• CT can visualize small density differences,
e.g. grey matter, white matter, and CSF. CT
can detect and diagnose disease that cannot
be seen with X-ray.
• More expensive than X-ray, lower resolution.
• Ionizing radiation.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
21
Functional Magnetic Resonance
Imaging
Plane 3
From http://www.fmri.org/
Picture naming task
Plane 6
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
22
What Is It?
• Two of sixteen planes through brain of
subject participating in an image-naming
experiment.
• Images are superposition of anatomical
scans (gray) and functional scans (colored).
• Plane 3 shows functional activity in the visual
cortex (bottom)
• Plane 5 shows activity in the speech area (
image right).
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
23
What Good Is It?
• This set of images is part of research on brain
function (good for publication).
• Functional imaging is used prior to brain
surgery, to identify structures such as the
motor areas that should be avoided, and focal
areas for epilepsy, that should be
resectioned.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
24
MRI Signal Source
 0  H0
H0
0
When a nuclear magnet is tilted
away from the external magnetic
field it rotates (precesses) at the
Larmour frequency. For
hydrogen, the Larmour frequency
is 42.6 MHz per Tesla.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
25
Detected Signal in MRI
Spinning magnetization
induces a voltage in
external coils, proportional
to the size of magnetic
moment and to the
frequency.
H0
0
s(t)
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
26
MRI Image Formation
• Magnetic field gradients cause signals
from different parts of the body to have
different frequencies.
• Signals collected with multiple gradients
are processed by computer to produce
an image, typically of a section through
the body.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
27
Features of MRI
• No ionizing radiation – expected to not have
any long-term or short-term harmful effects
• Many contrast mechanisms: contrast between
tissues is determined by pulse sequences
• Can produce sectional as well as projection
images.
• Slower and more expensive than X-ray
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
28
Magnetic Resonance
Summary
• No ionizing radiation (safe)
• Tomography at arbitrary angle
• Many imaging modes (water, T1, T2,
flow, neural activity)
• Slow
• Expensive
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
29
Ultrasound Imaging
Twin pregnancy during week 10
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
30
What Is It?
• Ultrasound image of a woman’s
abdomen
• Image shows a section through the
uterus. Two embryos in their amniotic
sacs can be seen.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
31
What Good Is It?
• This image allows a safe means for
early identification of a twin pregnancy.
• Obstetric ultrasonography can be used
to monitor high-risk pregnancies to
allow optimal treatment.
• Pre-natal scans are part of baby picture
albums.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
32
Ultrasound Scanner
Transducer
travel
Object
Image
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
• A picture is built up
from scanned lines.
• Echosonography is
intrinsically
tomographic.
• An image is
acquired in
milliseconds, so that
real time imaging is
the norm.
33
Ultrasound Imaging Overview
• Imaging is in real time - used for
interventional procedures.
• Moving structures and flow (Doppler) can be
seen. Used for heart imaging.
• Ultrasound has no known harmful effects (at
levels used in clinical imaging)
• Ultrasound equipment is inexpensive
• Many anatomical regions (for example, Head)
cannot be visualized with ultrasound.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
34
Single Photon Computed
Tomography
Images on left show three
sections through the
heart.
A radioactive tracer,
Tc99m MIBI (2-methoxy
isobutyl isonitride) is
injected and goes to
healthy heart tissue.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
35
What Is It?
• Three sectional (tomographic) images
of a living heart. Colored areas are
measures of metabolic activity of left
ventricle muscle. Areas damaged by an
infarct appear dark. This seems to be a
normal heart.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
36
What Good Is It?
• Used for staging (choosing treatment
before or after a heart attack), and
monitoring the effectiveness of
treatment.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
37
Radionuclide Imaging
• Basic Idea
• Collimator
• Tomography
Basic idea: A substance (drug) labeled with a radioactive
isotope is ingested. The drug goes to selective sites.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
38
Collimator
Only rays that are normal to the camera surface
are detected.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
39
Ga
cammma
e
ra
a
m
m
Ga mera
ca
SPECT
Gamma
camera
Single Photon Emission Computed Tomography.
Shown here is a three-headed tomography system. The
cameras rotate around the patient. A three-dimensional
volume is imaged.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
40
Features of Radionuclide
Imaging
• The image is produced from an agent
that is designed to monitor a
physiological or pathological process
– Blood flow
– Profusion
– Metabolic activity
– Tumor
– Brain receptor concentration
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
41
Fluorescence Microscopy
Image of living tissue
culture cells.
Three agents are used
to form this image.
They bond to the
nucleus (blue),
cytoskeleton (green)
and membrane (red).
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
42
What Is It?
• Optical microscope image of tissue
culture.
• Image is formed with fluorescent light.
• Tree agents are used. They bond to
– DNA in nucleus, blue
– Cytoskeleton, green
– Lipid membranes, red
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
43
What Good Is It?
• This image seems to be a demonstration of
fluorescent agents.
• Tissue culture is used in pharmaceutical and
physiological research, to monitor the effect
of drugs at the cellular level.
• Fluorescent labeling and imaging allows invivo evaluation of the location and
mechanism of a drug’s activity.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
44
Optical Imaging
• Optical imaging (visible and near infrared) is
undergoing very rapid development.
• Like radionuclide imaging, agents can be designed to
bind to almost any substrate.
• Intrinsic contrast, such as oxy- vs. deoxy-hemoglobin
differential absorption are also exploited.
• There has been a growth in new optical imaging
methods.
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
45
Thoughts on Imaging
• Three entities in imaging
– Object
– Image
– Observer
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
46
Image vs. Object
• Images (and vision) are twodimensional
– Surface images
– Projection images
– Sectional images (tomograms)
• Image eliminates data
– 3D object - 2D image
– Moving object - still image
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
47
Creative Imaging
• Imaging procedures create information
– Functional MRI for the first time allows
non-invasive study of the brain
– Doppler ultrasound for the study of flow
– Agents for the study of gene expression,
in-vivo biochemistry
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
48
End of First Lecture
More to Come!
MIPR Lecture 1
Copyright Oleh Tretiak, 2004
49