Basics of digital image processing
Lecture 4
September 23, 2006
What is image processing
„
„
„
Is enhancing an image or extracting
information or features from an image
Computerized routines for information
extraction (eg, pattern recognition,
classification) from remotely sensed
images to obtain categories of information
about specific features.
Many more
Image Processing Includes
„
„
„
„
„
Image quality and statistical evaluation
Radiometric correction
Geometric correction
Image enhancement and sharpening
Image classification
„
„
„
„
„
Pixel based
Object-oriented based
Accuracy assessment of classification
Post-classification and GIS
Change detection
Image Quality
„
Many remote sensing datasets contain high-quality,
accurate data. Unfortunately, sometimes error (or
noise) is introduced into the remote sensor data by:
the environment (e.g., atmospheric scattering,
„
cloud),
random or systematic malfunction of the remote
„
sensing system (e.g., an uncalibrated detector
creates striping), or
improper pre-processing of the remote sensor
„
data prior to actual data analysis (e.g., inaccurate
analog-to-digital conversion).
154
155
Cloud
155
160
162
MODIS
True
143
163
164
Clouds in ETM+
Striping Noise and Removal
CPCA
Combined Principle
Component Analysis
Xie et al. 2004
Speckle Noise and
Removal
Blurred objects
and boundary
G-MAP
Gamma Maximum
A Posteriori Filter
Univariate descriptive image statistics
„
„
„
The mode is the value that
occurs most frequently in a
distribution and is usually the
highest point on the curve
(histogram). It is common,
however, to encounter more
than one mode in a remote
sensing dataset.
The median is the value midway
in the frequency distribution.
One-half of the area below the
distribution curve is to the right
of the median, and one-half is to
the left
The mean is the arithmetic
average and is defined as the
sum of all brightness value
observations divided by the
number of observations.
n
µk =
∑ BV
ik
i =1
n
Cont’
n
„
„
„
„
„
„
„
„
Min
Max
Variance
Standard deviation
Coefficient of
variation (CV)
Skewness
Kurtosis
Moment
vark =
∑ (BV
i =1
ik
− µk )
2
n −1
sk = σ k = vark
CV
σ
=
µ
k
k
Multivariate Image Statistics
„
Remote sensing research is often concerned
with the measurement of how much radiant
flux is reflected or emitted from an object in
more than one band. It is useful to compute
multivariate statistical measures such as
covariance and correlation among the several
bands to determine how the measurements
covary. Variance–covariance and correlation
matrices are used in remote sensing principal
components analysis (PCA), feature
selection, classification and accuracy
assessment.
Covariance
„
The different remote-sensing-derived spectral measurements
for each pixel often change together in some predictable
fashion. If there is no relationship between the brightness
value in one band and that of another for a given pixel, the
values are mutually independent; that is, an increase or
decrease in one band’s brightness value is not accompanied
by a predictable change in another band’s brightness value.
Because spectral measurements of individual pixels may not
be independent, some measure of their mutual interaction is
needed. This measure, called the covariance, is the joint
variation of two variables about their common mean.
n
n
SPkl = ∑ (BVik ×BVil ) −
i =1
n
∑ BV ∑ BV
i =1
ik
i =1
n
il
SPkl
cov kl =
n −1
Correlation
To
Toestimate
estimatethe
thedegree
degreeof
ofinterrelation
interrelationbetween
betweenvariables
variablesin
inaamanner
mannernot
not
influenced
influencedby
bymeasurement
measurementunits,
units,the
thecorrelation
correlationcoefficient,
coefficient,isis
commonly
commonlyused.
used.The
Thecorrelation
correlationbetween
betweentwo
twobands
bandsof
ofremotely
remotelysensed
sensed
data,
data,rrklkl,,isisthe
theratio
ratioof
oftheir
theircovariance
covariance(cov
(covklkl))to
tothe
theproduct
productof
oftheir
their
standard
thus:
standarddeviations
deviations(s(skkssl);
l); thus:
cov kl
rkl =
sk sl
IfIf we
we square
square the
the correlation
correlation coefficient
coefficient (r(rklkl),), we
we obtain
obtain the
the sample
sample coefficient
coefficient ofof
2
determination
whichexpresses
expressesthe
theproportion
proportionof
ofthe
thetotal
totalvariation
variationininthe
thevalues
valuesof
of
determination(r(r2),),which
“band
“bandl”l”that
thatcan
canbe
beaccounted
accountedfor
foror
orexplained
explainedby
byaalinear
linearrelationship
relationshipwith
withthe
thevalues
values
of
of0.70
0.70results
resultsininan
an
ofthe
therandom
randomvariable
variable“band
“bandk.”
k.”Thus
Thusaacorrelation
correlationcoefficient
coefficient(r(rklkl))of
2
rr2value
valueof
of0.49,
0.49,meaning
meaningthat
that49%
49%of
ofthe
thetotal
totalvariation
variationof
ofthe
thevalues
valuesof
of“band
“bandl”l”ininthe
the
sample
sampleisisaccounted
accountedfor
forby
byaalinear
linearrelationship
relationshipwith
withvalues
valuesof
of“band
“bandk”.
k”.
Pixel
Band 1
(green)
Band 2
(red)
Band 3
(ni)
Band 4
(ni)
(1,1)
130
57
180
205
(1,2)
165
35
215
255
(1,3)
100
25
135
195
(1,4)
135
50
200
220
(1,5)
145
65
205
235
example
SP12
(
675)(232 )
= (31,860) −
540
cov12 =
= 135
4
Band 1
(Band 1 x Band
2)
Band 2
130
7,410
57
165
5,775
35
100
2,500
25
135
6,750
50
145
9,425
65
675
31,860
232
5
Band 1
Band 2
Band 3
Band 4
Mean (µk)
135
46.40
187
222
Variance (vark)
562.50
264.80
1007
570
(s k )
23.71
16.27
31.4
23.87
(mink)
100
25
135
195
(maxk)
165
65
215
255
Range (BVr)
65
40
80
60
Univariate statistics
Band 1
Band 1 562.25
Band 2
135
Band 3
718.75
Band 4
537.50
Band 2
Band 3
Band 4
-
-
-
264.8
0
-
-
275.25 1007.5
0
64
Covariance
663.75
570
Band
1
Band 1
-
Band 2 0.35
Band Band 3 Band
2
4
-
-
-
-
-
-
Band 3 0.95
0.53 covariance
-
Band 4 0.94
0.16
0.87
Correlation coefficient
-
Types of radiometric correction
„
„
„
Detector error or sensor error (internal
error)
Atmospheric error (external error)
Topographic error (external error)
Atmospheric correction
Various Paths of
Satellite Received Radiance
„
There are several ways to
atmospherically correct
remotely sensed data.
Some are relatively
straightforward while
others are complex,
being founded on
physical principles and
requiring a significant
amount of information to
function properly. This
discussion will focus on
two major types of
atmospheric correction:
„
„
Total radiance L
S
at the sensor
Solar
irradiance
E
0
Tθ
LT
0
Tθ
2
Ed
1
1,3,5
4
θv
Absolute atmospheric
correction, and
Relative atmospheric
correction.
Scattering, Absorption
Refraction, Reflection
Lp
90Þ
Diffuse sky
irradiance
Remote
sensor
detector
θ0
3
LI
5
Reflectance from
neighboring area,
r
λn
Reflectance from
study area,
rλ
v
60 miles
or
100km
Atmosphere
Absolute atmospheric correction
„
„
Solar radiation is largely unaffected as it travels through the
vacuum of space. When it interacts with the Earth’s atmosphere,
however, it is selectively scattered and absorbed. The sum of
these two forms of energy loss is called atmospheric attenuation.
Atmospheric attenuation may 1) make it difficult to relate handheld in situ spectroradiometer measurements with remote
measurements, 2) make it difficult to extend spectral signatures
through space and time, and (3) have an impact on classification
accuracy within a scene if atmospheric attenuation varies
significantly throughout the image.
The general goal of absolute radiometric correction is to turn
the digital brightness values (or DN) recorded by a remote sensing
system into scaled surface reflectance values. These values can
then be compared or used in conjunction with scaled surface
reflectance values obtained anywhere else on the planet.
a)a)Image
Imagecontaining
containingsubstantial
substantialhaze
hazeprior
priortotoatmospheric
atmosphericcorrection.
correction.b)b)Image
Imageafter
after
atmospheric
atmosphericcorrection
correctionusing
usingATCOR
ATCOR(Courtesy
(CourtesyLeica
LeicaGeosystems
Geosystemsand
andDLR,
DLR,the
the
German
GermanAerospace
AerospaceCentre).
Centre).
relative radiometric correction
„
„
When required data is not available for
absolute radiometric correction, we can
do relative radiometric correction
Relative radiometric correction may be
used to
„
„
Single-image normalization using histogram
adjustment
Multiple-data image normalization using
regression
Single-image normalization using
histogram adjustment
„
„
The method is based on the fact that infrared
data (>0.7 µm) is free of atmospheric
scattering effects, whereas the visible region
(0.4-0.7 µm) is strongly influenced by them.
Use Dark Subtract to apply atmospheric
scattering corrections to the image data. The
digital number to subtract from each band
can be either the band minimum, an average
based upon a user defined region of interest,
or a specific value
Dark Subtract using band minimum
Topographic correction
„
„
„
Topographic slope and aspect also introduce
radiometric distortion (for example, areas in
shadow)
The goal of a slope-aspect correction is to
remove topographically induced illumination
variation so that two objects having the same
reflectance properties show the same
brightness value (or DN) in the image despite
their different orientation to the Sun’s position
Based on DEM, sun-elevation
Conceptions of geometric correction
„
„
Geocoding: geographical referencing
Registration: geographically or nongeographically (no coordination system)
„
Image to Map (or Ground Geocorrection)
The correction of digital images to ground coordinates using ground control
points collected from maps (Topographic map, DLG) or ground GPS points.
„
„
„
Image to Image Geocorrection
Image to Image correction involves matching the coordinate systems or
column and row systems of two digital images with one image acting as a
reference image and the other as the image to be rectified.
Spatial interpolation: from input position to output position or coordinates.
„
RST (rotation, scale, and transformation), Polynomial, Triangulation
„
Root Mean Square Error (RMS): The RMS is the error term used to
determine the accuracy of the transformation from one system to another. It is
the difference between the desired output coordinate for a GCP and the actual.
Intensity (or pixel value) interpolation (also called resampling): The process of
extrapolating data values to a new grid, and is the step in rectifying an image that
calculates pixel values for the rectified grid from the original data grid.
„
Nearest neighbor, Bilinear, Cubic
Image enhancement
„
„
„
„
„
„
„
„
„
„
image reduction,
image magnification,
transect extraction,
contrast adjustments (linear and non-linear),
band ratioing,
spatial filtering,
fourier transformations,
principle components analysis,
texture transformations, and
image sharpening
Contrast Enhancement (stretch)
„
„
Materials or objects reflect or emit similar amounts of radiant flux (so
similar pixel value)
Low-contrast imagery with pixel range less than the designed
radiometric range
„
„
20-100 for TM less than the designed 0-255
To improve the contrast:
„
Linear technique
„
„
„
„
„
Non-linear technique
„
„
Minimum-maximum contrast stretch
Percentage linear contrast stretch
Standard deviation contrast stretch
Piecewise linear contrast stretch
Histogram equalization
Contrast enhancement is only intended to improve the visual quality
of a displayed image by increasing the range (spreading or
stretching) of data values to occupy the available image display range
(usually 0-255). It does not change the pixel values, unless save it as
a new image. It is not good practice to use saved image for
classification and change detection.
Minimum-maximum contrast
stretch
BVout
⎛ BVin − min k
= ⎜⎜
⎝ max k − min k
⎞
⎟⎟quant k
⎠
where:
where:
--BV
BVininisisthe
theoriginal
originalinput
inputbrightness
brightnessvalue
value
--quant
quantkkisisthe
therange
rangeof
ofthe
thebrightness
brightnessvalues
valuesthat
thatcan
canbe
be
displayed
displayedon
onthe
theCRT
CRT(e.g.,
(e.g.,255),
255),
--min
minkkisisthe
theminimum
minimumvalue
valuein
inthe
theimage,
image,
--max
maxkkisisthe
themaximum
maximumvalue
valuein
inthe
theimage,
image,and
and
--BV
is the output brightness value
BVout
out is the output brightness value
Percentage linear and
standard deviation contrast
stretch
„
„
„
X percentage (say 5%) top or low values of the image
will be set to 0 or 255, rest of values will be linearly
stretched to 0 to 255
ENVI has a default of a 2% linear stretch applied to each
image band, meaning the bottom and top 2% of image
values are excluded by positioning the range bars at the
appropriate points. Low 2% and top 2% will be saturated
to 0 and 255, respectively. The values between the range
bars are then stretched linearly between 0 and 255
resulting in a new image.
If the percentage coincides with a standard deviation
percentage, then it is called a standard deviation contrast
stretch. For a normal distribution, 68%, 95.4%, 99.73%
values lie in ±1σ, ±2 σ, ±3 σ. So 16% linear contrast
stretch is the ±1σ contrast stretch.
original
Saturating the water
Stretching the land
Special linear contrast stretch
Or Stretch on demand
Saturating the land
Stretching the water
Piecewise linear contrast stretch
„
„
When the histogram of an image is not
Gaussian (bimodal, trimodal, …), it is
possible to apply a piecewise linear contrast
stretch.
But you better to know what each mode in
the histogram represents in the real world.
Stretch both
land and water
Principle Components Analysis (PCA)
„
There are large correlations among remote sensing bands. PCA will result
in another uncorrelated datasets: principal component images (PCs). PC1
contains the largest variance
„
„
The first two or three components (PCs) contain over 90% of information
from the original many bands. It is a great compress operation
The new principal component images that may be more interpretable than
the original data.
Purposes of image classification
Land use and land cover (LULC)
Vegetation types
Geologic terrains
Mineral exploration
Alteration mapping
…….
What is image classification or
pattern recognition
„
Is a process of classifying multispectral (hyperspectral) images into
patterns of varying gray or assigned colors that represent either
„
„
„
„
clusters of statistically different sets of multiband data, some of which
can be correlated with separable classes/features/materials. This is the
result of Unsupervised Classification, or
numerical discriminators composed of these sets of data that have been
grouped and specified by associating each with a particular class, etc.
whose identity is known independently and which has representative
areas (training sites) within the image where that class is located. This is
the result of Supervised Classification.
Spectral classes are those that are inherent in the remote sensor
data and must be identified and then labeled by the analyst.
Information classes are those that human beings define.
unsupervised classification, The
computer or algorithm automatically
group pixels with similar spectral
characteristics (means, standard
deviations, covariance matrices,
correlation matrices, etc.) into unique
clusters according to some statistically
determined criteria. The analyst then
re-labels and combines the spectral
clusters into information classes.
supervised classification. Identify known a priori
through a combination of fieldwork, map
analysis, and personal experience as training
sites; the spectral characteristics of these sites are
used to train the classification algorithm for
eventual land-cover mapping of the remainder of
the image. Every pixel both within and outside the
training sites is then evaluated and assigned to the
class of which it has the highest likelihood of
being a member.
Hard vs. Fuzzy classification
„
„
Supervised and unsupervised classification
algorithms typically use hard classification logic
to produce a classification map that consists of
hard, discrete categories (e.g., forest,
agriculture).
Conversely, it is also possible to use fuzzy set
classification logic, which takes into account the
heterogeneous and imprecise nature (mix
pixels) of the real world. Proportion of the m
classes within a pixel (e.g., 10% bare soil, 10%
shrub, 80% forest). Fuzzy classification
schemes are not currently standardized.
Pixel-based vs. Object-oriented
classification
„
„
In the past, most digital image classification was based on
processing the entire scene pixel by pixel. This is commonly
referred to as per-pixel (pixel-based) classification.
Object-oriented classification techniques allow the
analyst to decompose the scene into many relatively
homogenous image objects (referred to as patches or
segments) using a multi-resolution image segmentation
process. The various statistical characteristics of these
homogeneous image objects in the scene are then subjected
to traditional statistical or fuzzy logic classification. Objectoriented classification based on image segmentation is often
used for the analysis of high-spatial-resolution imagery (e.g.,
1 × 1 m Space Imaging IKONOS and 0.61 × 0.61 m Digital
Globe QuickBird).
Unsupervised classification
„
„
„
Uses statistical techniques to group n-dimensional data into their natural
spectral clusters, and uses the iterative procedures
label certain clusters as specific information classes
K-mean and ISODATA
„
„
„
For the first iteration arbitrary starting values (i.e., the cluster properties)
have to be selected. These initial values can influence the outcome of the
classification.
In general, both methods assign first arbitrary initial cluster values. The
second step classifies each pixel to the closest cluster. In the third step the
new cluster mean vectors are calculated based on all the pixels in one
cluster. The second and third steps are repeated until the "change" between
the iteration is small. The "change" can be defined in several different ways,
either by measuring the distances of the mean cluster vector have changed
from one iteration to another or by the percentage of pixels that have
changed between iterations.
The ISODATA algorithm has some further refinements by splitting and
merging of clusters. Clusters are merged if either the number of members
(pixel) in a cluster is less than a certain threshold or if the centers of two
clusters are closer than a certain threshold. Clusters are split into two
different clusters if the cluster standard deviation exceeds a predefined value
and the number of members (pixels) is twice the threshold for the minimum
number of members.
Supervised classification:
training sites selection
„
„
„
Based on known a priori through a combination of fieldwork,
map analysis, and personal experience
on-screen selection of polygonal training data (ROI), and/or
on-screen seeding of training data (ENVI does not have
this, Erdas Imagine does).
„
The seed program begins at a single x, y location and evaluates
neighboring pixel values in all bands of interest. Using criteria
specified by the analyst, the seed algorithm expands outward like
an amoeba as long as it finds pixels with spectral characteristics
similar to the original seed pixel. This is a very effective way of
collecting homogeneous training information.
„
From spectral library of field measurements
Selecting
ROIs
Alfalfa
Cotton
Grass
Fallow
Supervised classification methods
„
„
„
„
Various supervised classification algorithms may be used to assign an unknown pixel to one
of m possible classes. The choice of a particular classifier or decision rule depends on the
nature of the input data and the desired output. Parametric classification algorithms
assumes that the observed measurement vectors Xc obtained for each class in each spectral
band during the training phase of the supervised classification are Gaussian; that is, they are
normally distributed. Nonparametric classification algorithms make no such assumption.
Several widely adopted nonparametric classification algorithms include:
one-dimensional density slicing
„
parallepiped,
„
minimum distance,
„
nearest-neighbor, and
„
neural network and expert system analysis.
„
The most widely adopted parametric classification algorithms is the:
maximum likelihood.
„
Hyperspectral classification methods
Binary Encoding
„
Spectral Angle Mapper
„
Matched Filtering
„
Spectral Feature Fitting
„
Linear Spectral Unmixing
„
Supervised
classification
method:
Spectral Feature
Fitting
Source: http://popo.jpl.nasa
.gov/html/data.html
Accuracy assessment of classification
„
„
Remote sensing-derived thematic information are
becoming increasingly important. Unfortunately, they
contain errors.
Errors come from 5 sources:
„
„
„
„
„
„
„
Geometric error still there
None of atmospheric correction is perfect
Clusters incorrectly labeled after unsupervised classification
Training sites incorrectly labeled before supervised
classification
None of classification method is perfect
We should identify the sources of the error, minimize it,
do accuracy assessment, create metadata before being
used in scientific investigations and policy decisions.
We usually need GIS layers to assist our classification.
training vs. ground reference
„
Several ways to do error evaluation
„
Based on training pixels (areas)
„
„
„
Based on ground reference pixels
„
„
„
„
The problem is that the locations of training sites are usually not
random. They are biased by analyst’s a priori knowledge of
where certain LULC types exist in the scene.
This will results in higher classification accuracies than the one
below
These sites are not used to train the classification algorithm and
therefore represent unbiased reference information
It is possible to collect some ground sites prior to the
classification, perhaps at the same time as the training data
But majority of test reference is often collected after
classification.
Landscape often change rapidly. Therefore, it is best to
collect both the training and ground reference as close
to the data of remote sensing data acquisition as
possible. (for example, agriculture crops change fast)
Error (Confusion) Matrix
„
„
„
„
„
„
Producer (analyst) accuracy is a measure indicating the probability that
the classifier has labeled an image pixel into Class A given that the
ground truth is Class A. it is the probability of a reference pixel being
correctly classified.
Omission error represent pixels that belong to the ground truth class but
that the classification technique has failed to classify them into the
proper class.
User accuracy is a measure indicating the probability that a pixel is Class
A given that the classifier has labeled the pixel into Class A. it is the
probability that a pixel classified on the map actually represents that
category on the ground.
Commission error represent pixels that belong to another class but are
labeled as belonging to the class.
Overall accuracy is total classification accuracy.
Kappa coefficient (Khat) is a discrete multivariate technique of use in
accuracy assessment. Khat>80% represent strong agreement and good
accuracy. 40%-80% is middle, <40% is poor.
Example: they took 407 samples
(pixels) based on the stratified random
sampling after classification. First
made 5 files (each contain one class),
using a random number generator to
get points.
Post-classification and GIS
saltandpepper
types
„
„
„
„
„
„
Majority/Minority Analysis
Clump Classes
Morphology Filters
Sieve Classes
Combine Classes
Classification to vector (GIS)
Change detection
„
„
Change detect involves the use of multi-temporal datasets to
discriminate areas of land cover change between dates of imaging.
Ideally, it requires
„
„
„
Same or similar sensor, resolution, viewing geometry, spectral bands,
radiomatric resolution, acquisition time of data, and anniversary dates
Accurate spatial registration (less than 0.5 pixel error)
Methods
„
„
„
„
„
„
„
Independently classified and registered, then compare them
Classification of combined multi-temporal datasets,
Principal components analysis of combined multi-temporal datasets
Image differencing (subtracting), (needs to find change/no change threshold,
change area will be in the tails of the histogram distribution)
Image ratioing (dividing), (needs to find change/no change threshold,
change area will be in the tails of the histogram distribution)
Change vector analysis
Delta transformation
Example: stages of development
Sun
Sun City
City ––
Hilton
Hilton Head
Head
1994
1994
1996
1996
1974
1,040 urban
hectares
1994
3,263 urban
hectares
315%
increase