Joint Solution of Urban Structure Detection from Hyperion Hyperspectral data
Lin Cong, Brian Nutter, Daan Liang
Wind Science and Engineering
Department of Electrical and Computer Engineering
Department of Construction Engineering & Engineering Technology
Texas Tech University, Lubbock, TX, USA
e-mail: {lin.cong, brian.nutter, daan.liang}@ttu.edu
7000
INTRODUCTION
directional energy in Fourier domain
direction of maximum energy
Problem: Damage condition of residential areas are more
concerned than that of natural areas in post-hurricane damage
assessment. Recognition of residential and natural areas from
commonly used low-spatial-resolution hyperspectral images is
thus important.
(a)
directional energy in Fourier domain
6000
(b)
10
gray level j
20
30
5000
4000
3000
2000
40
1000
50
Solution: A spatial-feature extraction method based on
hierarchical Fourier transform – Co-occurrence matrix is
developed. Spatial and spectral features are then combined to a
joint feature vector. Best feature combinations are determined by
K-fold cross validation.
60
0
10
20
30
40
gray level i
50
60
0
20
40
60
80
100
angle (degree)
120
140
160
180
(c)
(d)
Figure 2. (a) A sample of residential region; (b) The Fourier transform of the residential region;
(c) Plot of the directional energy distribution in Fourier domain; (d) Co-occurrence matrix
calculated with the direction of the maximum energy and offset equal to one;
x 10
2.5
12
directional energy in Fourier domain
direction of the maximum energy
METHOD
(b)
(a)
Flow chart
directional energy in Fourier domain
2
10
20
gray level j
Hyperspectral
data
30
1.5
Spectral
correlation
Fourier
Transform
Co-occurrence
matrix
PCA
components
60
0
10
20
30
40
gray level i
50
60
20
40
60
80
100
angle (digree)
120
140
160
180
(c)
Figure 3. (a) Another sample of residential region; (b) The Fourier transform of the residential
region; (c) Plot of the directional energy distribution in Fourier domain; (d) Co-occurrence
matrix calculated with the direction of the maximum energy and offset equal to one;
2
x 10
11
directional energy in Fourier domain
direction of the maximu energy
1.8
Feature
selection
K-means
clustering
directional energy in Fourier domain
1.6
(b)
(a)
10
20
gray level j
Bayes
Classification
0
(d)
Texture
measures
30
50
Datasets
1.4
1.2
1
0.8
0.6
40
Error
1.88%
1.90%
1.97%
2.01%
2.02%
2.04%
2.05%
2.07%
2.08%
2.10%
2.10%
2.11%
2.13%
2.13%
2.13%
2.15%
2.15%
2.16%
2.16%
2.17%
Rank Combination
21
1111011000
22
1101100000
23
1010111001
24
1011001001
25
1011011001
26
1111101001
27
1011011000
28
1011010001
29
1111101000
30
1111110001
31
1001011001
32
1111110000
33
1111001000
34
1111100001
35
1111010000
36
1101101001
37
1111111010
38
1101010000
39
1111111011
40
1111100000
Error
2.17%
2.18%
2.19%
2.21%
2.21%
2.21%
2.23%
2.23%
2.24%
2.25%
2.26%
2.26%
2.27%
2.28%
2.28%
2.29%
2.30%
2.31%
2.31%
2.31%
Rank
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
Combination
0000100000
0010000110
0010000111
0000000111
0010100110
0000000110
0000100110
0010100101
0010000101
0000100101
0000000101
0011000010
0101000000
0001000010
0010000010
0000000010
0011000000
0001000000
0010000000
0100000000
Error
12.5%
12.7%
12.7%
12.7%
12.8%
12.8%
12.9%
13.0%
13.0%
13.1%
13.2%
13.2%
13.7%
14.7%
15.3%
15.8%
16.1%
19.4%
24.4%
27.2%
Spectral Solution
(avg. error: 17.39%)
Spatial Solution
(avg. error: 19.34%)
Joint Solution
(avg. error: 12.99%)
Rank Combination
1
1110001001
2
1110011001
3
1110111000
4
1110001000
5
1110010001
6
1110101000
7
1110011000
8
1110010000
9
1110110000
10
1110101001
11
1110110001
12
1110111001
13
1111011000
14
1111001000
15
1111001001
16
1111011001
17
1111010000
18
1111111000
19
1111010001
20
1110100001
Error
5.67%
5.77%
5.81%
5.84%
5.84%
5.86%
5.87%
5.89%
5.96%
6.01%
6.03%
6.06%
6.19%
6.29%
6.29%
6.34%
6.39%
6.42%
6.50%
6.57%
Rank Combination
21
1111111001
22
1111101000
23
1111110001
24
1111101001
25
1110000001
26
1111100001
27
1111110000
28
1111100000
29
1111000001
30
1110011010
31
1110101010
32
0110111001
33
1110110010
34
1110001100
35
1110111011
36
1110101011
37
1110111100
38
0110011001
39
1110111010
40
0110001001
60
20
30
40
gray level i
50
0.2
60
0
20
40
60
80
100
angle (degree)
120
140
160
Error
6.60%
6.63%
6.66%
6.68%
6.75%
6.80%
6.83%
6.93%
7.12%
7.36%
7.39%
7.43%
7.44%
7.50%
7.52%
7.55%
7.56%
7.58%
7.58%
7.59%
Rank
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
Combination
0000100010
1101000010
1000100010
0101000010
0001000010
1000000010
1000100000
1001000010
1011000000
0011000000
0100000000
0101000000
0000100000
1010000000
1100000000
0001000000
1101000000
0010000000
1001000000
1000000000
Error
18.4%
18.5%
18.7%
19.1%
21.0%
21.6%
21.7%
22.2%
24.0%
24.6%
26.4%
26.5%
26.6%
27.0%
27.0%
27.7%
28.1%
28.9%
31.8%
39.8%
Results
180
(c)
(d)
Classified as Natural or River
3106
25617
Error Rate
Classified as Residential
4.59%
Residential
62704
38.28%
Natural + River
16116
Classified as Natural or River
5011
25389
Error Rate
Classified as Residential
7.40%
Residential
65225
38.83%
Natural + River
11699
Classified as Natural or River
2490
29806
Error Rate
3.68%
28.19%
1. Training data of New Orleans dataset is used to train the Bayes
classifier, and then the Lubbock dataset is classified.
Figure 9. “Cross” classification
results of Lubbock dataset.
Blue: residential areas correctly
classified; Green: natural areas
correctly classified; Red:
residential areas misclassifed as
natural areas; Pink: natural
areas misclassifed as residential
areas.
(a) Purely spectral
(b) Purely spatial
Spectral Solution
(avg. error: 69.95%)
Spatial Solution
(avg. error: 12.87%)
Joint Solution
(avg. error: 20.25%)
Classified as Residential
Classified as Natural
Error Rate
Residential Region
40214
18451
31.35%
Residential Region
55192
3473
5.92%
Classified as Residential
Classified as Natural
Error Rate
Residential Region
56524
2141
3.65%
Classified as Residential
Classified as Natural
Error Rate
(5) Angular Second Moment (ASM)
ASM   Pi , j 2
2
i , j 1
i , j 1
(2) Dissimilarity (DIS)
(6) Maximum Probability (MAX)
N
DIS   Pi , j i  j
MAX  max( Pi , j )
i , j 1
(a)
(b)
(c)
(d)
(3) Homogeneity (HOM)
HOM 
(7) Entropy (ENT)
N
ENT    Pi , j log 2 Pi , j
Pi , j
N
 1  (i  j )
i , j 1
2
(a) Ground truth
i , j 1
(4) Similarity (SIM)
SIM 
(b) Purely spectral
(c) Purely spatial
(d) Joint solution
Figure 7. Results of Bayes classification for Lubbock dataset. (a) Manually made ground truth;
(b) – (d) Results by using purely spectral features, purely spatial features, joint features,
respectively. Blue: residential areas correctly classified; Green: natural areas correctly
classified; Red: residential areas misclassifed as natural areas; Pink: natural areas misclassifed
as residential areas.
Pi , j
N
 1 i  j
i , j 1
Table 3: Error rates of the Bayes classification for Lubbock dataset
Spectral Solution
(avg. error: 15.45%)
Spatial Solution
(avg. error: 13.43%)
CON
(e)
(f)
(g)
(h)
DIS
HOM
SIM
ASM
MAX
ENT
Figure 5. Texture measures of Lubbock dataset
Joint Solution
(avg. error: 10.84%)
Classified as Residential
Classified as Natural
Error Rate
Residential Region
50443
8222
14.20%
Residential Region
48319
10346
21.26%
Classified as Residential
Classified as Natural
Error Rate
Residential Region
51127
7538
12.85%
Classified as Residential
Classified as Natural
Error Rate
Natural Region
12281
61734
16.59%
Natural Region
7479
66536
10.10%
Natural Region
6848
67168
9.25%
Figure 1. (a) Original hyperspectral image taken over Lubbock, TX in 01/2003; (b) – (c) The
top two significant PCA bands of Lubbock dataset; (d) Spectral correlation against the spectrum
of construction asphalt; (e) Original hyperspectral image taken over New Orleans, LA in
04/2005; (f) – (g) The top two significant PCA bands of New Orleans dataset; (h) Spectral
correlation against the spectrum of construction asphalt;
Fourier transform – Co-occurrence matrix
• Residential areas display periodic street patterns while the natural
areas are universal.
• Fourier Transform is applied to detect the directions orthogonal to
the street patterns.
• Gray level co-occurrence matrix is calculated between neighboring
pixels with an offset of one in the direction orthogonal to the street
patterns.
CON
DIS
18.38%
Natural Region
24721
49294
33.40%
Figure 10. “Cross” classification
results of New Orleans dataset.
Blue: residential areas correctly
classified; Green: natural areas
correctly classified; Red:
residential areas misclassifed as
natural areas; Pink: natural areas
misclassifed as residential areas.
N
N
Natural Region
69053
4962
93.30%
Natural Region
13607
60408
2. Training data of Lubbock dataset is used to train the Bayes
classifier, and then the New Orleans dataset is classified.
Texture measures
(1) Contrast (CON)
(c) Joint solution
Table 5: Error rates of the “cross” classification for Lubbock dataset
Bayes Classification
Figure 4. (a) A sample of natural region; (b) The Fourier transform of the natural region; (c) Plot
of the directional energy distribution in Fourier domain; (d) Co-occurrence matrix calculated
with the direction of the maximum energy and offset equal to one;
CON   Pi , j (i  j )
Natural + River
15888
“Cross” Bayes Classification
0.4
10
Classified as Residential
Residential
64609
Table 2: Performance of a subset of all joint feature combinations for New Orleans dataset.
50
PCA transform
Rank Combination
1
1100111001
2
1111111001
3
1101011001
4
1101011000
5
1101001001
6
1101010001
7
1110111001
8
1101101000
9
1111001001
10
1111011001
11
1011100000
12
1101110000
13
1001100000
14
1100111000
15
1111111000
16
1101111000
17
1111010001
18
1110111000
19
1101111001
20
1101100001
Table 4: Error rates of the Bayes classification for New Orleans dataset
1
0.5
40
Table 1: Performance of a subset of all joint feature combinations for Lubbock dataset.
Features are listed in the combinations following the order: PCA1, PCA2, spectral correlation,
CON, DIS, HOM, SIM, ASM, MAX, ENT. A “1” means that the feature in the associated
position is selected in the combination, and a “0” means that associated feature is not selected.
HOM
SIM
ASM
MAX
(a) Purely spectral
(b) Purely spatial
(c) Joint solution
Table 6: Error rates of the “cross” classification for New Orleans dataset
Spectral Solution
(avg. error: 42.07%)
Spatial Solution
(avg. error: 18.20%)
Joint Solution
(avg. error: 18.20%)
Classified as Residential
Classified as Natural or River
Error Rate
Classified as Residential
Classified as Natural or River
Error Rate
Classified as Residential
Classified as Natural or River
Error Rate
Residential
61881
5834
8.62%
Residential
53840
13875
20.49%
Residential
53937
13778
20.35%
Natural + River
40110
1395
96.64%
Natural + River
5999
35506
14.45%
Natural + River
6079
35426
14.65%
Conclusion
1. Improved accuracy in Bayes classification between residential
and natural areas was achieved by using both spectral and
macroscopic spatial information.
2. The spatial features extracted by proposed Fourier transform –
Co-occurrence matrix method seem to be reliable in “cross”
classification, although the purely spectral information
between different datasets is so different that it fails the cross
classification.
ENT
Figure 6. Texture measures of New Orleans dataset
Future work
Feature Selection
K-fold cross validation is applied on the training dataset to
determine the best combinations of the spectral and spatial
features.
This material is based upon work supported by the National Science Foundation under Grant No. 0800487
(a) Ground truth
(b) Purely spectral
(c) Purely spatial
(d) Joint solution
Figure 8. Results of Bayes classification for New Orleans dataset. (a) Manually made ground
truth; (b) – (d) Results by using purely spectral features, purely spatial features, joint features,
respectively. Blue: residential areas correctly classified; Green: natural areas correctly
classified; Red: residential areas misclassifed as natural areas; Pink: natural areas misclassifed
as residential areas.
1. More testing and verification on additional datasets are needed
in the future.
2. The segmentations of residential and natural areas can be used
for model choice in spectral unmixing.
3. The spectral unmixing results at the same position before and
after a hurricane can be compared to assess the damage level.