Geologist’s Field Assistant: Improving Image Analysis Algorithms for Remote
Science Exploration
V.C. Gulick (1), S.D. Hart (1), S.T. Ishikawa (2)
(1) NASA Ames/ SETI Institute, MS 239-20, (2) NASA Ames/UCSD
whether they are plutonic or volcanic (PV) – each
representing distinct criteria under which the specimen will
be evaluated. The specimen is characterized based on the
results from a nearest neighbor analysis, i.e., it is placed in
the same class as the database entry that most closely
resembles it. Comparisons are made using the Earth
Mover's Distance (EMD) [1] as a similarity metric.
The “leave-one-out” procedure was introduced as a way
to evaluate GFA's own precision at identifying rocks. For
this procedure, each entry in the database is picked
individually and isolated from the rest, which become the
training set.
This way, the isolated rock can be
“identified” using a larger training set. After all the
samples are processed, the percent correctness is computed
for each category and any possible areas of improvement
are assessed.
Abstract
There are inherent limitations to our endeavors of
human planetary exploration.
Among the most
protuberant is the lack of mobility and sensory input
available to suited astronauts, which is exacerbated by the
similarities in appearance of minerals that must be
distinguished in order to choose appropriate rock
samples.
These issues underscore the need for
autonomous alternatives and introduce the task of
designing future exploration rovers that are capable of
unassisted rock identification. These problems are being
addressed by using pattern recognition as a means for
interpreting imaging data. Consequently, it is in our
interest to create an accurate and robust system that
minimizes human dependency and, instead, autonomously
provides high level scientific advice to astronauts or
robotic explorers.
1.2. The Image Analysis Module
Analyses of the imaging data is performed on the basis
of visual attributes, namely texture [2] and color [3]. The
results are used as statistical data that will be interpreted
by a reasoning layer. As a result, it is important to
evaluate the pattern recognition algorithms and make
further improvements that will increase the reliability of
GFA.
Texture – Texture features are obtained from an image
by using Gabor filters. These filters are derived from
Gabor wavelets through dilations and rotations, and
comprise a dictionary of filters. The idea is that each filter
breaks down the image into different features based on
varying frequencies. When a sample is being identified,
feature vectors are extracted in this same manner and
compared one-by-one to the entries in the database. The
nearest neighbor analysis assigns an EMD value to each
pair of features, the smaller number representing a closer
match. The sample is then reported to be in the same class
as its closest match, e.g. for the PV test, a sample whose
closest match is a plutonic sample is labeled “P” for
plutonic.
Color – Color features are extracted in a similar fashion
by applying color segmentation using the mean shift
algorithm [3] to create a feature vector. Feature vectors of
1. Overview of the Existing System
The Geologist’s Field Assistant (GFA) was designed to
aid Geologists in identifying rock specimen by using raw
imaging data from a high-resolution camera and spectral
data from a vis/near-IR and Raman spectrometer. The
data is processed by three sequential layers that calibrate
the system, recognize patterns, and present the beliefs.
The pattern recognition layer is divided into the image
analysis and spectral analysis modules. These provide
“evidence” to the reasoning layer that employs a Bayesian
network as a probabilistic reasoning device that delivers
statistical results based on the input. The result is a report
of system beliefs about the specimen.
1.1. Pattern Recognition
A specimen is compared against a pre-compiled image
database containing features, or signatures, of previously
classified ground truth samples.
The samples are
classified and grouped into signature databases according
to their main texture (MT), general composition (GC), and
1
unknown samples are once again compared to a color
signature database using the nearest neighbor method and
the closest matches are determined.
2.1. Discussion
The general trend seems to indicate that the accuracy
improves as the database grows in number. This implies
that there are unique traits specific to each class that are
being recognized in individual rocks. Out of 173 samples
labeled by geologists as ground truth phaneritic, 83% of
the samples were correctly identified using the training sets
(Table 2). Aphanitic samples, on the other hand, were
almost half in number and had a much lower correctness
rate (38%). This suggests that simply collecting more rock
specimen will make GFA more accurate. Note, too, that
the probability of incorrectly classifying aphanitic samples
as phaneritic (and vice versa) is significantly low. This is
because the porphyritic samples act as a buffer, since they
contain the characteristics of both aphanitic and phaneritic
samples.
While determining the best values for these parameters,
it becomes increasingly important to have a way of
visualizing these filers and their effects on the images they
are being applied to. This will provide a more qualitative
understanding as to what the filters are doing and whether
or not they are effectively extracting unique features. We
can then fine-tune these parameters accordingly in an
attempt to calibrate the code and optimize our results.
2. Results
Correctness results for texture analysis have varied by
adjusting the number of scales and orientations in the
filters, as well as reducing the image resolution. This
suggests that there is a need to optimize these parameters
in order to produce the best possible results.
Texture results were evaluated using the leave-one-out
method described in Section 1.1. A database of roughly
400 2048 x 1536 pixel rock images was used with a
reduction factor of 4 on each image. Results for PV test,
based on percent correctness, are displayed in Table 1.
Correct
Total
% Correctness
Plutonic
153
179
85.4749
Volcanic
175
205
85.3659
Table 1. Texture results using Gabor filters at 3
scales/6 orientations
GFA was able to distinguish between plutonic
(generally fine-grained) and volcanic (generally coarsegrained) about 85% of the time using Gabor filters set to 3
scales and 6 orientations (Table 1). Tests were also
conducted on main texture (MT), which is categorized into
three main types: aphanitic (fine-grained), porphyritic
(fine-grained with scattered crystals), and phaneritic
(coarse-grained). These results, using Gabor filters set at 5
scales and 10 orientations, are displayed in Table 2. The
shaded boxes indicate the number of samples that were
correctly identified under their respective category, i.e., a
total of 95 aphanitic samples were tested using the leaveone-out method, 36 of which were correctly identified.
The remainder were incorrectly identified as porphyritic
(46) and phaneritic (13).
Aphanitic
Porphyritic
Phaneritic
Aphanitic
36
37
8
Porphyritic
46
55
21
Phaneritic
13
26
144
Total
95
118
173
3. Improvement of GFA
GFA is based on a probabilistic reasoning system in
which evidence is presented and processed according to
geological rules that have been coded into the system. As
a result, every piece of evidence – be it the presence of the
mineral pyroxene or large pink-colored crystals commonly
found in granite – greatly helps to narrow down the
possibilities. This calls for the implementation of more
functions that enhance GFA’s ability to detect these
singularities.
One function that has yet to be implemented is the
detection of individual crystals within the rock images.
This can be accomplished by designing specific texons that
represent features such as porphyries, vesicles (cavities)
and crystals – or any other particular arrangement in rocks
that are unique to a specific rock type – and specifically
look for them in an image. Ideally, the system would be
competent enough to recognize these singularities.
The introduction of a lightness scale should also be
considered since the general composition of a specimen is
highly correlated with how light/dark it is. This may be
able to push some hard classifications into the correct
direction. Since mafics are generally darker than felsics, a
sample that was falsely identified as mafic could have a
second chance if it had a higher lightness rating taken into
account (presumably because lighter rocks tend to be felsic
in composition). This could potentially be the deciding
factor for cases of multiple close-match queries; however,
% Correct
37.8947
46.6102
83.237
Table 2 - Texture results using Gabor filters at 5
scales/10 orientations
2
before this is implemented, experiments should be
conducted to see how given rock attributes are related to
their lightness.
Another insight is to introduce a way of weighting highresolution pattern recognition results with lowerresolutions ones and have a decision tree structure that
decides on what signature databases to use based on types
of rocks. Experiments have suggested that the color
analysis results using mean shift clustering tend to be more
accurate at lower resolutions. The opposite seems to true
for Gabor wavelet analysis of texture.
References
[1] Y. Rubner, C. Tomasi, and L.J Guibas. “The Earth Mover’s
Distance as a Metric for Image Retrieval,” IEEE Conf. on
Computer Vision. Bombay, India. 1998.
[2] B.S. Manjunath and W.Y. Ma. “Texture Features for
Browsing and Retrieval of Image Data,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, Vol. 18, No.
8:837–842, 1996.
[3] D. Comaniciu, P. Meer. “Robust Analysis of Feature
Spaces: Color Image Segmentation,” cvpr, p.750, 1997
IEEE Computer Society Conference on Computer Vision
and Pattern Recognition (CVPR’97), 1997.
3
List of Qualifications
Milestones and Deadlines
Dates: June 2003 – August 2003
Location: NASA Ames Research Center
Job Description: Programmer & Field Assistant
Duties/Responsibilities:
 Help create a graphical display of telemetry data
for “K-9” mars rover prototype.
 Assist in field tests conducted at the “Marscape.”
Week 1-3: Begin designing a system to (visually) identify
specific minerals in sample images. Develop a parameter
field showing correctness rates as a function of the 1)
number of scales, 2) orientations, and 3) resolution
reduction factor. Note: Feature extraction takes up to
several days since the database contains hundreds of highresolution images.
Week 3-5: Implementation of mineral detector function.
Week 5-6: Show preliminary results (using only a few
training samples) to see how correctness rate is affected.
Week 6-8: Design and implement lightness scale.
Week 9-10: Report results using full database. Begin
writing report (draft).
March 6: Experiments and final report complete.
Dates: June 2004 – December 2004
Location: NASA Ames Research Center
Job Description: Programmer
Duties/Responsibilities:
 Update the Geologist’s Field Assistant (GFA)
system that identifies unknown rock samples via
color and texture analysis.
 Work with geologists to organize and benchmark
various implementations of algorithms and report
the results.
Dates: July 2005 – Present
Location: NASA Ames Research Center
Job Description: Programmer
Duties/Responsibilities:
1. Fix memory issues associated with highresolution texture analysis.
2. Perform test runs of “unknown” rock samples and
evaluate correctness.
3. Assess strengths and weaknesses of current
algorithms.
4. Use compression algorithms and demonstrate
their efficiency on images obtained by the 2005
Mars Exploration Rovers (MER) mission.
4