SCIENCE Vol21 (2) 1997, pp. 219-246
ISSN 0364-0213
Copyright 0 1997 Cognitive Science Society, Inc.
Allrights of reproduction in any form reserved.
COGNITIVE
The Texture
Understanding
Lexicon:
the Categorization
of Visual Texture Terms and Their
Relationship
to Texture
Images
NALINIBHUSHAN
Smith College
A. RAVISHANKARRAO
IBM Watson Research Center
GERALD L. LOHSE
TheWharton School, University of Pennsylvania
In this paper we present the results of two experiments. The first is on the categorization of texture words in the English language. The goal was to determine whether there is a common basis for subjects’ groupings of words related
to visual texture, and if so, to identify the underlying dimensions used to categorize those words.
Eleven major clusters were identified through hierarchical cluster analysis,
ranging from ‘random’ to ‘repetitive’. These clusters remained intact in a
multidimensional scaling solution. The stress for a three-dimensional
solution obtained through multidimensional scaling was 0.18, meaning that 82%
of the variance in the data is explained through the use of three dimensions.
It appears that the major dimensions of texture descriptors are repetitive
versus nonrepetitive; linearly oriented versus circularly oriented; and simple
versus complex.
In the second experiment we measured the strength of association between
texture words and texture images. The goal was to determine whether there is
any systematic correspondence between the domains of texture words and
texture images. Pearson’s coefficient of contingency, a measure of the strength
of association, was found to be 0.63 for words corresponding to given images
and 0.56 for images corresponding to given words. Thus the texture categories
in the verbal space and those in the visual space are strongly tied.
In sum, our two experiments show (a) that despite the tremendous variety
in the words we have to describe textures, there is an underlying structure to
the lexical space which can be derived from the experimental data; and (b)
Direct all correspondence to: Nalini Bhushan, Dept. of Philosophy,
mail: nbhushan@emestine.smith.edu.
Smith College, Northampton,
MA 01063; e-
219
220
BHUSHAN,RAO, AND LOHSE
that the association. between a category of words and a category of images
was strongest when both categories represent the same underlying property.
This suggests that subjects’ organizations of texture terms are systematically
tied to their organization of texture images.
INTRODUCTION AND BACKGROUND
If language is intimately tied to cognition, then the words that we use to talk about what we
see may also reveal to us how we internally configure what we see. In particular, the way
in which human beings categorize the words and images of a domain such as texture may
provide insight into how they internally structure the concepts of that domain.
Philosophers since the time of Plato have been interested in properties of objects and the
manner in which they intrude upon human consciousness. However, the property of texture
has historically not elicited much interest. In the case of the ancients, it had partly to do
with their preoccupation with essences. In the case of the modems since (Locke, 1973, this
neglect was due to a similar preoccupation, this time in the form of a concern with properties that were either ‘primary’ (intrinsic to the object) such as shape, size, and density; or
‘secondary’ (extrinsic or relational) such as color, sound, and feel. Significantly, these
were both simple properties (not analyzable into others more basic). Texture, a property of
an object (or surface) that one might think of quite naturally as “composed” of simpler
properties that make up the object configuration, such as shape, size, and density of the
basic elements, was of lesser interest. Oddly enough, when it has merited mention, though,
texture has been variously categorized as an instance of Locke’s primary (Bennett, 1971)
or of Locke’s secondary (Wright, 1988) properties. Although Locke is partially responsible
for this confusion, it is clear that putting texture into either category would constitute a violation of his own constraints on both primariness and secondariness-that they be simple
properties. And in fact, in some of his reflections he seems to have acknowledged this and
held instead that texture occupies a unique place wedged somehow between the primary
and the secondary categories. Thus while other complex properties such as being malleable
or magnetic are simply altentate ways that the primary properties such as shape, weight,
density and so on come together in a particular object, texture, in contrast, “is that emergent
spatial configuration that results when two or more atoms are conjoined to form a body”
(Smith, 1990). This complexity has historically made texture ontologically less interesting
to philosophers and empirically less understood by researchers. We (Bhushan & Rao,
1995) have recently discussed the place of texture in the context of the distinction, where
we argue against the plausibility of finding this unique space among the simples for texture
between Lockean primary and secondary properties, and make a case for viewing texture
as a complex secondary property, and in this way as distinct from color, which has historically been taken to be a simple secondary property.’
In contrast to philosophers, cognitive scientists have recognized texture as a visual cue
that plays a significant role in a variety of cognitive tasks. A common use is in describing
and differentiating different kinds of objects, for example, wallpaper, furniture, carpets,
sand, and grass. A working definition of texture in this context is the surface markings on
THETEXTURE
LEXICON
221
an object or the 2-D appearance of a surface. Texture has been studied extensively by
researchers in human perception, computer vision, and material science (Julesz, 1981; Rao,
1990; Voort, 1986). Among the achievements of this research are the identification of low
level features used in human texture perception and the development of algorithms for
computerized texture measurement and analysis. However, even in empirical research,
explicit attempts to categorize and taxonomize different kinds of texture have been quite
rare. More recently, although studies have been performed to understand categorization of
texture images (Rao & Lohse, 1993), few studies have tied to understand the categorization of texture words or the correspondence between texture words and texture images. In
contrast to the work that has been done on color, the state of understanding of surface properties such as texture still appears to be chaotic. In that earlier study, Rao and Lohse performed a set of experiments to identify higher level features used in the perception of
texture images. It was that study that provided the impetus for the two further studies
undertaken in this paper. There they identified the three orthogonal dimensions of visual
texture to be repetitive vs. nonrepetitive; high-contrast and nondirectional vs. low-contrast
and directional; granular, coarse and low-complexity vs. nongranular, tine and high-complexity. Since the earlier study was done with texture images, the first study of the current
paper shifts the focus to language and deals with texture words. The natural next step is
then taken in our second study in this paper where we perform a composite experiment that
examines the two-way mapping of the visual texture space onto the lexical texture space.
Ideally, one would like to understand the domain of texture to the level of maturity
achieved in the case of color, where for instance one knows that color is three dimensional,
characterized by a triplet such as brightness, hue and saturation components, or RedGreen-Blue components; and where there are systematic procedures for naming color
(Levkowitz & Herman, 1993).
To provide a more general justification for the research direction pursued in this paper,
we motivate it from three different scenarios, all of which arise from the increasing usage
of images on platforms such as personal computers. The first scenario examines the issue
of improved human interfaces for computer graphics systems. The second scenario
addresses the question of searching image databases, and the last scenario looks at an
industrial inspection task. Since the scope of each of these scenarios is large, we restrict
ourselves to examining each of them in the context of surface texture.
The goal of computer graphics is the realistic generation of visual images corresponding
to physical objects. Texture is an important property of objects that must be exploited to
generate photorealistic images. In the computer graphics world, a controlled generation of
textures requires a detailed understanding of mathematical models. The end users of texture, however, are typically artists and designers who do not possess such knowledge of
mathematics. In an attempt to overcome this difficulty, Englert and Schendel (1992)
designed a hierarchical language for the description of textures. This language uses a syntax
that is similar to the programming language PASCAL and allows the user to specify texture
elements and their placement. Alternately, fractal models are used to generate random textures. Such a formal specification of textures is a right step towards the building of better
interfaces. However, it still involves a fair amount of programming. The simplicity and ease
222
BHUSHAN, RAO, AND LOHSE
of use of natural language interfaces has not been tapped. It would be desirable to have an
option whereby a user could simply specify terms such as rough or very rough and have the
system automatically tune the required parameters. Such an operation is possible through
the use of fuzzy logic and has already been demonstrated in the case of color by Farhoosh
and Schrack (1986). Later in this section, we discuss the issue of color in more detail.
The next scenario we imagine is that of image databases. The availability of commercial
systems like the Kodak Photo-CD have made image databases widespread. Instead of having files with text, the user has access to files containing images. An important issue is how
the user can specify the content of these images for the tasks of indexing and retrieval.
Thus, the user might want to search for all images that contain a boy wearing a striped shirt,
or standing on a chequered floor, or walking down a rough, bumpy road. Note that surface
texture terms (in italics) naturally enter into a query such as this. Of course, other visual
cues also need to be used such as shape, color, and position. One of the tasks for the designers of such database systems is to understand how users conceptualize these different
visual cues. An identification of the categories used by humans for each visual cue will
allow the system to treat members within a category in like fashion, for example, treat harmonious and repetitive surfaces alike. Note that such categorization is not possible by simple lexical analysis without understanding the structure of the perceptual space.
(Techniques such as latent semantic indexing [Dumais et al., 19881 can be used to circumvent some of the problems in performing database searches; for example, the use of synonymous words should lead to the same result. However, these techniques have been
developed only for textual databases. The issues arising in image databases are quite different due to the nature of the data: text vs. images. A detailed exploration of such issues is
outside the scope of this paper.)
The final scenario we deal with is the specification and description of surface properties
in industrial applications like semiconductor manufacturing. Rao (1990) reported that a
great deal of jargon is used in the manufacturing industry. Terms such as worm-hole, hillocks, grass and speedboat are used to describe surface properties. The occurrence of such
surfaces in the product signifies a defect, which is due to malfunctions in the manufacturing process. It is desirable to standardize such nomenclature as it streamlines the cataloging
of defects and also allows engineers to develop the right algorithms to measure these properties. For such standardization to occur, it is necessary to understand the perceptual categories in surface texture.
Our examination of the above three scenarios demonstrates the importance of understanding the categorization of texture terms.
We now review different attempts to provide natural language interfaces for the description of visual phenomena. The representative visual cues that we discuss are color, motion,
and texture. Berlin and Kay (1969) present an in-depth study of color terms across ninetyeight languages. Their study revealed that there are exactly eleven basic color categories
from which the basic color terms of any given language are drawn. These eleven basic
color categories are white, black, red, green, yellow, blue, brown, purple, pink, orange and
grey. Languages vary in the number of the above eleven categories they contain, starting
from a minimum of two, as all languages contain terms for white and black.
THE TEXTURE
LEXICON
223
The concept of a “basic” color term, suggested by Berlin and Kay, is based on the following rules: (a) its meaning is not predictable from the meanings of its parts; (b) its significance is not included in that of any other color term; (c) its application must not be
restricted to a narrow class of objects; (d) it must be psychologically salient for informants (p. 6).
The notion of terms that are semantically as well as psychologically basic implies that
there might be some color terms that are at the root of all other color terms, to which all
other terms might be semantically reduced, and in terms of which all colors might be psychologically understood. Thus these would be necessary and sufficient to account for every
color term and concept there is. Tbe very notion of ‘basic’ as well as the Berlin and Kay
method for deciding which colors count as basic has recently been severely criticized
(Brakel, 1993). Nonetheless, it seems to us that the strategy of searching for the lowest
common denominator of subject groupings of texture terms can yield useful insights about
the class of texture words. Based on the results of the first of two experiments described in
this paper, we will attempt to define texture categories in the English language. According
to Jackendoff (1983) “fuzziness is an inescapable characteristic of the concepts that language expresses. To attempt to define it out of semantics is only evasion” (p. 117). This
recommends a multivariate analysis of people’s texture judgments that reveals categories
as they are spread along a dimension and have relationships with categories spread along
others, understood in terms of proximity relations between terms, rather than a resolution
in terms of necessary and sufficient conditions requisite for terms to fall into (or be
excluded from) a category.
Maerz and Paul (1950) compiled a dictionary of color terms, which contains over 2000
color terms, and constitutes the most extensive range of colors published. The dictionary
contains color terms along with matching sample color plates. The observation that motivated their study, back in the 194Os,was that “. . . while standardization has been arrived at
in practically all other fields, in the use of color names for identifying color sensations a
condition prevails that is usually characterized as chaotic; a state of affairs conducive not
only of misunderstanding, but even, on occasion, of financial loss.” Thanks to their efforts,
and those of other researchers such as Berlin and Kay (1969) and Berk et al. (1982) we
have a mature understanding of color, which is standardized.
The Color Naming System (CNS) (Berk et al., 1982) standardizes the specification of
color by using simple, easily understood primitives from the English language. The CNS
quantizes the three dimensions of color, viz. hue, lightness and saturation, into three discrete sets of symbols. Thus, lightness has the five discrete values of very dark, dark,
medium, light and very light; saturation has four values: grayish, moderate, strong and
vivid; hue can take the values black, very dark gray, dark gray, gray, light gray, very
light gray, white, blue, purple, red, orange, brown, yellow and green. Further moditications of hue names are possible through the use of the suffix “ish,” for example, greenish
blue. A grammar specifies how these words may be combined. Examples include colors
such as light greenish blue, medium strong red etc. Farhoosh and S&rack (1986) present
a methodology to convert these symbolic color names into quantitative color values, and
vice versa.
224
BHUSHAN, RAO, AND LOHSE
We should point out that even in color research, practically all of the categorization
work on color, as far as we can tell, has been done on color chips. It would be interesting
to conduct a similar research on color words. Since color words are somewhat more familiar for subjects and tend to wear their affiliation on their sleeve (e.g., lemon green, fir
green), we would expect subjects to categorize color words more neatly than they would
texture words. It would be worthwhile to see a study that demonstrated this empirically.
Becket and Badler (1990) have implemented a computer graphics system which can
generate imperfect textures such as stains, scratches, smudges, mould and rust. They concentrate on such imperfections to give otherwise artificial looking images a realistic flavor.
The part of their system which is relevant to the subject of this paper is the natural language
interface to allow specification of the imperfection. For instance, the system can recognize
a description such as “make a very shiny copper cube which is somewhat scratched near
the edges and somewhat blotched. ” The system has models for the generation of these
imperfections and uses the adverbs such as slight, somewhat, extremely to adjust the severity of the imperfection. The Texture Naming System that we are aiming to devise upon
completion of these experiments shares the spirit of this approach, though it is aimed at
capturing a larger class of textures than just imperfections.*
The specification of the visual cue of motion is important in tasks such as movie animation. Green and Sun (1988) have developed a programming system to facilitate animation
through a high level specification of motion. The motion of an object is specified in terms
of a motion verb acting on the object-thus, a tree could blow in the wind, and people
could walk. These verbs are implemented as rule-based primitives acting on objects. The
primitives are parameterized to include direction and extent of motion. Interestingly, their
study deals with verbs relating to motion, whereas ours deals with adjectives relating to
surface properties.
Finally, a word about texture words in special disciplines. In the discipline of petrology,
the term ‘texture’ refers to the geometrical relationships among the component crystals of
a rock and any amorphous materials that may be present (MacKenzie, 1982). Such textural
information is useful in studying the formation and evolution of rocks. As an example of
geometrical relationships, consider the case of inequigranular textures, where crystals of
different sizes are present. Inequigranular textures consist of textures such as seriate texture, where crystals show a continuous range of sizes, and poikilitic texture, where large
crystals of one mineral enclose numerous smaller crystals of another mineral. In our study
we have drawn upon a general purpose dictionary in collecting texture terms, since our
goal is to see whether there is any structure to the larger and more frequently used body of
texture words we have in English. Consequently, these more discipline-relative and discipline-defined texture words do not figure in our study.
It is important to note that texture is a property that can be analyzed either visually or
through touch. Though this paper is restricted to the visual aspect of texture, the tactile
aspect has been the subject of some interesting experiments. Yoshida (1968) used multidimensional scaling approach to identify the principal dimensions of touch. Subjects were
asked to touch a set of fifty sample surfaces such as felt, cloth, silk, steel and rubber, and
rate these samples on a set of twenty scales such as soft-hard, rough-smooth and wet-dry.
THETEXTURE
LEXICON
225
This study found the main dimension to be metallicness versus fibrelikeness. The physical
dimensions that differentiate these two extremes are specific gravity, thermal conductivity,
plasticity and hardness.
Ohno (1980) investigated the visual perception of surface roughness of building materials. Minsky et al. (1990) has developed a system to communicate surface texture through
force displays. For a nice overview of the psychological literature on the perception of texture by touch, see Lederman (1982).
GOALS
In this paper we first analyze words from the English language that deal with texture, for
example, coarse, tine, grainy, mottled, repetitive, rough, woven, and wrinkled. The goal is
to determine whether people’s actual categorizations of texture words reveal an underlying
common structure. The impetus for the study in this paper was provided by our earlier
work in the categorization of texture images, cited above, where it was found that there was
a strong agreement between the categorizations offered by different subjects. This led to
the identification of three major dimensions along which texture could be specified. It was
felt that a similar experiment in the verbal domain would shed further light on the more
general topic of human perception of textural properties. A second experiment analyzes
subjects’ two-way mapping data between texture words and texture images to measure the
strength of association between them.
METHODS
In this section we address three important issues pertaining to classification tasks, namely,
selection of the set of items, the set of subjects and the analysis technique. First, the sample
of texture words must be representative of words dealing with surface properties of objects.
Second, the sample of human subjects must be a representative sample. Finally, the analysis technique used to create the classification is important, as different techniques can give
rise to different classifications for the same data set.
The selection of words was done by searching through an on-line dictionary as
described below. We chose a reasonable variability in the sample of subjects used in the
study. We used a sorting procedure to collect one type of similarity data and two methods
of analysis, multidimensional scaling and hierarchical clustering, to analyze these data. We
believe the representativeness of the words, subjects and analytical techniques provides a
sound basis for categorizing textural words people use to visualize surface properties.
Items
We used a three step procedure to attain the final list of words used in the categorization
task: an on-line dictionary search; an elimination of words based on their frequency of
occurrence; a further elimination based on principal components analysis of data gathered
from a pilot study. These steps are described in further detail below.
226
BHUSHAN,RAO, AND LOHSE
On-Line Dictionary Searching
Since there is no standardized lexicon of words related to surface texture, our first task was
to create a comprehensive list of such words. We used the on-line Websters Dictionary,
available on the VM system at the IBM T.J. Watson Research Center. We used a boot-strap
technique and used the result of successive searches to carry out larger searches. We started
with an initial list of known texture words culled from previous sources (see Rao & Lohse,
1993b), the authors’ knowledge, and a quick manual search of the dictionary for texture
terms. These texture words were collected more or less intuitively, constrained by looking
in the dictionary for words that had the terms ‘texture’ or ‘pattern’ in their definition. For
example, net = a fabric made from string, cord etc. loosely knotted or woven in an openwork pattern (Websters, 2nd ed.). This list, shown in Table 1 was used to query the on-line
dictionary to find all synonyms. A typical item in the output was as follows (these are synonyms, respectively, of the words bubbly and bumpy):
ADJECTIVE
1. carbonated,curly, effervescent, fizzy, foamy, frothy, lathery, sparkling,sudsy
2. animated, bouncy, elated, excited, happy, lively, merry
ADJECTIVE
1. bone-breaking, bouncy, choppy, irregular, jarring, jerky, jolting, jolty, knobby,
lumpy, pitted, potholed, rough, rutted, uneven
Note that the response to the query contains words like excited and happy, which do not
relate to surface texture, and thus have to be filtered out. After such filtering, we obtained
a list, shown in Table 2, which was used to generate the final query. This gave rise to the
list in Table 3, containing 367 words related to surface texture. This list is quite representative of texture words, though it is not complete.
Paring the List
We used three passes to pare the list in Table 3 down to a reasonable size that could be used
in a sorting task. In the first pass, we eliminated categories of words dealing with properties
that were not strictly related to surface texture. Thus, words dealing with the interaction of
light with the object, such as translucent, transparent, and opaque were eliminated.
In the second pass, we used the dictionary of word frequency (Francis & Kucera, 1982)
to eliminate words used infrequently in the English language. This resulted in a list of 141
words, which was then administered in a pilot study to a set of 20 subjects at Smith College, using the procedure described in the next section. This list is shown in Table 4.
In the third pass, the results from the above pilot study were used to perform a principal
components analysis. The first 7 principal components explained 70% of the variance in
the data. The next 10 components explained an additional 10% of the variance, and the next
27 components explained a further 10% of the variance. We used the scores of the 141
words on the first 7 principal components to select a subset. Words that had a high score on
at least one of these principal components were retained and others rejected. The reasoning
behind this is that words that do not rank high in their principal components do not possess
sufficient discriminating power.
227
TABLE 1
The Initial Ust of Texture Words
bubbly,
bumpy, circular,
facetted,
feathery,
granulated,
oriented,
roving,
fine, fluffy,
honeycombed,
patterned,
ruffled,
spotted,
coarse, corrugated,
foamy,
streaked,
pearlescent,
striated,
wavy, weave, woven,
frosted,
iridescence,
scaly, scratchy,
creased, crimped,
perforated,
serrated,
stringy,
fuzzy,
matt, motte,
pleated puckered,
swirly,
regular,
slubbed,
tarnished,
wiry, wolly, wrinkled,
criss-crossed,
gauzy, glabrous,
lacy, lumpy,
sleed, slimy,
striped,
web, whirly,
frothy,
knobbly,
crinkled,
smooth,
turbid,
glossy,
grainy,
metallic,
mottled,
repetitive,
speckled,
waffled,
dotted,
rippled,
net,
rough,
spiny, spongy,
waffle-weave,
waved,
zigzog
The List of Texture Words After the First On-line Dictionary Search
banded,
barred,
chequered,
crumbly,
dotted,
blend, blotchy,
elastic,
curdled,
entwine,
freckled,
grooved,
perforated,
netlike,
rumpled,
sheeny,
spiders-web,
stroked,
variegated,
brindled,
sheer,
corizontal,
network,
veined,
dim, dishevelled,
filigree,
furrowed,
notched,
shiny, siklen,
indefinite,
unclear,
viscous,
sinewy,
toothed,
uneven,
slash,
sprinkled,
glassy,
distorted,
tousled,
translucent,
uniform,
well-ordered,
transparent,
winding,
rhythmic,
serrulated,
stony, streaked,
uninterrupted,
mesh,
muddy,
overlapping,
recurrent,
serrated,
foggy,
gronular
intertwined,
matting,
specked, speckeled,
stippled,
fluted,
gossamer,
interlaced,
marbled,
potholed,
scalloped,
smear,
glozed,
indistinct,
unfocused,
web, webbing,
crows-foot,
disordered,
open, out-of-focus,
polka-dot,
sawtoothed,
spotty, springy,
bushy, channelled,
flat, flecked, fleecy, flowing,
gathered,
oily, opaque,
polished,
sawlike,
broken,
cobweb, craffy, crisscross,
dense,
ill-defined,
plaited,
rutted,
tabby, tangled,
unchanging,
feathery,
frizzy,
bristly,
lacelike, lattice, layer, level, lined, lustrous,
spin, spotted,
studded,
unbroken,
frilly,
pied, pitted, plain,
rocky, rugged,
shaggy,
even, faint,
hazy, honeycomb,
napped,
curly, dappled,
frictionless,
jagged, kinky, knitted,
murky,
braided,
choppy, clotted, cloudy, coarse-grained,
crystal-like,
foleded,
blurred,
ridged,
shadowy,
speckledy,
striate,
twist,
stripe,
stripy,
twisted,
unvoried,
unwrinkled,
zigzag
The result of this phase was the generation of the list finally used in the experiment,
depicted in Table K3
Subjects
The experiment was administered to a total of forty subjects. Of these, twenty subjects
were students at Smith College, and twenty subjects were staff members at the IBM T.J.
Watson Research Center. The backgrounds of the subjects varied from science and engineering to iiberal arts. Ages ranged from 20 to 61.
Procedure
Each of the 98 texture words was printed on a 3” x 5" card. The order of the 98 items was
randomized for each subject. The subjects were asked to perform a free-form sorting task,
BHUSHAN, RAO, AND LOHSE
TABLE 3
The Final List After Dictionary Searching,
Containing 367 Words Related to Surface Texture
asymmetrical,
blemished,
bubbly,
banded,
barred,
blend, botchy, blurred,
buckle, bumpy,
close-knit,
congealed,
crepuscular,
crumpled,
crimped,
crystalline,
denticulate,
burnished,
crinkled,
criss-crossed,
curdled,
drilled,
faded, faint, fair,
feathery,
foggy, folded,
frosted,
frothy,
furbished,
gleaming,
glistening,
grooved,
indented,
glossy,
hairy,
jagged, iell, iumbled,
mesh, messy,
network,
lined, luminous,
metallic,
piebald,
repetitive,
reticulate,
rugged,
scarred,
powdery,
scattered,
serrated,
simple,
serrulated,
sinewy,
spiders-web,
stainless,
stippled,
tousled,
unblemished,
unwrinkled,
zigzag,
whirly,
winding,
splotch,
sheeny,
unclear,
striated,
trellis,
trough,
undefiled,
veined, viscous,
wiry, wizened,
woolly,
repeated,
roving,
ruched,
scalloped,
scaly,
shiny,
springy,
striped,
shirr,
stoked,
warped,
stained,
studded,
toothed,
turning,
undulation,
unvarnished,
silken,
soft, sparkling,
sprinkled,
tight-curled,
worn,
serpentine,
snorled,
tucked, turbid,
waffled,
periodic,
polka-dot,
regular,
shimmering,
smudged,
meandering,
netlike,
out-of-focus,
semitransparent,
undimmed,
untarnished,
lattice, layered,
polished,
rocky, rough,
stringy,
irregular,
perforated,
sawtoothed,
threaded,
indefinite,
nebulous,
recurrent,
spot, spotted,
tenebrous,
unsymmetrical,
wirework,
sheer,
smooth,
spongy,
streaked,
tatting,
varnished,
rippled,
glazed,
grid, grille,
matte, matted,
oriented,
fluted,
frizzy,
glassy,
gravelly,
napped,
peppered,
random,
fluffy,
frilly,
lacquer,
pockmarked,
seethrough,
smeared,
transpicuous,
unruly,
morked,
dented,
even, facetted,
ill-defined,
lacework,
rutty, sandy, satiny,
shaggy,
stratum,
unchanging,
vague, variegated,
zoned
ridged,
tarnished,
transparent,
unpatterned,
well-ordered,
riddled,
slubbed,
dense,
iridescent,
open, openwork,
pleated,
scrunched,
grating,
horizontal,
murky,
pebbly, pellucid,
plaited,
frictionless,
crumbly,
distorted,
entwine,
intertwined,
marbled,
ragged,
shadowy,
storey,
granular,
muddy,
radiant,
spiny, spiralled,
unbroken,
uninterrupted,
lustrous,
ruptured,
sleed, slimy,
stony,
fretted,
lacelike,
oily, opaque,
scratched,
tabby, tangled,
translucent,
ribbed,
rumpled,
shaded,
slosh,
speckled,
swirly,
puckered,
scrambled,
knotted,
pearlescent,
deformed,
disorganized,
gauzy, glabrous,
interlaced,
mottled,
pitted, plain,
rhythmic,
ruled,
misty,
obscure,
patchy, patterned,
potholed,
ruffled,
sunny,
notched,
pied, pile, pimpled,
porous,
lumpy,
misshapen,
craggy, creased,
entanglement,
gathered,
grainy,
indistinct,
knobbly,
clear, cleft,
coot, cobweb, coiled,
definite,
disordered,
freckled,
garbled,
gouged,
indiscernible,
broken,
crotchet, crows-foot,
dozzling,
hazy, holey, honeycombed,
kinky, knitted,
nonuniform,
overlopping,
fuzzy,
bristly,
fine, flat, flecked, fleecy, flowing,
fragmented,
gossamer,
harmonious,
indeterminate,
level, light, limpid,
furrowed,
filmy,
bleary,
choppy, circular,
cross, crosshatched,
dishevelled,
filigree,
fractured,
brindled,
cracked, cragged,
curly, cyclical, dappled,
fibrous,
foamy,
bleached,
coarse, coarse-groined,
corrugated,
disfigured,
bright,
chequered,
dull, dusky, dusty, elastic, enamel,
fluting,
gritty,
coalesced,
corkscrew,
bespeckle,
bore, braided,
bushy, channelled,
contorted,
dim, discontinuous,
downy,
bent, bespattered,
bootlace,
clotted, cloudy, coagulated,
complex,
dotted,
bearded,
twisted,
uneven,
uniform
unvarying,
wavy, weave, web,
woven, wreath,
wrinkled,
229
TABLE 4
The List Containing 141 Words, Used in the Pilot Study
asymmetrical,
banded,
barred,
spattered,
blemished,
clotted, cloudy, coarse, cobweb, coilded,
complex,
crinkled,
crystalline,
crosshatched,
dotted,
entwine,
fretted,
frilly,
harmonious,
knitted,
facetted,
frothy,
fibrous,
furrowed,
knotty,
lacelike,
oriented,
pockmarked,
lattice,
rippled,
scratched,
simple,
sinewy,
sprinkled,
stained,
stratified,
waffled,
meshed,
speckled,
stringy,
winding,
messy,
regular,
striped,
mottled,
asymmetrical
(l),
banded (2), blemished
(8), cobwebbed
cracked (14), crinkled
discontinous
(15), crosshatched
(20), disordered
flecked (27), flowing
gouged
(9), coiled (lo),
spiralled,
studded,
(40), interlaced
(16), crows-feet
(29), freckled
(36), harmonius
(41), intertwined
(53), netlike
polka-dotted
rhythmic
simple
(60), porous
(66), ribbed
(73), smeared
spotted
(80), sprinkled
twisted
(87), uniform
winding
(54), nonuniform
(94), wizened
(61), potholed
(62), random
(67), ridged (68), rumpled
(74), smooth
(81), stained
(82), stratified
(88), veined (89), waffled
(63), regular
(83), striated
(90), webbed
(95), woven (96), wrinkled
(13),
(25), fine (26),
(39),
(45),
(51), messy (52),
(57), pitted (58), pleated
(64), repetitive
(71), scrambled
(84), studded
(59),
(65),
(72),
(77), speckled (78), spiralled
(91), well-ordered
(97), zigzagged
uniform,
zigzag
(32), gauzy (33),
(44), knitted
(50), meshed
(56), periodic
(76), spattered
tangled,
(24), fibrous
(31), furrowed
(69), scaly (70), scattered
(75), smudged
scrambled,
(18), cyclical (19),
(43), iumbled
(49), matted
ribbed,
Data from 40 Subiects
(37), holey (38), honeycombed
(55), perforated
netlike,
pleoted,
spotted,
(12), corrugated
(23), facetted
(42), irregular
lacelike (46), latticed (47), lined (48), marbled
mottled
spongy,
woven, wrinkled,
(17), crystalline
(30), frilly
muddy,
pitted,
rhythmic,
swirly,
kinky,
(5), bubbly (6), bumpy (7),
complex (1 l), corkscrewed
(21), dotted (22), entwined
(28), fractured
(34), grid (35), grooved
indefinite
(3), blotchy (4), braided
grooved,
scaly, scarred,
TABLE 5
The Final list Containing 98 Words, Used in Gathering Grouping
chequred
freckled,
jumbled,
pimpled,
repetitive,
wiry, wizened,
gritty,
irregular,
periodic,
disordered,
fragmented,
grid, grille,
sandy, scalloped,
smudged,
striated,
well, whirly,
random,
rumpled,
smooth,
streaked,
wavy, webbed,
matted,
powdery,
bumpy, chequred,
cracked, creased,
fractured,
intertwined,
pebbly, perforated,
rocky, ruled,
smeared,
corrugated,
gravelly,
interlaced,
lined, marbled,
bubbly,
curly, cyclical, dense, discontinous,
granular,
indefinite,
potholed,
rough,
corkscrew,
fine, flecked, flowing,
gauzy, gouged,
patchy, patterned,
polka, porous,
ridged,
veined,
filigree,
holey, honeycombed,
nonuniform,
riddled,
crows, crumpled,
blotchy, braided,
(85), swirly
(79),
(86),
(92), whirly
(93),
(98)
resulting in groups of similar items. Subjects were asked to visualize the surface corresponding to each word and to group words together if their visualized surfaces appeared
similar. Subjects were given no explicit criteria for judging similarity and could create any
number of groups and any number of items per group. Once the subjects had completed
their initial groupings, they described each group and explained why all the items in the
group were similar. The experimenter recorded the groupings and the accompanying
descriptions.
BHUSHAN, RAO, AND LOHSE
RESULTS
Hierarchical Cluster Analysis
Hierarchical cluster analysis organizes a set of entities into homogeneous units by representing the objects of interest as the leaves of a tree, or hierarchy. The nonterminal nodes
of the tree are clusters. At each stage, the algorithm builds a tree by successively joining
the most similar pair of items into a new cluster (where items may be individual objects or
clusters). The root of the tree is a single cluster that contains all items.
In order to identify groups or clusters of items in the subjects’ sortings, a matrix of similarities was constructed by counting the number of times each pair of words was grouped
together in the subjects’ sorts. For example, words #4 and #5 appeared in the same initial
groupings for 26 of the 40 subjects; therefore, the corresponding entry in the matrix is 26.
The entries in the matrix ranged from 0, when two words never appeared together, to 40,
when the words appeared together in every subjects’ clustering.
Joining algorithms determine the method used to update the similarity matrix at each
stage of the clustering process. We used the most commonly applied joining algorithm,
called complete linkage, which incorporates the Johnson (1967) “max” method. This
method uses the maximum pairwise distance between points of two clusters to compute
between-cluster distances.
We performed the cluster analyses using Release 6.07 of the SAS system running on an
HP model 730. The resulting tree had 11 primary classes or clusters of words. These 11
classes are shown in Figure 1.
Multidimensional Scaling
Multidimensional scaling (MDS) helped identify properties that distinguish each cluster in
the complete linkage diagram, as shown in Figure 1. MDS estimates the coordinates of a
set of objects in a specified n-dimensional space from proximity data measuring the distances between pairs of objects. The MDS technique assigns coordinates to the objects
such that similar objects are close together and dissimilar objects are far apart. Thus we get
a representation of the structure of the similarity data in a spatial form rather than a hierarchical form.
MDS was conducted using Release 6.08 of the SAS system running on a VAX model
6000-410. SAS used nonlinear least squares to estimate dimension coefficients that reflect
unweighted Euclidean distance between pairs of objects. This technique expresses the
structure of the data spatially rather than hierarchically. MDS reports goodness-of-fit using
stress formulas 1 and 2; stress is the square root of the error sum of squares (Kruskal &
Wish, 1978). For a two dimensional solution, the stress in the MDS fit is 0.41, which is
unacceptable. Hence we sought a solution in a higher, three dimensional space, the result
of which is shown in Figure 2. Two dimensional projections of the 3-d space in Figure 2
are shown in Figures 3(a)-(c).4
The 11 groups idenitified through cluster analysis remain basically intact. Tbe stress for
the 3D solution was 0.18 (the lower the stress, the better is the tit).5
THETEXTURE
LEXICON
231
Figure 1. The result of performing hierorchicol cluster onolysis.
Figure 2. The result of performing multidimensional scaling using three dimensions.
Each word is represented by a colored blob with a number next to it. This number is an index to
be used in conjunction with Table 5.
BHUSHAN, l?AO, AND LOHSE
Figure 3a.
The result of performing multidimensional
scaling
using three dimensions. The projection into the x-y plane is shown.
WIZENED
WRINKLQ
CRINKI Fl7
=ACTURED
RUMPLED
MATTED
FlSROUS w
-
=-
_a+:!!!!!!
SPIRALLEO
CROSSHATCHED
umcE
CYCUCAl
FLmM
11111
FACETTED
RHYlliYIC
CRYSTALLINE
REPETITIVE
PERIOMC.
WELL-ORDERED
liAR?dOMOUS
REQULAR
-I-
SMJOTH
UNIFORM
FINE
SIMPLE
Figure 3b.
The result of performing multidimensional scaling
using three dimensions. The projection into the x-z plane is shown.
233
:I
-11
Y
Figure 3~. The result of performing multidimensional scaling
using three dimensions. The projection into the y-z plane is shown.
This means that three dimensions account for 82% of the variability in the subject data.
This shows a reasonable fit, considering the variability in the subjects and the large number
of words used. Thus, the MDS solution lends further credibility to the groupings. (As a
comparison, in an earlier on perception of texture pictures (Rao & Lohse, 1993c), we
obtained a stress of 0.12 for a three dimensional solution).
DISCUSSION: INTERPRETATION
OF THE DIMENSIONS
Eleven major classes emerged from hierarchical clustering. We now describe the characteristics of these classes, as this will aid us in interpreting the dimensions in the MDS plots.
The clusters have been color coded in order to aid visual comparison.6
Cluster I, encoded in dark green, consists of the words pleated, corrugated, ribbed,
grooved, ridged,furrowed, lined, striated, strat$ed, and zigzag. These words describe linearly oriented textures, where the texture has an orientation along a straight line. In other
words, this cluster exhibits strong directionality along one dimension. Since the words
pleated and zigzag appear in this group, it implies that the entire texture need not have a
single orientation: local variations are permitted as long as they are linear.
234
BHUSHAN, RAO, AND LOHSE
In contrast to cluster I is cluster III, encoded in light blue. Cluster III consists of the
wordsflowing, whirly, swirly, winding, corkscrew, spiralled, coiled and twisted. The common characteristic of these words is that they share the property of having circular orientation, in the sense that the underlying texture exhibits curves that are roughly circular, for
example, spiral or coiled.
Cluster II, encoded in light yellow, consists of the words matted, fibrous, knitted,
woven, meshed, netlike, cross-hatched, chequered, grid, honeycombed, and wafJZed.These
words correspond to textures that are structured in a weave-like manner. The structure of
the texture is derived from the fact that the constituent elements may be considered to have
two directions (let’s say horizontal and vertical), which have been combined systematically
to form a weave. In other words, the weave may be considered ‘orthogonal’.
The largest cluster, number IV, is encoded through yellow. It consists of the wordsfacetted, crystalline, lattice, regular, repetitive, periodic, rhythm harmonious, well-ordered,
cyclical, simple, uniform, fine, and smooth. All these words refer to well-ordered, repetitive textures. The specific nature of the repetition is not specified here as it was in Cluster
III.
Cluster V, encoded in orange, consists of the words complex, messy, random, disordered, jumbled, scrambled, discontinuous, indefinite, asymmetrical, nonuniform, and
irregular. These words deal with random, disordered textures. There is neither a structuring element nor any kind of dominant orientation.
Cluster VI, encoded in red, consists of the words spattered, sprinkled, freckled, speckled, flecked, spotted, polka-dot, and dotted. These words also refer to random textures, but
the nature of the randomness derives from an arbitrary placement of primitives over a
plane. The primitives are small and could be either circular (as in spotted or dotted) or
formed of elongated blobs (as in freckled or flecked). These have sharp edges on the pattern-elements as opposed to Cluster VII which has soft or indistinct edges on the pattemelements.
Cluster VII, encoded in deep blue, consists of the words gauzy, cobweb, webbed, interlaced, entwine, intertwined, braided, frilly, and lacelike. These words also refer to structured textures exhibiting a weave. The words in this cluster are quite similar to that in
cluster II. Thus we would expect clusters II and VII to appear in close proximity in the
MDS plots of Figure 3, which is indeed the case. The difference between the clusters is
probably due to the fact that cluster VII permits a certain amount of variation or randomness in the texture, for example, as in gauzy or cobweb. This could be characterized as
‘nonorthogonal’.
The pink cluster, number VIII, contains mottled, blemished, blotchy, smeared, smudged
and stained. These words refer to random disfigurement of a surface. There is a definite
negative aesthetic connotation associated with these words.
Cluster IX, shown in light green, consists of wizened, crows-feet, rumpled, wrinkled,
crinkled, cracked, and fractured. These refer to textures that show random linear orientation. That is, there is a local orientation at each point in the texture, which is largely
along some preferred mean direction. Random variations about this mean direction are per-
THE TEXTURE LEXICON
235
mitted. This description captures the essence of words in this cluster such as wrinkled and
cracked.
The smallest cluster, number X, is depicted in white and contains the words marbled,
veined, and scaly. Since the size of this cluster is small, it is difficult to attach a definite
interpretation to it. However, these terms are suggestive of textures characterized by noise
functions as discussed by Perlin (1985).
The last cluster, number XI, encoded through magenta, consists of the words bubbly,
bumpy, studded, porous, potholed, pitted, holey, pe$orated, and gouged. These words
depict random three dimensional imperfections on a surface. In other words, these surfaces have raised or depressed features, which are roundish rather than linear in shape.
Identification of Dimensions
From the MDS plots, we offer the following interpretation for what the axes represent. The
X axis represents repetitive versus nonrepetitive. Words to the extreme right on the X axis
correspond to clusters II, IV, and VII which represent repetitive, well-ordered textures.
Words to the extreme left on this axis arise from clusters VIII, V and VI, all of which represent nonrepetitive, disordered textures.
The Y axis represents the nature of orientation. Towards the negative end of the Y axis
we have cluster I containing linearly oriented textures, whereas towards the positive end
we have cluster III containing circularly oriented textures. The clusters in the middle of this
range could have either no orientation (e.g., cluster VI) or multiple linear orientation (e.g.,
clusters IV and II).
The Z axis represents the complexity of the surface that corresponds to the word. Words
such as simple, fine, smooth and uniform appear at the negative end of the Z axis, whereas
words such as crows-feet, lacelike and frilly appear at the positive end of the Z axis. The
positive end of the Z axis presents words that represent complexity whereas the negative
end consists of words that represent simplicity.
The multidimensional scaling (MDS) technique we employ here is particularly well
suited to the analysis of this kind of data as its strength is looking for implicit dimensions
along which words are categorized as well as the relationships between these dimensions.7
The categories that emerge, although in no sense regarded by us as ‘basic’ or ‘essential’ in
a psychological building-block sense, are psychologically significant and suggest what
may well be generic texture concepts. This does not mean that each subject has some
generic image in mind that corresponds to each ‘basic’ category. Certainly one person’s
geometry may be more complex or one’s experience greater.* Though MDS yields the
number of the most important dimensions, the meaning of these dimensions is usually hidden. We must point out that our interpretation of these dimensions is subjective. We tried
to decrease this subjectivity by having several people perform this interpretation separately
and combining their recommendations.
As a pertinent contrast, Rao and Lohse (1993a) conducted a similar experiment to
understand the categorization and dimensions of texture images. The fundamental difference between the former study and the current study is that the former study used texture
images, while the current study used only texture words. The study with images used a
236
BHUSHAN, RAO, AND LOHSE
smaller number of items, 56 images, compared with the 98 words used in the current study.
A three dimensional MDS solution accounted for the following variabilities in the subjects’
data: 88% in the image study and 82% in the word study. This means that the variability in
the categorization of images is less than that of words. This is probably due to the fact that
subjects were sorting specific texture images in the first study, whereas they were sorting
words, subject to their own understanding of the words in the current study. This may also
account for the fact that the three dimensions identified in Rao and Lohse (1993a) do not
correspond exactly to the three dimensions identified in the present paper.
Finally, we compare our findings with the characterization of tactual dimensions discussed earlier. Yoshida (1968) identified the dimensions of touch to be metallicness vs.
fibre-likeness. The dimension of fibre-likeness in the tactual domain seems to correspond
with the dimension of orientation identified in this section. The end of the scale represented
by metallicness in the tactual domain is a little harder to compare as it arises due to a combination of the properties smooth and cool. Smoothness would appear to correspond with
the dimension of well-ordered or repetitive. However, since coolness is a thermal property,
it does not map well onto a visual property of an object, especially in the range of temperatures that allow the object to be touched safely.
ANALYZING THE ASSOCIATION
BETWEEN IMAGES AND WORDS
In our earlier study (Rao & Lohse, 1993b; Rao & Lohse, 1993c), we identified the following major categories in texture images (visual space): granular, marble-like, lace-like, random granular, random nongranular, directional and well-ordered. Examples of these
images are presented in Figure 4.
In this paper we have identified the following major categories in texture words (lexical
space): linearly oriented, weave-like, circularly oriented, well-ordered, random dots, random weave, disfigured, random linear, random three dimensional and a miscellaneous category (formed of outliers). In order to understand the interaction between the visual space
and the lexical space, we must determine a mapping between the categories in each space.
Such a mapping determines the strength of association between a category in lexical space
with categories in visual space and vice versa. Thus, one can answer questions like: “Given
a directional texture image, what kinds of words are associated with this image?’ or “Given
weave-like words, what types of texture images are conjured by these words?’
Contingency Tables
We make use of techniques for categorical data analysis involving contingency tables
(Agresti, 1990). As opposed to a continuous variable, a categorical variable is one for
which the measurement scale consists of a set of categories. In our case, the categorical
variables are texture words and texture images, each of which can fall into the above mentioned categories.
Let X and Y denote two categorical variables, X having I levels and Y having J levels.
Thus there are IJ possible combinations of classifications. Let us create a table with I rows
THETEXTURE
LEXICON
(b) marbldike
Figure 4.
(Rao & Lohse,
Examples
1993~;
textures
of images for each of the categories
Rao & Lohse,
1993b).
Three
237
examples
identified
in our first
study
from each category have been chosen.
BHUSHAN, BAO, AND LOHSE
238
and J columns, so that each cell of the table represents one of the IJ possible outcomes.
When each cell contains frequency counts of the outcomes, the table is called a contingency table.
One can define a measure of association in contingency tables, which is based on the
deviation of the observed frequencies from the expected frequencies on the assumption of
independence. Let us use Pearson’s chi-squared statistic
tfo-fJ2
x2= z-r;-
(1)
for the measure of deviation. Heref, are the observed frequencies, and& the expected frequencies. x2 increases with increasing degree of association between the variables X and
Y. Pearson’s coefficient of contingency measures the degree of association, and is defined as
2
c=
(2)
x
i
I+(*
where n is the sum of all the frequencies in the contingency table. The reader is referred to
Agresti (1990) for further details.
Experimental Methods
We used the following method to determine the mapping between the lexical and verbal
domains. Note that there are two mappings to be determined: given a word find images,
and given an image find words. To determine the mapping from images to words, each subject was given a list of 98 texture words (from Table 5), randomized in order. For each
word, the subject was asked to pick out all the textures that matched the word from a set of
56 textures used in Rao and Lohse (1993b; Rao & Lohse, 1993~). The presentation of
images and words were reversed for the mapping from words to images.
A total of twenty subjects were used, with ten for each mapping. Each subject performed only one type of mapping, that is, either words to images or vice versa. The mapping was quantified in the following manner. Since subjects produced a response for each
word and each image, we obtain a 98 x 56 matrix mapping images to words and a 56x98
matrix mapping words to images. This information is collapsed into the categories mentioned above to aid interpretation. This generates a 7 x 11 contingency table mapping
images to words and a 11x7 contingency table mapping words to images.
Let Mlw denote the contingency table mapping images to words. If a subject associated
a word in the jth word category with an image in the ith image category, the matrix element
M&i,j) is incremented by one. Thus, a pooled contingency table Mlw over all the subjects
is created.
Similarly, a pooled contingency table M, mapping words to images is constructed. The
contingency tables are shown in Figures 5(a) and 6(a), respectively. Since the words and
images are not uniformly distributed across the various categories, we scale the raw votes
in the contingency tables. The raw vote in each bin is divided by the total number of possi-
THETEXTURE
LRXICON
DirwhioIul
w&lor&sad
239
66
26
‘0
so
66
I
I6
la
a1
11
26
169
279
3
*x9
(19
63
136
31
2.
21
96
Figure 5a.
The contingency
toble tvtw showing
the words
associated
with each image.
E
".rbl.-lik
0.37
6.61
Figure 5b.
The contingency
table showing
the words
associated
with each image.
The matrix has been scaled to account for the uneven distribution
words
and images
amongst
the various
of
categories.
ble word-image or image-word combinations for that bin. For instance, the category of
‘linearly oriented’ words has 10 members, and the category of ‘directional’ texture images
has 6 members. Thus the raw vote for the bin “linearly oriented-directional” is divided by
60. The scaled contingency tables are shown in Figures 5(b) and 6(b), respectively.
Results
The two highest association weights for each row are highlighted in Figures 5(b) and 6(b),
respectively. One can interpret this as: given a category of texture images (or words),
which categories of texture words (or images) are most strongly associated with this?
One can see that like categories are most strongly associated with each other. For
instance, Figure 6(b) shows that directional images, as in Figure 4(f) are most strongly
associated with linearly oriented or circularly oriented words. Similarly, Figure 5(b) shows
BHUSHAN, RAO, AND LOHSE
240
Figure 60.
Figure 6b.
The contingency
table Mw, showing
The contingency
table showing
the images associated
the images
associated
with aoch WOI
with each word.
The matrix hos been scaled to account for the uneven distrubution
words
and imoges amongst
the various
categories.
of
THETEXTURE
LRXICON
241
that weave-like words and well-ordered words are most strongly associated with wellordered images as in Figure 4(g).
Pearson’s coefficient of contingency C is 0.63 for the table mapping images to words,
and is 0.56 for the table mapping words to images. These indicate a strong association
between texture words and images. It also appears that there is a stronger association going
from images to words than vice versa. In other words, it is easier to specify words related
to a given image, rather than images related to a given word. Though this finding is preliminary, and needs corroboration through additional studies, it suggests guidelines for designing user interfaces to image databases. Initial queries about textural properties may be
issued through words, (e.g. “find me all instances of striped shirts”) but further interaction
should be complemented with example images. The system can guide the user through a
search in texture space by getting visual feedback, for example, by showing images with
varying density of stripes.
To conclude, our results show that the texture categories we have found in the visual
space and verbal space are strongly tied together.
CONNECTIONS TO OTHER WORK ON WORD
AND PICTURE CATEGORIZATION
Researchers in cognitive psychology have been interested in word and picture categorization tasks for a variety of reasons. We have remained silent about these studies largely
because they are orthogonal to our interests in the paper. Nonetheless, one line of research
that bears mention involves differential categorization tasks involving words and pictures
that are designed so as to provide evidence in support of one or other theory about the form
of representation of semantic knowledge in humans. There is an ongoing debate about
whether “the internal code” consists of pictures, or words or both or whether there is a more
abstract semantic code that involves neither (Glaser, 1992; Theios 62 Amrhein, 1989).
In these studies the crucial piece of data is differential response times for words and pictures; in contrast, our interest is in the categories themselves rather than the rate at which
words or pictures are categorized or the implications for the form of knowledge representation. Nevertheless, we believe that the dimensions uncovered may be tested for psychological reality since it allows for the design of an experiment that will predict subjects’
conceptual space based upon the lexical structural space suggested by our interpretation.
Since the goal of our first study is to see what subjects do with a largely unfiltered group
of texture words, we do not distinguish between words that directly describe the texture
itself (e.g., mottled, spotted) and words that are ‘higher-level’ and describe more generally
the kind of texture that a given word suggests (e.g., assymetrical, uniform, etc.) Some of
the studies referred to above have made this distinction between abstract and concrete
words and have measured response times for each. They found that subjects took longer to
process the abstract words. Since we did not measure response time, other studies of this
nature could complement ours. This time difference was in fact procedurally observed in
our study though it does not show up in the results. Subjects had a harder time grouping the
242
BHUSHAN,BAO, AND LOHSE
more abstract words, either consigning them to the category ‘miscellaneous’, or using them
retroactively to label the group of words from which they were drawn.
Although our first study shows that subjects exhibit a linguistic competence in this
domain, we do not attempt a psychological theory of that competence. However, our interpretation of the dimensions produced by statistical analyses of the results of subjects’ performance suggests a semantic explanation of their competence in terms of three basic
principles of semantic organization for texture words.
Finally, the statistical techniques we use rely crucially upon the construction of similarity matrices and thus assumes their ability to play a functional role in the identification and
explanation of categories. We do not justify this use of similarity in our paper; however, we
are sympathetic to the work of Harnad (1987; Harnad, 1992) and that of Medin in particular (Medin, 1987; Medin et al., 1990; Goldstone et al., 1991; Medin, 1993) who discuss
directly the status of similarity as an explanatory construct in categorization and its role in
a psychological theory of category construction and learning.
CONCLUSION
In this paper we have presented the structure and results of two experiments related to
visual texture. The purpose of the first was to determine how subjects categorized texture
words to identify if possible the dimensions along which the organization took place. To
this end, we formed a list of 98 words dealing with surface texture properties. These words
were analyzed by 40 subjects who performed an unsupervised classification. The subjects
arranged the words into as many categories as they pleased, based on similarity criteria that
they decided upon. This grouping data was used to generate a pooled similarity matrix,
which was analyzed through hierarchical clustering and MDS algorithms.
The hierarchical clustering algorithm identified eleven clusters in the data, which possessed the following characteristics: linearly oriented, circularly oriented, weave-like
structured, well-ordered, disordered, random disfigurement, random linear orientation, and
random three dimensional imperfections. The MDS solution for a three dimensional case
had a fit whose stress was 0.18. In other words, three dimensions were capable of accounting for 82% of the variability in the subject data. This is a reasonable fit considering the
large number of words and the variation in subjects’ backgrounds.
We interpreted the axes along the three dimensions to be regular vs. random; linearly
oriented vs. circularly oriented, simple vs. complex. As pointed out earlier, the names
given for the axes in the MDS plots are subjective. The interpretation provided in this paper
was subject to the authors’ intuition. One way of overcoming this limitation of MDS is
through the use a scale-based rating for different texture features.
As Jackendoff has put it, “it is in the nature of experience not to wear its internal structure on its sleeve” (p. 35). Twisting this to our purposes, we might say that the texture
words in the English language appear as a messy bunch that do not indicate their group
affiliation on their sleeve. The design of our experiment, as a free-form sorting task, as
well as the decision to use MDS in analyzing its results, is a reflection of this fact. MDS is
a statistical technique that, relatively free of specific theoretical demands, ‘uncovers’ or
THE TEXTURE LEXICON
243
‘recovers’ the hidden structure that is in the data. Interpretation comes in only at the end,
when the algorithm has done its structuring work. The categories chosen by individual
subjects were the result of their attentive effort to place words into groups via the activity
of imagining the surface of an object that exhibited the property described by the word in
question. These varied quite a bit between subjects in terms of number of categories as
well as what words went into each. Nonetheless, the statistical analysis of the pooled data
yielded three clear dimensions along which words were categorized. One explanation that
is suggested by these results is that subjects are operating with some very general semantic
principles of organization which are not transparent to them.’ In a more speculative vein,
the results of what is a task in the public behavioral domain may well serve to reveal not
merely general organizational principles but also the intricate structure of the English
speakers’ conceptual space for textures. lo But this would require further experimentation.
What the results of our experiment do show quite clearly is that, despite the tremendous
variety in the words, we have to describe textures, there is an underlying structure that
emerges from statistical analysis.
This led us to design a second experiment which had twenty subjects mapping texture
words onto texture images and vice versa. This brings together our earlier work on texture
images and our current study of texture words, and tests whether the dimensions identified
through image categorization are found by subjects to correspond with the analogous
dimensions identified through word categorization. Our results show that there is a strong
association between texture words and texture images. Like categories of texture words are
associated most strongly with like categories of texture images.
The benefits of conducting this research are as follows:
1.
2.
3.
4.
5.
It enables the identification of implicit dimensions that humans may use in understanding texture words.
The understanding of texture words will provide a foundation for the specification of
&face properties. This will be useful in designing natural language interfaces for
computer graphics.
Since the results of our first study, as we have seen, show a strong convergence in
English speakers’ judgments about texture, this suggests that there is impetus for similar work along cross-cultural lines, for texture terms in other languages. It remains an
intriguing open question (a) whether any underlying dimensions would be at all identifiable for other languages; and (b) if they are, whether those dimensions would be at
all similar to the ones we have identified for English.’ ’
We have found through our second study that a mapping can be established between
images and symbols dealing with texture. This will allow for a standardized conversion from images to symbols and vice versa.
Finally, one can check to see if our results from the current study, obtained by statistical analysis of the data, correspond to the actual way in which subjects internally configure their textural conceptual space. If they do correspond, we should be able to
predict, on the basis of the structure of the lexical space obtained through analysis,
what answer subjects would give on a test that asks them to correlate the relationships
of various texture words. ‘*
Acknowledgements
The authors thank student assistants Jemrifer Fleishmann, Amy Hoff and Nirit Simon of
Smith College for their help in collecting and analyzing texture words, and for running
experiments on subjects. We are grateful to P.V. Kamesam at IBM Research for discussions related to contingency tables. Comments from Jill devilliers at Smith College, R.
Bhaskar at IBM Research, and the two anonymous reviewers of this journal improved the
readability and presentation of this paper. We thank Tim Shortell of Smith College for rendering the 3D plot. Nalini Bhushan acknowledges the Smith Picker Fellowship for course
release time and a grant from the Howard Hughes Foundation for undergraduate research
which provided funds for purchasing the required software.
NOTES
2.
3.
4.
5.
In dissenting from this tradition, Dennett (1991) makes use of recent empirical research in color vision to
argue, contrary to the intuitions of many philosophers and researchers in color vision, that color is not a simple property, as Locke had assumed. That is, color is not a homogeneous power or disposition of a specific
arrangement of microscopic textural properties of surfaces of objects. For Dennett (1991). color is a relational, heterogeneous or complex dispositional property of objects.
The recent texture generation work of Witkin and Kass (1991) is also worth noting in this context.
We performed a simple test to check if there was a difference in the means of frequency of word usage for
the two classes of the original 141 words and the 98 words for our experiment. The frequency of word usage
was obtained from a standard source (Francis & Kucera, 1982). By employing the standard statistical test
for the difference of means, we arrived at a value of 0.54 for the standardized variable (z score). Since the
value of z lies inside the range of -2.58 to 2.58 for a two tailed test at a 0.01 level of significance, we accept
the hypothesis Ho that the two populations have the same mean. Thus, we are 99% confident that the means
are not statistically different, and we believe that we are not losing any critical information in paring the
original list of 141 words down to the 98 words we used.
The coordinates of the points obtained initially through MDS were subject to rotation of 85” about the 2 axis
and 75” about the Y axis. This resulted in the configuration shown in Figure 3. In order to guide the rotation,
we placed groups with opposing properties, such as well-ordered vs. random at opposite ends of an axis.
The stress for the 4D solution was 0.14. Since the stress did not decrease significantly, we chose to work
with the 3D solution. For the reader who may not have a feel for these stress values, we note that a stress
between 0.1 and 0.2 is considered a “moderate” fit, a stress between 0.05 to 0.1 is a “good” fit, and the best
fits have stress less than 0.05.
We are happy to send color copies of our paper to those particularly interested in viewing the original colored figures.
The issue of how best to represent similarity relationships-whether
by feature, trees or dimensionally-has
been the subject of much research in cognitive psychology (Tversky, 1977; Ortony, 1979; Barsalou, 1987).
We thank an anonymous reviewer for raising this as a possible misunderstanding
of our position, occasioning the opportunity for clarification.
In our pilot study, after subjects had placed the words into categories, we asked them if they would describe
each category to us as best as they could; many were uncomfortable doing so. One reason seemed to be that
they were striving to have their descriptions indicate not simply the nature of the categories themselves, but
also the differences and relationships between them. Often subjects helped themselves to the description
“unable to describe,” which was an option suggested by us. And yet, the remarkable convergence of their
behavioral decisions suggests, although it by no means proves, that subjects were operating with general
principles of textural organization.
10. This is not to speculate on the actual form in which these concepts are. represented, thta is, as words, pictures
or in some more abstract way. Our goal is a more circumscribed one.
11. We are in the process of conducting such an experiment for Tamil, one of the languages of South India.
12. For instance, folowing in the footsteps of the Rumelhart and Abrahamson (1972) experiment, complete the
following:
Random repetitive: fibrous: ?
Choose one out of the following:
a) smooth; (b) disordered;
(c) harmonious;
(d) banded.
REFERENCES
Agresti, A. (1990). Caregoricul
Data Analysis. New York: Wiley.
Barsalou, L. (1987). The instability of graded structure: Implications for the nature of concepts. In U. Neisser
(Ed.) Concepts and conceptual development (pp. 101-104). New York: Cambridge University Press.
Becket, W., & Badler, N. (1990). Imperfection
Animation, I, 2632.
for realistic image synthesis. J. of Visualization and Computer
Bennett, J. (197 1). Locke, Berkeley, Hume: Central themes. Oxford: Clarendon
Press.
Berk, T., Brownston, L., & Kaufman, A. (1982). A new color naming system for graphics languages.
puter Graphics and Applications, 2(3), 3744.
Berlin, B., & Kay, P. (1969). Basic Color Terms. Berkeley: University of California
IEEE Com-
Press.
Bhushan, N., & Rao, A.R. (1995). Is alligator skin more wrinkled than tree bark? the role of texture in object
description. In Understanding images (pp. 171-196). New York. Springer Verlag.
Brakel, J.V. (1993). The plasticity of categories:
103-135.
The case of colour. British Journal ofPhilosophical
Science, 44,
Dennett, D. (199 1). Consciousness explained. Boston: Little, Brown & Co.
Dumais, S., Fumas, Cl., Landauer, T., Deerwester, S., & Harshman, R. (1988). Using latent semantic analysis to
improve access to textual information. In CHI (pp. 281-285). ACM.
Englert, G., & Schendel, M. (1992). Fractals and formal texture specification. In J. Encarnacao
geometry and computer graphics (pp. 83-102). New York: Springer Verlag.
Farhoosh,
H., & S&rack,
(Ed.), Fractul
G. (1986). Cns-hls mapping using fuzzy sets. CGA, 6(6), 28-35.
Francis, W.N., & Kucera, H. (1982). Frequency analysis ofEnglish usage: lexicon and grammar. Boston: Houghton Mifflin.
Glaser, W.R. (1992). Picture naming. Cognition, 42.61-105.
Goldstone, R.L., Medin, D.L., & Gentner, D. (1991). Relational similarity
in similarity judgements. Cognitive Psychology, 23(2), 222-262.
Green, M., & Sun, H. (1988). A language
and Applications, 52-64.
and system for procedural
Hamad, S. (Ed.) (1987). Categoricalperception:
Press.
of features
modeling and motion. Computer Graphics
The groundwork ofcognifion. New York: Cambridge
Hamad, S. (1992). Connecting object to symbol in modeling cognition.
tionism in context. New York: Springer Verlag.
Jackendoff,
and the nonindependence
University
In A. Clarke & R. Lutz (Eds.), Connec-
R. (1983). Semantics and cognition. New York: MIT Press.
Johnson, S.C. (1967). Hierarchical
clustering
schemes. Psychomettika,
Julesz, B. (1981). Textons, the elements of texture perception
Kruskal, J., & Wish, M. (1978). Multidimensional
32.241-254.
and their interactions.
Nature, 290, 1619-1645.
scaling. Los Angeles: Sage Publications.
Lederman, S. (1982). The perception of texture by touch. In W. Schiff & E. Foulke (Eds.), Tuctaalperception:
source book. New York: Cambridge University Press.
Levkowitz, H., & Herman, G.T. (1993). Glhs: a generalized lightness, hue, and saturation
Graphical Models and Image Processing, 55(4), 27 l-285.
Locke, J. (1975). An essay concerning human understanding.
York: Cambridge University Press.
MacKenzie,
[Edited with an Introduction
A
color model. Cvgip:
by P.H. Nidditch] New
W.S. (1982). Atlas ofigneous rocks and their textures. New York: John Wiley.
Maerz, A., & Paul, M. (1950). A dictionary of color. New York: McGraw Hill.
Medin, D.L. (1987). Concepts and conceptual
structure. American Psychologist,
44( 12), 1469-1481.
Medin, D.L., Goldstone, R.L., & Gentner, D. (1990). Similarity involving attributes and relations: Judgements
similarity and difference are not inverses. Psychological Science, I( 1), 64-69.
of
Medin, D.L., Goldstone, R.L., & Gentner, D. (1993). Respects for similarity. Psychological Review, 100(2), 254
278.
Minsky, M., et al. (1990). Feeling and seeing: Issues in force display. Computer Graphics, 24(2), 235-243. Special Issue on 1990 Symposium on Interactive 3D Graphics.
246
BHUSHAN, BAO, AND LOHSE
Ohno, R. (1980). Visual perception of texture: development of a scale of the perceived surface roughness of building materials. In Environmental Design Research Association (vol. 11, pp. 193-200).
Ortony, A. (1979). Beyond literal similarity. Psychological Review, 86, 161-180.
Perlin, K. (1985). An image synthesizer. Cornpurer Graphics, 3.287-296.
Rao, A.R. (1990). A taxonomyfor texture description and identification. New York: Springer Verlag.
Rao, A., & Lohse, G. (1993a). Identifying high level features of texture perception. CVGIP: Graphical Models
and Image Processing, .55(3), 218-233.
Rao, A., & Lohse, G. (1993b). Towards a texture naming system: identifying relevant dimensions of texture perception. IBM Research Technical Report RC 19140, International Business Machines, Yorktown, New
York.
Rao, A., & Lohse, G. (1993~). Towards a texture naming system: Identifying relevant dimensions of texture. In
IEEE Conference on Visualization (pp. 220-227). IEEE Computer Society Press, Los Alamitos, CA.
Smith, A. (1990). Of primary and secondary qualities. Philosophical Review, XCIX(2). 221-254.
Theios, J., & Amrhein, P. (1989). Theoretical analysis of the cognitive processing of lexical and pictorial stimuli:
Reading, naming, and visual and conceptual comparisons. Psychological Review, 98.5-24.
Tversky, A. (1977). Features of similarity. Psychological Review, 85.327-352.
Voort, V. (1986). Applied Metallography. New York: Van Nostrand Reinhold.
Witkin, A., & Kass, M. (1991). Reaction-diffusion
textures. Computer Graphics, 4,299-308.
Wright, C. (1988). Moral values, projection and secondary qualities. Proceedings of rheAristorelian Society (Supplement 62, pp. l-26).
Yoshida, M. (1968). Dimensions of tactual impressions. Japanese Psychological Research, 10(3), 123-137.